(Department of Electrical Engineering and Informatic, Universitas Negeri Malang, Indonesia)(2) Fadillah Prasojo
(Department of Electrical Engineering and Informatic, Universitas Negeri Malang, Indonesia)(3) * Anik Nur Handayani
(Department of Electrical Engineering and Informatic, Universitas Negeri Malang, Indonesia)(4) Emanuel Hitipeuw
(Department of Electrical Engineering and Informatic, Universitas Negeri Malang, Indonesia)(5) Aji Prasetya Wibawa
(Department of Electrical Engineering and Informatic, Universitas Negeri Malang, Indonesia)(6) Yoeh Wen Liang
(Departemen of Information Science and Engineering, Saga University, Japan)*corresponding author
AbstractClassification complexity is the main challenge in recognizing sign language through the use of computer vision to classify Indonesian Sign Language (SIBI) images automatically. It aims to facilitate communication between deaf or mute and non-deaf individuals, with the potential to increase social inclusion and accessibility for the disabled community. The comparison of algorithm performance in this research is between the neural network algorithm and multi-layer perceptron classification in letter recognition. This research uses two methods, namely a neural network and a multi-layer perceptron, to measure accuracy and precision in letter pattern recognition, which is expected to provide a foundation for the development of better sign language recognition technology in the future. The dataset used consists of 32,850 digital images of SIBI letters converted into alphabetic sign language parameters, which represent active signs. The developed system produces alphabet class labels and probabilities, which can be used as a reference for the development of more sophisticated sign language recognition models. In testing using the neural network method, good discrimination results were obtained with precision, recall and accuracy of around ±81%, while in testing using the multi-layer perceptron method around ±86%, showing the applicative potential of both methods in the context of sign language recognition. Testing of the two normalization methods was carried out four times with comparison of the normalized data, which can provide further insight into the effectiveness and reliability of the normalization technique in improving the performance of sign language recognition systems.
KeywordsImage Readability Approach; Indonesian Sign Language System; SIBI; Neural Network; Multi Layer Perseptron;
|
DOIhttps://doi.org/10.31763/sitech.v4i2.1362 |
Article metrics10.31763/sitech.v4i2.1362 Abstract views : 158 | PDF views : 32 |
Cite |
Full Text Download
|
References
[1] H. Ramchoun, M. A. J. Idrissi, Y. Ghanou, and M. Ettaouil, “Multilayer perceptron: Architecture optimization and training with mixed activation functions,” ACM International Conference Proceeding Series, vol. Part F1294., pp. 1-6, 2017, doi: 10.1145/3090354.3090427.
[2] A. P. S. Lins and T. B. Ludermir, “Hybrid optimization algorithm for the definition of MLP neural network architectures and weights,” Proc. - HIS 2005 Fifth Int. Conf. Hybrid Intell. Syst., vol. 2005, pp. 6–11, 2005, doi: 10.1109/ICHIS.2005.61.
[3] M. F. Naufal and S. F. Kusuma, “Comparison Analysis Of Machine Learning And Deep Learning Algorithms For Image Classification Of Indonesian Language Signing Systems (Sibi),” JTIIK, vol. 10, no. 4, pp. 873–882, 2023, doi: 10.25126/jtiik.2023106828.
[4] A. A. Pratama, E. Rakun, and D. Hardianto, “Human Skeleton Feature Extraction from 2-Dimensional Video of Indonesian Language Sign System SIBI (Sistem Isyarat Bahasa Indonesia) Gestures,” ICCAI, pp. 100–105, 2019, doi: 10.1145/3330482.3330484.
[5] S. Khalid, T. Khalil, and S. Nasreen, “A Survey of Feature Selection and Feature Extraction Techniques in Machine Learning,” Sci. Inf. Conf., pp. 372–378, 2014, doi: 10.1109/SAI.2014.6918213.
[6] S. Grania et al., “Application of the Perceptron Algorithm in Artificial Neural Networks in Division of Departments,” J. Tenologi Inf., vol. 11, No. 2, pp. 25–29, 2015. [Online]. Available at: https://journal.ubm.ac.id/index.php/teknologi-informasi/article/view/353.
[7] S. A. Sindarto, D. E. Ratnwati, and I. Arwani, “Image Classification of the Indonesian Sign Language System (SIBI) using the Convolutional Neural Network Method on Android-based Software,” J. Pengemb. Teknol. Inf. dan Ilmu Komput., vol. 6, no. 5, pp. 2129-2138, 2022.[Online]. Available at: https://j-ptiik.ub.ac.id/index.php/j-ptiik/article/view/10990.
[8] I. Mekongga, R. Gernowo, and A. Sugiharto, “The Prediction of Bandwidth On Need Computer Network Through Artificial Neural Network Method of Backpropagation,” J. Sist. Inf. Bisnis, vol. 2, no. 2, pp. 98–107, 2012, doi: 10.21456/vol2iss2pp098-107.
[9] N. Thiracitta and H. Gunawan, “SIBI Sign Language Recognition Using Convolutional Neural SIBI Sign Language Recognition Using Convolutional Neural Network Combined with Transfer Learning and non-trainable Network Combined with Transfer Learning and non-trainable Parameter,” Procedia Comput. Sci., vol. 179, no. 2019, pp. 72–80, 2021, doi: 10.1016/j.procs.2020.12.011.
[10] F. Huang, Z. Cao, S. Jiang, C. Zhou, J. Huang, and Z. Guo, “Landslide susceptibility prediction based on a semi-- supervised multiple-layer perceptron model,” Lanslides, vol. 17, pp. 2919–2930, 2020, doi: 10.1007/s10346-020-01473-9.
[11] L. Yusnita, R. Roestam, and R. B. Wahyu, “Implementation of Real-Time Static Hand Gesture Recognition Using Artificial Neural Network,” J. CommIT, vol. 11, no. 2, pp. 85–91, 2017, doi: 10.21512/commit.v11i2.2282.
[12] P. Sibi, S. A. Jones, and P. Siddarth, “Analysis of different activation functions using back propagation neural networks,” JATIT, vol. 47, no. 3, pp. 1264–1268, 2013. [Online]. Available at: https://www.jatit.org/volumes.
[13] S. Savalia and V. Emamian, “Cardiac Arrhythmia Classification by Multi-Layer Perceptron and Convolution Neural Networks,” MDPI, p. 12, 2018, doi: 10.3390/bioengineering5020035.
[14] M. B. S. Bakti and Y. M. Pranoto, “Recognition of Indonesian Sign System Numbers Using the Convolutional Neural Network Method,” Semin. Nas. Inov. Teknol., pp. 11–16, 2019. [Online]. Available at: https://proceeding.unpkediri.ac.id/index.php/inotek/article/view/504.
[15] A. N. Handayani, M. I. Akbar, H. Ar-Rosyid, M. Ilham, R. A. Asmara, and O. Fukuda, “Design of SIBI Sign Language Recognition Using Artificial Neural Network Backpropagation,” 2022 2nd Int. Conf. Intell. Cybern. Technol. Appl., pp. 192–197, 2022, doi: 10.1109/ICICyTA57421.2022.10038205.
[16] A. Mardiana and T. Wahyuni, “Design And Design Of A Vocabulary Recognition Android Application For Deaf Disabilities Using The Sign System Method,” Infotech J., vol. 5, no. 1, pp. 64–68, 2019. [Online]. Available at: https://core.ac.uk/download/pdf/228882667.pdf.
[17] Y. sartika S. Prihandi, Ifan Ranggadara, Indra Dwiasnati, Saruni Sari, “Implementation of Backpropagation Method for Identified Javanese Scripts Implementation of Backpropagation Method for Identified Javanese Scripts,” J. Phys. Conf. Ser., p. 7, 2020, doi: 10.1088/1742-6596/1477/3/032020.
[18] N. Nurmila and A. Sugiharto, “Algoritma Back Propagation Neural Network untuk Pengenalan Pola Karakter Huruf Jawa,” J. Masy. Inform., vol. 1, no. 1, pp. 1–10, 2010, doi: 10.14710/jmasif.1.1.74.
[19] A. Setiawan, A. S. Prabowo, and E. Y. Puspaningrum, “Handwriting Character Recognition Javanese Letters Based on Artificial Neural Network,” Int. J. Comput. Netw. Secur. Inf. Syst., vol. 1, no. 1, pp. 39–42, 2019. [Online]. Available at: https://ijconsist.org/index.php/ijconsist/article/view/12.
[20] A. Nur, H. Wahyu, K. Lindi, and K. Arai, “Recognition of Handwritten Javanese Script using Backpropagation with Zoning Feature Extraction,” J. KEDS, vol. 4, no. 2, pp. 117–127, 2021, doi: 10.17977/um018v4i22021p117-127.
[21] T. Jayalakshmi and A. Santhakumaran, “Statistical Normalization and Back Propagationfor Classification,” Int. J. Comput. Theory Eng., vol. 3, no. 1, pp. 89–93, 2011, doi: 10.7763/ijcte.2011.v3.288.
[22] Z. Zhao, S. Xu, and B. H. Kang, “Investigation and improvement of multi-layer perception neural networks for credit scoring.pdf,” J. Elsevier, pp. 3508-3516, 2014, doi: 10.1016/j.eswa.2014.12.006.
[23] A. S. Eesa and W. K. Arabo, “A Normalization Methods for Backpropagation: A Comparative Study,” Sci. J. Univ. Zakho, vol. 5, no. 4, p. 319, 2017, doi: 10.25271/2017.5.4.381.
[24] L. N. Hayati, A. N. Handayani, W. S. G. Irianto, R. A. Asmara, D. Indra, and M. Fahmi, “Classifying BISINDO Alphabet using Tensorflow Object Detection API,” Ilk. J. Ilm., vol. 15, no. 2, pp. 358–364, 2023, doi: 10.33096/ilkom.v15i2.1692.358-364.
[25] S. Samadianfard, S. Hashemi, K. Kargar, and M. Izadyar, “Wind speed prediction using a hybrid model of the multi-layer perceptron and whale optimization algorithm,” Energy Reports, vol. 6, pp. 1147–1159, 2020, doi: 10.1016/j.egyr.2020.05.001.
[26] S. Moghanian and F. B. Saravi, “GOAMLP : Network Intrusion Detection With Multilayer Perceptron and Grasshopper Optimization Algorithm,” IEEE, vol. 8, pp. 215202-215213, 2020, doi: 10.1109/ACCESS.2020.3040740.
[27] Y. Park and S. Lek, “Artificial Neural Networks Multilayer Perceptron for Ecological Modeling.pdf,” J. Elsevier, vol. 28, pp. 123–140, 2016, doi: 10.1016/B978-0-444-63623-2.00007-4.
[28] G. Bueno, O. D. Id, A. P. Id, J. Ruiz-santaquiteria, and J. Salido, “applied sciences Automated Diatom Classification ( Part A ): Handcrafted Feature Approaches,” MDPI, vol. 7, pp. 1–22, 2017, doi: 10.3390/app7080753.
[29] A. Bogoliubova, “Accuracy Assessment Of Automatic Image Processing For Land Cover Classification Of St . Petersburg Protected Area * 1,” ACTA, vol. 13, pp. 5–22, 2014. [Online]. Available at: https://yadda.icm.edu.pl/baztech/element/bwmeta1.element.baztech-8eff476e-c9b7-43c3-a566-724896da62c1.
[30] M. Sholawati, K. Auliasari, and F. X. Ariwibisono, “Development Of Sibi Alphabetic Sign Language Recognition Application Using Convolutional Neural Network (CNN) Method,” J. JATI, vol. 6, no. 1, pp. 134–144, 2022, doi: 10.36040/jati.v6i1.4507.
[31] A. R. Yessa and M. Hardjianto, “Prediction of Water Use Using Backpropagation Neural Network Method and Particle Swarm Optimization,” Bit-Tech, vol. 2, no. 3, pp. 0148–0157, 2020, doi: 10.32877/bt.v2i3.158.
[32] M. Jahangir, H. Afzal, M. Ahmed, K. Khurshid, and R. Nawaz, “ECO-AMLP: A Decision Support System using an Enhanced Class Outlier with Automatic Multilayer Perceptron for Diabetes Prediction,” p. 19, 2017. [Online]. Available at: https://arxiv.org/abs/1706.07679.
[33] A. P. Markopoulos, S. Georgiopoulos, and D. E. Manolakos, “On the use of back propagation and radial basis function neural networks in surface roughness prediction,” J. Ind. Eng. Int., pp. 389-400, 2016, doi: 10.1007/s40092-016-0146-x.
[34] B. Kijsirikul and S. Sinthupinyo, “Approximate ILP Rules by Backpropagation Neural Network: A Result on Thai Character Recognition,” Lect. Notes Comput. Sci., Apr. pp. 162-173, 2003, doi: 10.1007/3-540-48751-4_16.Refbacks
- There are currently no refbacks.
Copyright (c) 2023 Muhammad Cahyo Bagaskoro, Fadillah Prasojo, Anik Nur Handayani, Emanuel Hitipeuw, Aji Prasetya Wibawa, Yoeh Wen Liang
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
___________________________________________________________
Science in Information Technology Letters
ISSN 2722-4139
Published by Association for Scientific Computing Electrical and Engineering (ASCEE)
W : http://pubs2.ascee.org/index.php/sitech
E : sitech@ascee.org, andri@ascee.org, andri.pranolo.id@ieee.org
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0