(1) Department of Electrical and Informatics, Univesitas Negeri Malang, Jawa Timur, Indonesia. 2) Department of Informatics, Institut Sains dan Teknologi Terpadu Surabaya, Jawa Timur, Indonesia)(2) * Ilham Ari Elbaith Zaeni
(Univesitas Negeri Malang, Indonesia)(3) Aji Prasetya Wibawa
(Univesitas Negeri Malang, Indonesia)(4) Gunawan Gunawan
(Institut Sains dan Teknologi Terpadu Surabaya, Indonesia)(5) Nickolas Hartono
(Institut Sains dan Teknologi Terpadu Surabaya, Indonesia)(6) Yosi Kristian
(Institut Sains dan Teknologi Terpadu Surabaya, Indonesia)*corresponding author
AbstractDetecting deception has significant implications in fields like law enforcement and security. This research aims to develop an effective lie detection system using Electroencephalography (EEG), which measures the brain's electrical activity to capture neural patterns associated with deceptive behavior. Using the Muse II headband, we obtained EEG data across 5 channels from 34 participants aged 16-25, comprising 32 males and 2 females, with backgrounds as high school students, undergraduates, and employees. EEG data collection took place in a suitable environment, characterized by a comfortable and interference-free setting optimized for interviews. The research contribution is the creation of a lie detection dataset and the development of an autoencoder model for feature extraction and a deep neural network for classification. Data preparation involved several pre-processing steps: converting microvolts to volts, filtering with a band-pass filter (3-30Hz), STFT transformation with a 256 data window and 128 overlap, data normalization using z-score, and generating spectrograms from power density spectra below 60Hz. Feature extraction was performed using an autoencoder, followed by classification with a deep neural network. Methods included testing three autoencoder models with varying latent space sizes and two types of classifiers: three new deep neural network models, including LSTM, and six models using pre-trained ResNet50 and EfficientNetV2-S, some with attention layers. Data was split into 75% for training, 10% for validation, and 15% for testing. Results showed that the best model, using autoencoder with latent space size of 64x10x51 and classifier using the pre-trained EfficientNetV2-S, achieved 97% accuracy on the training set, 72% on the validation set, and 71% on the testing set. Testing data resulted in an F1-score of 0.73, accuracy of 0.71, precision of 0.68, and recall of 0.78. The novelty of this research includes the use of a cost-effective EEG reader with minimal electrodes, exploration of single and 3-dimensional autoencoders, and both non-pretrained classifiers (LSTM, 2D convolution, and fully connected layers) and pretrained models incorporating attention layers. KeywordsAutoencoder; Deep Learning; Lie Detection; Electroencephalogram; EEG; Muse II
|
DOIhttps://doi.org/10.31763/ijrcs.v4i3.1497 |
Article metrics10.31763/ijrcs.v4i3.1497 Abstract views : 554 | PDF views : 291 |
Cite |
Full Text Download
|
References
[1] B. Ranganayakulu, S. M. M, “Brain Fingerprint Technology,” International Research Journal of Modernization in Engineering Technology and Science, vol. 4, no. 06, pp. 4537-4543, 2022, https://www.irjmets.com/uploadedfiles/paper/issue_6_june_2022/27189/final/fin_irjmets1656499429.pdf.
[2] S. Deepika, B. Kaviyadharshini, S. Sharmila, S. N. Sangeethaa, and S. Jothimani, “A Study on Brain Fingerprinting Technology,” Journal of Ubiquitous Computing and Communication Technologies, vol. 4, no. 3, pp. 125-137, 2022 https://doi.org/10.36548/jucct.2022.3.001.
[3] A. Farizal, A. D. Wibawa, D. P. Wulandari and Y. Pamungkas, "Investigation of Human Brain Waves (EEG) to Recognize Familiar and Unfamiliar Objects Based on Power Spectral Density Features," 2023 International Seminar on Intelligent Technology and Its Applications (ISITIA), pp. 77-82, 2023, https://doi.org/10.1109/ISITIA59021.2023.10221052.
[4] E. P. F. Bareeda, B. S. S. Mohan, and K. V. A. Muneer, “Lie Detection using Speech Processing Techniques,” Journal of Physics: Conference Series, vol. 1921, no. 1, p. 012028, 2021, https://doi.org/10.1088/1742-6596/1921/1/012028.
[5] A. Pranolo, Y. Mao, A. P. Wibawa, A. B. P. Utama and F. A. Dwiyanto, "Robust LSTM With Tuned-PSO and Bifold-Attention Mechanism for Analyzing Multivariate Time-Series," IEEE Access, vol. 10, pp. 78423-78434, 2022, https://doi.org/10.1109/ACCESS.2022.3193643.
[6] A. P. Wibawa, A. B. P. Utama, H. Elmunsyah, U. Pujianto, F. A. Dwiyanto, and L. Hernandez, “Time-series analysis with smoothed Convolutional Neural Network,” Journal of Big Data, vol. 9, no. 1, p. 44, 2022, https://doi.org/10.1186/s40537-022-00599-y.
[7] I. A. E. Zaeni, U. Pujianto, A. R. Taufani, M. Jiono and P. S. T. Muhammad, "Concentration Level Detection Using EEG Signal on Reading Practice Application," 2019 International Conference on Electrical, Electronics and Information Engineering (ICEEIE), pp. 354-357, 2019, https://doi.org/10.1109/ICEEIE47180.2019.8981453.
[8] M. Yazid et al., "Simple Detection of Epilepsy From EEG Signal Using Local Binary Pattern Transition Histogram," IEEE Access, vol. 9, pp. 150252-150267, 2021, https://doi.org/10.1109/ACCESS.2021.3126065.
[9] A. T. Hermawan, I. A. E. Zaeni, A. P. Wibawa, G. Gunawan, W. H. Hendrawan, Y. Kristian, “A multi representation deep learning approach for epileptic seizure detection,” Journal of Robotics and Control, vol. 5, no. 1, pp. 187-204, 2024, https://doi.org/10.18196/jrc.v5i1.20870.
[10] A. Rochmah, S. Sendari and I. A. E. Zaeni, "Sleepiness Detection For The Driver Using Single Channel EEG With Artificial Neural Network," 2019 International Conference on Advanced Mechatronics, Intelligent Manufacture and Industrial Automation (ICAMIMIA), pp. 80-85, 2019, https://doi.org/10.1109/ICAMIMIA47173.2019.9223371.
[11] M. Žabčíková, Z. Koudelkova, and R. Jašek, “Concealed information detection using EEG for lie recognition by ERP P300 in response to visual stimuli: A review,” WSEAS Transactions on Information Science and Applications, vol. 19, pp. 171-179, 2022, https://publikace.k.utb.cz/handle/10563/1011270?locale-attribute=en.
[12] M. Zabcikova, Z. Koudelkova and R. Jasek, "EEG-based lie detection using ERP P300 in response to known and unknown faces: An overview," 2022 26th International Conference on Circuits, Systems, Communications and Computers (CSCC), pp. 11-15, 2022, https://doi.org/10.1109/CSCC55931.2022.00011.
[13] I. J. Mohammed, L. E. George, “A Survey for Lie Detection Methodology Using EEG Signal Processing,” Journal of Al-Qadisiyah for computer science and mathematics, vol. 14, no. 1, pp. 42-54, 2022, https://doi.org/10.29304/jqcm.2022.14.1.903.
[14] A. I. Simbolon, A. Turnip, J. Hutahaean, Y. Siagian and N. Irawati, "An experiment of lie detection based EEG-P300 classified by SVM algorithm," 2015 International Conference on Automation, Cognitive Science, Optics, Micro Electro-Mechanical System, and Information Technology (ICACOMIT), pp. 68-71, 2015, https://doi.org/10.1109/ICACOMIT.2015.7440177.
[15] I. Nwogu, M. Frank, and V. Govindaraju, “An automated process for deceit detection,” Biometric Technology for Human Identification VII, vol. 7667, pp. 261-270, 2010, https://doi.org/10.1117/12.851407.
[16] N. Baghel, D. Singh, M. K. Dutta, R. Burget and V. Myska, "Truth Identification from EEG Signal by using Convolution neural network: Lie Detection," 2020 43rd International Conference on Telecommunications and Signal Processing (TSP), pp. 550-553, 2020, https://doi.org/10.1109/TSP49548.2020.9163497.
[17] P. Nejedly et al., “Utilization of temporal autoencoder for semi-supervised intracranial EEG clustering and classification,” Scientific Reports, vol. 13, no. 1, p. 744, 2023, https://doi.org/10.1038/s41598-023-27978-6.
[18] Q. Li, Y. Liu, Y. Shang, Q. Zhang, and F. Yan, “Deep Sparse Autoencoder and Recursive Neural Network for EEG Emotion Recognition,” Entropy, vol. 24, no. 9, p. 1187, 2022, https://doi.org/10.3390/e24091187.
[19] P. Sirpal, R. Damseh, K. Peng, D. K. Nguyen, and F. Lesage, “Multimodal Autoencoder Predicts fNIRS Resting State From EEG Signals,” Neuroinformatics, vol. 20, p. 537-558, 2022, https://doi.org/10.1007/s12021-021-09538-3.
[20] Y. Wang, S. Qiu, D. Li, C. Du, B. -L. Lu and H. He, "Multi-Modal Domain Adaptation Variational Autoencoder for EEG-Based Emotion Recognition," IEEE/CAA Journal of Automatica Sinica, vol. 9, no. 9, pp. 1612-1626, 2022, https://doi.org/10.1109/JAS.2022.105515.
[21] X. Li et al., “Latent factor decoding of multi-channel EEG for emotion recognition through autoencoder-like neural networks,” Frontiers in Neuroscience, vol. 14, p. 87, 2020, https://doi.org/10.3389/fnins.2020.00087.
[22] X. Huang et al., “A Novel Epilepsy Detection Method Based on Feature Extraction by Deep Autoencoder on EEG Signal,” International Journal of Environmental Research Public Health, vol. 19, no. 22, p. 15110, 2022, https://doi.org/10.3390/ijerph192215110.
[23] G. H. Khan, N. A. Khan, M. A. Bin Altaf, and Q. Abbasi, “A Shallow Autoencoder Framework for Epileptic Seizure Detection in EEG Signals,” Sensors, vol. 23, no. 8, p. 4112, 2023, https://doi.org/10.3390/s23084112.
[24] R. Ramos-Aguilar, J. A. Olvera-López, I. Olmos-Pineda, and S. Sánchez-Urrieta, “Feature extraction from EEG spectrograms for epileptic seizure detection,” Pattern Recognition Letters, vol. 133, pp. 202–209, 2020, https://doi.org/10.1016/j.patrec.2020.03.006.
[25] B. Mandhouj, M. A. Cherni, and M. Sayadi, “An automated classification of EEG signals based on spectrogram and CNN for epilepsy diagnosis,” Analog Integrated Circuits and Signal Processing, vol. 108, pp. 101-110, 2021, https://doi.org/10.1007/s10470-021-01805-2.
[26] J. G. Proudfoot, R. Boyle, and R. M. Schuetzler, “Man vs. machine: Investigating the effects of adversarial system use on end-user behavior in automated deception detection interviews,” Decision Support Systems, vol. 85, pp. 23-33, 2016, https://doi.org/10.1016/j.dss.2016.02.008.
[27] M. C. Morris and J. Z. Morris, “The importance of virtue ethics in the IRB,” Research Ethics, vol. 12, no. 4, pp. 201-216, 2016, https://doi.org/10.1177/1747016116656023.
[28] D. R. Edla, S. Dodia, A. Bablani, and V. Kuppili, “An Efficient Deep Learning Paradigm for Deceit Identification Test on EEG Signals,” ACM Transactions on Management Information Systems, vol. 12, no. 3, pp. 1-20, 2021, https://doi.org/10.1145/3458791.
[29] S. Niewiadomski, “Filter Handbook A Practical Design Guide,” First. Oxford: Heinemann Newnes, 1989, https://gctjaipur.wordpress.com/wp-content/uploads/2015/08/filter-handbook-a-practical-design-guide-s-niewiadomski.pdf.
[30] B. Blankertz, R. Tomioka, S. Lemm, M. Kawanabe and K. -r. Muller, "Optimizing Spatial filters for Robust EEG Single-Trial Analysis," IEEE Signal Processing Magazine, vol. 25, no. 1, pp. 41-56, 2008, https://doi.org/10.1109/MSP.2008.4408441.
[31] L. J. Gonçales, K. Farias, L. Kupssinskü, and M. Segalotto, “The effects of applying filters on EEG signals for classifying developers’ code comprehension,” Journal of Applied Research and Technology, vol. 19, no. 6, pp. 584-602, 2021, https://doi.org/10.22201/icat.24486736e.2021.19.6.1299.
[32] A. Kawala-Sterniuk et al., “Comparison of smoothing filters in analysis of eeg data for the medical diagnostics purposes,” Sensors, vol. 20, no. 3, p. 807, 2020, https://doi.org/10.3390/s20030807.
[33] J. S. Kumar and P. Bhuvaneswari, “Analysis of Electroencephalography (EEG) Signals and Its Categorization–A Study,” Procedia Engeering, vol. 38, pp. 2525-2536, 2012, https://doi.org/10.1016/j.proeng.2012.06.298.
[34] A. Zabidi, W. Mansor, Y. K. Lee and C. W. N. F. Che Wan Fadzal, "Short-time Fourier Transform analysis of EEG signal generated during imagined writing," 2012 International Conference on System Engineering and Technology (ICSET), pp. 1-4, 2012, https://doi.org/10.1109/ICSEngT.2012.6339284.
[35] P. Peng, Y. Song, L. Yang, and H. Wei, “Seizure prediction in EEG signals using STFT and domain adaptation,” Frontiers in Neuroscience, vol. 15, p. 825434, 2022, https://doi.org/10.3389/fnins.2021.825434.
[36] B. Boashash, “Time-frequency signal analysis and processing: a comprehensive reference,” Academic press, 2015, https://www.sciencedirect.com/book/9780123984999/time-frequency-signal-analysis-and-processing#book-description.
[37] H. Henderi, T. Wahyuningsih, E. Rahwanto, “Comparison of Min-Max normalization and Z-Score Normalization in the K-nearest neighbor (kNN) Algorithm to Test the Accuracy of Types of Breast Cancer,” International Journal of Informatics and Information Systems, vol. 4, no. 1, pp. 13-20, 2021, https://doi.org/10.47738/ijiis.v4i1.73.
[38] I. Aliyu and C. G. Lim, “Selection of optimal wavelet features for epileptic EEG signal classification with LSTM,” Neural Computing and Applications, vol. 35, pp. 1077-1097, 2023, https://doi.org/10.1007/s00521-020-05666-0.
[39] M. Grobbelaar et al., “A Survey on Denoising Techniques of Electroencephalogram Signals Using Wavelet Transform,” Signals, vol. 3, no. 3, pp. 577-586, 2022, https://doi.org/10.3390/signals3030035.
[40] H. Adeli, Z. Zhou, and N. Dadmehr, “Analysis of EEG records in an epileptic patient using wavelet transform,” Journal of Neuroscience Methods, vol. 123, no. 1, pp. 69-87, 2003, https://doi.org/10.1016/S0165-0270(02)00340-0.
[41] A. Géron, “Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems,” O’Reilly Media, 2019, https://www.oreilly.com/library/view/hands-on-machine-learning/9781492032632/.
[42] D. L. Gunawan and Y. Kristian, “A Novel Approach to Flexible Multi-Resolution Image Compression using Deep Learning Based Autoencoders on Overlapping Image Patch,” Procedia Computer Science, vol. 227, pp. 346-354, 2023, https://doi.org/10.1016/j.procs.2023.10.533.
[43] M. Tan and Q. Le, “Efficientnet: Rethinking model scaling for convolutional neural networks,” International conference on machine learning, 2019, https://doi.org/10.48550/arXiv.1905.11946.
[44] M. Tan and Q. V. Le, “Efficientnetv2: Smaller models and faster training,” International conference on machine learning, 2021, https://doi.org/10.48550/arXiv.2104.00298.
[45] I. N. Alam, I. H. Kartowisastro, and P. Wicaksono, “Transfer Learning Technique with EfficientNet for Facial Expression Recognition System,” Revue d’Intelligence Artificielle, vol. 36, no. 4, pp. 543-552, 2022, https://doi.org/10.18280/ria.360405.
[46] Y. Kristian, L. Zaman, M. Tenoyo, and A. Jodhinata, “Advancing Guitar Chord Recognition: A Visual Method Based on Deep Convolutional Neural Networks and Deep Transfer Learning,” ECTI-CIT Transactions, vol. 18, no. 2, pp. 235-249, 2024, https://doi.org/10.37936/ecti-cit.2024182.254624.
[47] K. Goutam, S. Balasubramanian, D. Gera, and R. R. Sarma, “LayerOut: Freezing Layers in Deep Neural Networks,” SN Computer Science, vol. 1, no. 295, 2020, https://doi.org/10.1007/s42979-020-00312-x.
[48] D. Tan, W. Zhong, X. Peng, Q. Wang and V. Mahalec, "Accurate and Fast Deep Evolutionary Networks Structured Representation Through Activating and Freezing Dense Networks," IEEE Transactions on Cognitive and Developmental Systems, vol. 14, no. 1, pp. 102-115, 2022, https://doi.org/10.1109/TCDS.2020.3017100.
[49] M. D. Coster and J. Dambre, “Leveraging Frozen Pretrained Written Language Models for Neural Sign Language Translation,” Information, vol. 13, no. 5, p. 220, 2022, https://doi.org/10.3390/info13050220.
[50] A. V. Nair, K. M. Kumar and J. Mathew, "An Improved Approach for EEG Signal Classification using Autoencoder," 2018 8th International Symposium on Embedded Computing and System Design (ISED), pp. 6-10, 2018, https://doi.org/10.1109/ISED.2018.8704011.
[51] H. Fu, P. Lei, H. Tao, L. Zhao, and J. Yang, “Improved semi-supervised autoencoder for deception detection,” PLoS One, vol. 14, no. 10, p. e0223361, 2019, https://doi.org/10.1371/journal.pone.0223361.
[52] M. Aslan, M. Baykara, and T. B. Alakus, “LieWaves: dataset for lie detection based on EEG signals and wavelets,” Medical & Biological Engineering & Computing, vol. 62, pp. 1571-1588, 2024, https://doi.org/10.1007/s11517-024-03021-2.
[53] T. Nagale and A. Khandare, “Comprehensive Review of Lie Detection in Subject Based Deceit Identification,” Intelligent Computing and Networking, pp. 89-105, 2023, https://doi.org/10.1007/978-981-99-3177-4_7.
[54] A. Horé and D. Ziou, "Image Quality Metrics: PSNR vs. SSIM," 2010 20th International Conference on Pattern Recognition, pp. 2366-2369, 2010, https://doi.org/10.1109/ICPR.2010.579.
[55] U. Sara, M. Akter, and M. S. Uddin, “Image Quality Assessment through FSIM, SSIM, MSE and PSNR—A Comparative Study,” Journal of Computer and Communications, vol. 07, no. 03, pp. 8-18, 2019, https://doi.org/10.4236/jcc.2019.73002.
[56] D. R. I. M. Setiadi, “PSNR vs SSIM: imperceptibility quality assessment for image steganography,” Multimedia Tools and Applications, vol. 80, pp. 8423-8444, 2021, https://doi.org/10.1007/s11042-020-10035-z.
[57] F. H. Raswa et al., "Multi-loss Function in Robust Convolutional Autoencoder for Reconstruction Low-quality Fingerprint Image," 2022 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), pp. 428-431, 2022, https://doi.org/10.23919/APSIPAASC55919.2022.9980345.
[58] L. Muflikhah et al., “Single nucleotide polymorphism based on hypertension potential risk prediction using LSTM with Adam optimizer,” Indonesian Journal of Electrical Engineering and Computer Science, vol. 33, no. 2, pp. 1126-1139, 2024, http://doi.org/10.11591/ijeecs.v33.i2.pp1126-1139.
[59] Y. Wang, Z. Xiao, and G. Cao, “A convolutional neural network method based on Adam optimizer with power-exponential learning rate for bearing fault diagnosis,” Journal of Vibroengineering, vol. 24, no. 4, pp. 666-678, 2022, https://doi.org/10.21595/jve.2022.22271.
[60] A. K. Shahade, K. H. Walse, V. M. Thakare, and M. Atique, “Multi-lingual opinion mining for social media discourses: an approach using deep learning based hybrid fine-tuned smith algorithm with adam optimizer,” International Journal of Information Management Data Insights, vol. 3, no. 2, p. 100182, 2023, https://doi.org/10.1016/j.jjimei.2023.100182.
[61] S. Dasari, B. Poonguzhali and M. Rayudu, "Transfer Learning Approach for Classification of Diabetic Retinopathy using Fine-Tuned ResNet50 Deep Learning Model," 2023 International Conference on Sustainable Communication Networks and Application (ICSCNA), pp. 1361-1367, 2023, https://doi.org/10.1109/ICSCNA58489.2023.10370255.
[62] J. Sun, Y. Yang, G. Xun, and A. Zhang, “Scheduling Hyperparameters to Improve Generalization: From Centralized SGD to Asynchronous SGD,” ACM Transactions on Knowledge Discovery from Data, vol. 17, no. 2, pp. 1-37, 2023, https://doi.org/10.1145/3544782.
[63] U. Ruby, P. Theerthagiri, J. Jacob, Y. Vamshidar, “Binary cross entropy with deep learning technique for Image classification,” International Journal of Advanced Trends in Computer Science and Engineering, vol. 9, no. 4, pp. 5393-5397, 2020, https://doi.org/10.30534/ijatcse/2020/175942020.
[64] C. Guo, X. Chen, Y. Chen, and C. Yu, “Multi-Stage Attentive Network for Motion Deblurring via Binary Cross-Entropy Loss,” Entropy, vol. 24, no. 10, p. 1414, 2022, https://doi.org/10.3390/e24101414.
[65] K. Mohan and A. Seal, “Deception Detection on ‘Bag-of-Lies’: Integration of Multi-modal Data Using Machine Learning Algorithms,” Proceedings of International Conference on Machine Intelligence and Data Science Applications, pp. 445-456, 2021, https://doi.org/10.1007/978-981-33-4087-9_38.
[66] D. Rad, N. Paraschiv, and C. Kiss, “Neural Network Applications in Polygraph Scoring—A Scoping Review,” Information, vol. 14, no. 10, p. 564, 2023, https://doi.org/10.3390/info14100564.
[67] H. Shayeste and B. M. Asl, “Automatic seizure detection based on Gray Level Co-occurrence Matrix of STFT imaged-EEG,” Biomedical Signal Processing and Control, vol. 79, p. 104109, 2022, https://doi.org/10.1016/j.bspc.2022.104109.
[68] Ł. Furman, W. Duch, L. Minati, and K. Tołpa, “Short-time Fourier transform and embedding method for recurrence quantification analysis of EEG time series,” European Physical Journal: Special Topics, vol. 232, pp. 135-149, 2023, https://doi.org/10.1140/epjs/s11734-022-00683-7.
[69] X. Li et al., “Deep Learning Attention Mechanism in Medical Image Analysis: Basics and Beyonds,” International Journal of Network Dynamics and Intelligence, vol. 2, no. 1, pp. 93-116, 2023, https://doi.org/10.53941/ijndi0201006.
[70] Z. Niu, G. Zhong, and H. Yu, “A review on the attention mechanism of deep learning,” Neurocomputing, vol. 452, pp. 48-62, 2021, https://doi.org/10.1016/j.neucom.2021.03.091.
Refbacks
- There are currently no refbacks.
Copyright (c) 2024 Arya Tandy Hermawan, Ilham Ari Elbaith Zaeni, Aji Prasetya Wibawa, Gunawan Gunawan, William Hartanto Hendrawan, Yosi Kristian
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
| About the Journal | Journal Policies | Author | Information |
International Journal of Robotics and Control Systems
e-ISSN: 2775-2658
Website: https://pubs2.ascee.org/index.php/IJRCS
Email: ijrcs@ascee.org
Organized by: Association for Scientific Computing Electronics and Engineering (ASCEE), Peneliti Teknologi Teknik Indonesia, Department of Electrical Engineering, Universitas Ahmad Dahlan and Kuliah Teknik Elektro
Published by: Association for Scientific Computing Electronics and Engineering (ASCEE)
Office: Jalan Janti, Karangjambe 130B, Banguntapan, Bantul, Daerah Istimewa Yogyakarta, Indonesia