Electroencephalogram as a Validation Method in Usability Testing

Guntur Maulana Zamroni; Dinan Yulianto; Bella Saphira; Fadel N. Akhmad; Fakhira A. Zahrah

doi:10.30595/juita.v11i1.16000

Authors

Guntur Maulana Zamroni Universitas Ahmad Dahlan
Dinan Yulianto Universitas Ahmad Dahlan
Bella Saphira Universitas Ahmad Dahlan
Fadel N. Akhmad Universitas Ahmad Dahlan
Fakhira A. Zahrah Universitas Ahmad Dahlan

DOI:

https://doi.org/10.30595/juita.v11i1.16000

Keywords:

E-marketplace, Electroencephalogram, Usability Testing, User Experience Questionnaire, Validation

Abstract

Usability testing is a recommended main testing method to evaluate the usability or ease of use of a software. However, this approach can induce bias. Usability testing with survey method using questionnaire instruments and interview has a risk of generating lack of objectivity from the participants because the facilitator can influence the evaluation results. The evaluation process is not optimal and the biases that arise from usability testing can affect the success or failure of a software in the market. A method for validating usability testing results needs to be studied to ensure whether the test results obtained are valid and unbiased. Electroencephalogram (EEG) is one of the tool that can be used to validate usability testing. This study aims to conduct usability testing of the three most popular e-marketplace applications in Indonesia. From the results it can be seen that usability testing using user-based testing gives a fairly good result in term of accuracy. Usability testing using EEG gives lower results than user-based testing. Nevertheless, we cannot rule out EEG as a usability testing method in the future.

Author Biographies

Guntur Maulana Zamroni, Universitas Ahmad Dahlan

Department of Informatics

Dinan Yulianto, Universitas Ahmad Dahlan

Department of Informatics

References

[1] M. A. Aristyana Dewi, A. N. Hidayanto, M. R. Shihab, and Y. Q. Zhu, “Trust transfer and its effects on the continuance usage of mobile service in B2C E-marketplaces,” Proc. ot 21st Pacific Asia Conf. Inf. Syst. “‘Societal Transform. Through IS/IT’”, PACIS 2017, 2017.

[2] E. Turban, D. King, J. K. Lee, T.-P. Liang, and D. C. Turban, Electronic Commerce - A Managerial and Social Perspective. 2015.

[3] A. L. Kusumatrisna et al., “Statistik E-Commerce 2021,” 2021. [Online]. Available: https://www.bps.go.id/publication/download.html?nrbvfeve=NjY3ODIxZTY3NDIxYWZkMmM4MWM1NzRi&xzmn=aHR0cHM6Ly93d3cuYnBzLmdvLmlkL3B1YmxpY2F0aW9uLzIwMjEvMTIvMTcvNjY3ODIxZTY3NDIxYWZkMmM4MWM1NzRiL3N0YXRpc3Rpay1lLWNvbW1lcmNlLTIwMjEuaHRtbA%3D%3D&twoadfnoarfeauf=MjA.

[4] A. Riskita, “22 Marketplace di Indonesia yang Paling Banyak Dikunjungi,” sirclo.com, 2022. https://store.sirclo.com/blog/marketplace-di-indonesia/.

[5] iPrice Group, “The Map of E-commerce in Indonesia,” iProce Group Sdn Bhd, 2022. https://iprice.co.id/insights/mapofecommerce/en/ (accessed Jun. 07, 2022).

[6] S. Ho, “What consumers think of Indonesia’s top 6 ecommerce sites,” Tech In Asia, 2018. https://www.techinasia.com/talk/consumers-think-ecommerce-players-indonesia (accessed Jun. 07, 2022).

[7] H. Lee and S. Seo, “A Comparison and Analysis of Usability Methods for Web Evaluation : The Relationship Between Typical Usability Test and Bio-Signals Characteristics ( EEG , ECG ),” Proc. 2010 DRS Conf., pp. 7–9, 2010, [Online]. Available: https://dl.designresearchsociety.org/drs-conference-papers/drs2010/researchpapers/73.

[8] M. I. Farouqi, I. Aknuranda, and A. D. Herlambang, “Evaluasi Usability pada Aplikasi Go-Jek Dengan Menggunakan Metode Pengujian Usability,” vol. 2, no. 9, pp. 3110–3117, 2018.

[9] A. Relawati, Y. Primanda, and G. M. Zamroni, “Unmoderated Remote Usability Testing: An Approach during Covid-19 Pandemic,” Int. J. Adv. Comput. Sci. Appl., vol. 13, no. 1, pp. 283–289, 2022, doi: 10.14569/IJACSA.2022.0130135.

[10] J. Hu, “A Method of Usability Testing by Measuring Brain Waves,” IEEE Trans Softw. Eng, pp. 159–164, 2000, [Online]. Available: /citations?view_op=view_citation&continue=/scholar%3Fhl%3Den%26as_sdt%3D0,50%26scilib%3D1&citilm=1&citation_for_view=t-UWhbIAAAAJ:_FxGoFyzp5QC&hl=en&oi=p.

[11] J. Frey, M. Daniel, J. Castet, M. Hachet, and F. Lotte, “Framework for electroencephalography-based evaluation of user experience,” Conf. Hum. Factors Comput. Syst. - Proc., pp. 2283–2294, 2016, doi: 10.1145/2858036.2858525.

[12] John Hopkins Medicine, “What is an EEG?,” The Johns Hopkins University, 2022. https://www.hopkinsmedicine.org/health/treatment-tests-and-therapies/electroencephalogram-eeg (accessed Jun. 03, 2022).

[13] P. Olejniczak, “Neurophysiologic Basis of EEG,” vol. 23, no. 3, pp. 186–189, 2006.

[14] P. A. Abhang, B. W. Gawali, and S. C. Mehrotra, Introduction to EEG- and Speech-Based Emotion Recognition. Elsevier Inc., 2016.

[15] M. A. D. Dourado and E. D. Canedo, “Usability heuristics for mobile applications: A systematic review,” ICEIS 2018 - Proc. 20th Int. Conf. Enterp. Inf. Syst., vol. 2, no. Iceis, pp. 483–494, 2018, doi: 10.5220/0006781404830494.

[16] J. Nielsen, “Usability 101: Introduction to Usability,” Nielsen Norman Group, 2012. https://www.nngroup.com/articles/usability-101-introduction-to-usability/ (accessed Jun. 03, 2022).

[17] Z. Rui and Z. Gu, “A Review of EEG and fMRI Measuring Aesthetic Processing in Visual User Experience Research,” Comput. Intell. Neurosci., vol. 2021, 2021, doi: 10.1155/2021/2070209.

[18] K. Moran, “Usability Testing 101,” Nielsen Norman Group, 2019. https://www.nngroup.com/articles/usability-testing-101/ (accessed Jun. 03, 2022).

[19] Martec Group, “The Future of UX Research: Uncovering Users’ True Emotions with Mixed Methodologies,” Martec Group, 2022. https://martecgroup.com/the-future-of-ux-research/ (accessed Jun. 07, 2022).

[20] C. D. Binnie and P. F. Prior, “Electroencephalography,” J. Neurol. Neurosurg. Psychiatry, pp. 1308–1319, 1994.

[21] B. Farnsworth, “What is EEG (Electroencephalography) and How Does it Work?,” iMotions, 2021. https://imotions.com/blog/what-is-eeg/ (accessed Jun. 09, 2022).

[22] K. Blocka and D. Yetman, “EEG (Electroencephalogram) Overview,” Healthline Media, 2021. https://www.healthline.com/health/eeg (accessed Jun. 09, 2022).

[23] E. Lattari et al., “Corticomuscular coherence behavior in fine motor control of force : a critical review,” no. November, 2010.

[24] N. Jalaudin, M. Kamal, and M. Amin, “Electroencephalography ( EEG ) analysis on human reflection towards relaxation of mind,” vol. 15, no. 2, pp. 185–189, 2019.

[25] L. E. Ismail and W. Karwowski, Applications of EEG indices for the quantification of human cognitive performance: A systematic review and bibliometric analysis, vol. 15, no. 12 December. 2020.

[26] L. A. Palinkas et al., “Purposeful sampling for qualitative data collection and analysis in mixed method implementation research,” Adm Policy Ment Heal., vol. 42, no. 5, pp. 533–544, 2015, doi: 10.1007/s10488-013-0528-y.

[27] Y. Lai, C. Lai, and H. Chiang, “Application of music listening and EEG analysis for sustained attention training,” Int. J. Biol. Biomed. Eng., vol. 9, pp. 133–140, 2015.