Usability Evaluation of a Virtual Reality Simulation System for Earthquake Preparedness Using the System Usability Scale (SUS) Method

Romeo Putra Dirama; Tranggono Tranggono; Rizqi Novita Sari

Romeo Putra Dirama Industrial Engineering, Universitas Pembangunan Nasional “Veteran” Jawa Timur, Indonesia
Tranggono Industrial Engineering, Universitas Pembangunan Nasional “Veteran” Jawa Timur, Indonesia
Rizqi Novita Sari Industrial Engineering, Universitas Pembangunan Nasional “Veteran” Jawa Timur, Indonesia

Keywords: Virtual Reality, Earthquake, Disaster Preparedness, System Usability Scale, Usability

Abstract

This study aims to evaluate the usability level of the Virtual Reality (VR) simulation system for earthquake preparedness training at the Environmental Agency of East Java Province using the System Usability Scale (SUS) method. The testing was conducted with nine respondents who performed two task scenarios in the simulation and subsequently completed the SUS questionnaire. The analysis covered three main aspects of usability: effectiveness, efficiency, and user satisfaction. The results showed an effectiveness level of 100% (“Highly Effective” category), an average efficiency of 0.01758 goals/second (“Very Fast” category), and an average SUS score of 73.88, which falls into the “Acceptable” category with an adjective rating of “Good.” These findings demonstrate that the VR simulation has strategic potential as an effective and efficient interactive training medium for improving the understanding of earthquake evacuation procedures. Nevertheless, improvements are needed in interface consistency and in reducing the initial learning curve to optimize the user experience.

References

Afrihyia, E., Chianumba, E. C., Forkuo, A. Y., Mustapha, A. Y., & Omotayo, O. (2025). Public health emergency preparedness and crisis response: Strengthening national strategies against emerging threats. International Journal of Advanced Multidisciplinary Research and Studies, 5. https://doi.org/10.62225/2583049X.2025.5.3.4382

Ahmadi, M., Ahmadi, A., & Yousefi, S. (2025). Human behavior and decision-making in earthquake emergencies: insights from virtual reality and serious games. International Journal of Construction Management, 25(11), 1254-1272. https://doi.org/10.1080/15623599.2024.2411071

Alene, G. H., Depina, I., Thakur, V., Perkis, A., & Bruland, O. (2024). QuickAware: a virtual reality tool for quick clay landslide hazard awareness. Natural Hazards, 120(2), 1869-1898. https://doi.org/10.1007/s11069-023-06274-6

Alenezi, M., Wardat, S., & Akour, M. (2023). The need of integrating digital education in higher education: Challenges and opportunities. Sustainability, 15(6), 4782. https://doi.org/10.3390/su15064782

Bawa, S. S. (2024). Enhancing usability and user experience in enterprise resource planning implementations. International Journal of Innovative Science and Research Technology, 9(2), 166. https://doi.org/10.5281/zenodo.10653054

Chotib, H. M. (2024). Building Resilience: The Role of Community-Based Disaster Preparedness Training in Empowering Lubuk Village, Indonesia. Indonesian Community Empowerment Journal, 4(2), 240-253. https://doi.org/10.37275/icejournal.v4i2.44

Eden, C. A., Chisom, O. N., & Adeniyi, I. S. (2024). Harnessing technology integration in education: Strategies for enhancing learning outcomes and equity. World Journal of Advanced Engineering Technology and Sciences, 11(2), 001-008. https://doi.org/10.30574/wjaets.2024.11.2.0071

Editya, A. S. (2022). Pengembangan simulasi mitigasi bencana banjir menggunakan teknologi virtual reality. Jurnal Ilmu Komputer dan Desain Komunikasi Visual, X(X), 169–178.

Ernawati, S., Anwariningsih, S. H., & Musslifah, A. R. (2023). Analisis tingkat usabilitas aplikasi kamus istilah psikologi menggunakan metode system usability scale. Tekinfo: Jurnal Ilmiah Teknik Industri dan Informasi, X(X), 147–157.

Faiz, T., Tsun, M. T. K., Mahmud, A. A., & Sim, K. Y. (2024). A scoping review on Hazard recognition and prevention using augmented and virtual reality. Computers, 13(12), 307. https://doi.org/10.3390/computers13120307

Ferreira, M. A., Oliveira, C. S., & Francisco, R. (2025). Tsunami risk mitigation: the role of evacuation routes, preparedness and urban planning. Natural Hazards, 121(6), 6719-6751.

Hakim, L., Wahyuni, H. I., Nasrullah, D., Romadhon, N., Deanofal, A., & Hariyono, A. R. (2025). Implementasi teknologi virtual reality sebagai media edukasi mitigasi bencana pesisir di Sekolah Dasar Muhammadiyah 9 Kenjeran. Journal of Community Development, X(X), 34–43. https://doi.org/10.47134/comdev.v6i1.1382

Hamzah, F., Abdullah, A. H., & Ma, W. (2024). Advancing education through Technology Integration, innovative pedagogies and emerging trends: A systematic literature review. Journal of Advanced Research in Applied Sciences and Engineering Technology, 41(1), 44-63. https://doi.org/10.37934/araset.41.1.4463

Hein, O., Rauschnabel, P., Hassib, M., & Alt, F. (2023, August). Sick in the car, sick in vr? understanding how real-world susceptibility to dizziness, nausea, and eye strain influences vr motion sickness. In IFIP Conference on Human-Computer Interaction (pp. 552-573). Cham: Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-42283-6_30

Indarta, Y., Samala, A. D., & Watrianthos, R. (2022). Metaverse: Tantangan dan peluang dalam pendidikan. JURNAL BASICEDU: Research & Learning in Elementary Education, X(X), 3351–3363. https://doi.org/10.31004/basicedu.v6i3.2615

Kalyani, L. K. (2024). The role of technology in education: Enhancing learning outcomes and 21st century skills. International journal of scientific research in modern science and technology, 3(4), 05-10.

Karkera, S., Devendra, N., Lakhani, B., Manahan, K., & Geisler, J. (2024). A review on modern teaching and learning techniques in medical education. EIKI Journal of Effective Teaching Methods, 2(1). https://doi.org/10.59652/jetm.v2i1.128

Kennedy, G. A., Pedram, S., & Sanzone, S. (2023). Improving safety outcomes through medical error reduction via virtual reality-based clinical skills training. Safety Science, 165, 106200. https://doi.org/10.1016/j.ssci.2023.106200

Kent Cawley, D. B. A., & McEntire, D. A. (2024). Technology in Emergency Management: A Review from Developed Nations with Considerations in a Global Context. Current and Emerging Trends in the Management of International Disasters.

Khan, A., Gupta, S., & Gupta, S. K. (2022). Emerging UAV technology for disaster detection, mitigation, response, and preparedness. Journal of Field Robotics, 39(6), 905-955. https://doi.org/10.1002/rob.22075?urlappend=%3Futm_source%3Dresearchgate.net%26utm_medium%3Darticle

Kholis, A. H., Hidayah, N., Srihono, A., Magalhaes, L., & Priyanti, R. P. (2022). Resilien sity (remaja siap peduli bencana dengan simulasi virtual reality). Jurnal Ilmiah Keperawatan (Scientific Journal of Nursing), X(X), 620–627. https://doi.org/10.33023/jikep.v8i4.1322

Khorram-Manesh, A., Johannessen, M. R., Nielsen, L. R., Carlström, E., Berntzen, L., Sandberg, L., & Sørensen, J. L. (2025). Cultivating disaster preparedness: Scoping review of technology’s contribution to situational awareness and disaster mindset in disaster medicine. Online Journal of Public Health Informatics, 17, e75404. https://doi.org/10.2196/75404

Kusumastuti, R. D., Nurmala, N., Arviansyah, A., & Wibowo, S. S. (2022). Indicators of community preparedness for fast-onset disasters: a systematic literature review and case study. Natural hazards, 110(1), 787-821. https://doi.org/10.1007/s11069-021-04970-9

Maulina, D. (2023). Analisis usability sistem aplikasi Netraku menggunakan metode usability testing. Jurnal Teknik Informatika Unika St. Thomas (JTIUST), X(X), 2657–1501.

Mehra, A., Devi, A., Kumar, A., Rana, S., Kumar, S., Uday, K. V., & Dutt, V. (2025, March). Virtual Reality in Crisis: How Perceived Risk and Warning Accuracy Shape Behavior in Landslide Scenarios. In 2025 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) (pp. 687-694). IEEE. https://doi.org/10.1109/VRW66409.2025.00140

Ramaseri Chandra, A. N., El Jamiy, F., & Reza, H. (2022). A systematic survey on cybersickness in virtual environments. Computers, 11(4), 51. https://doi.org/10.3390/computers11040051

Reale, C., Salwei, M. E., Militello, L. G., Weinger, M. B., Burden, A., Sushereba, C., ... & Anders, S. (2023). Decision-making during high-risk events: a systematic literature review. Journal of Cognitive Engineering and Decision Making, 17(2), 188-212. https://doi.org/10.1177/15553434221147415

Sagala, G., & Cabales, J. C. (2025). Role of Communication in Promoting Environmental Activitism Amongst Hikers in Western Visayas: An Ethnographic Research. Journal of Communication and Management, 4(04), 84-93. https://doi.org/10.58966/JCM2025449

Sigar, T. R. (2025). Analisis dampak penggunaan virtual reality dalam simulasi pelatihan kesiapsiagaan bencana alam berbasis sistem informasi. Journal Scientific of Mandalika (JSM), 6(7), 2809–0543. https://doi.org/10.36312/10.36312/vol6iss7pp1934-1944

Siregar, T. (2025). Integrating GeoGebra in mathematics education: Enhancing pedagogical practices among teachers and lecturers. Yerizon, Integrating GeoGebra in Mathematics Education: Enhancing Pedagogical Practices among Teachers and Lecturers (October 16, 2025). https://doi.org/10.20944/preprints202510.1359.v1?urlappend=%3Futm_source%3Dresearchgate.net%26utm_medium%3Darticle

Sulashvili, N., Pkhakadze, I., Beglaryan, M., Egnatievi, I., Giorgobiani, M., Gabunia, L., ... & Patsia, L. (2024). The scientific discussion of key issue aspects of modern advancing innovation of learning and teaching methods and pedagogical approaches in higher education institutions in medical educational study programs directions globally in general. Scientific Journal „Spectri “, 10(2). https://doi.org/10.52340/spectri.2024.10.01.12

Torres-Agüero, D., Armijo-Rivera, S., Vicencio-Clarke, S., Norte, G., Herrera, E., & Cordero, A. L. (2025). Enhancing Disaster Preparedness and Community Safety Through Virtual Simulation: A Scoping Review in Undergraduate Health Education. Disaster Medicine and Public Health Preparedness, 19, e351. https://doi.org/10.1017/dmp.2025.121

Virgantara Putra, O., Reza Pradhana, F., & Shaleha, F. (2022). Simulasi 3 dimensi rute evakuasi bencana di Gontor Putri Kampus 3 berbasis virtual reality (VR). Dalam Prosiding Seminar Nasional Hasil Penelitian dan Pengabdian kepada Masyarakat 2022 LP2M UST Jogja (hlm. 435–444).

Yazdi, M. (2024). Augmented reality (AR) and virtual reality (VR) in maintenance training. In Advances in Computational Mathematics for Industrial System Reliability and Maintainability (pp. 169-183). Cham: Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-53514-7_10

Yu, W. D., Wang, K. C., & Wu, H. T. (2022). Empirical comparison of learning effectiveness of immersive virtual reality–based safety training for novice and experienced construction workers. Journal of Construction Engineering and Management, 148(9), 04022078. https://doi.org/10.1061/(ASCE)CO.1943-7862.0002337

Yulianti, S. (2024). Analisis usability testing pada website UKM LDK Syahid UIN Syarif Hidayatullah Jakarta menggunakan metode system usability scale (SUS). JATI (Jurnal Mahasiswa Teknik Informatika), 8(4), 8190–8196. https://doi.org/10.36040/jati.v8i4.10715

Zielasko, D., & Riecke, B. E. (2024). Sitting or Standing in VR: About Comfort, Conflicts, and Hazards. IEEE Computer Graphics and Applications, 44(2), 81-88. https://doi.org/10.1109/MCG.2024.3352349

Zubair, M. U., Khan, M. A., Hassan, M. U., Ahmed, K., & Aziz, T. (2024). Enhancing Student Active Engagement in Class through Game-Based Learning: A Case of Civil Engineering Education. Sustainability, 16(14), 6010. https://doi.org/10.3390/su16146010