Mobile Augmented Reality Elements for Museum Hearing Impaired Visitors’ Engagement


  • Esraa Jaffar Baker School of Multimedia Technology and Communication, Universiti Utara Malaysia, Malaysia. The University of Mustansiriyah, Iraq.
  • Juliana Aida Abu Bakar Institute of Creative Humanities, Multimedia and Innovation, Universiti Utara Malaysia, Malaysia
  • Abdul Nasir Zulkifli Institute of Creative Humanities, Multimedia and Innovation, Universiti Utara Malaysia, Malaysia


Augmented Reality, Mobile Augmented Reality, Engagement App, Museum App, Hearing Impaired Application,


There is an increase attention on the provision of the ideal learning environment for museum visitors. This has made researchers to employ the concept of engagement in designing Mobile Augmented Reality application in order to attract museum visitors’ interest and ensure a more effective learning environment. However, most of the Mobile Augmented Reality applications for museum visitors are largely tailored to normal hearing visitors while the hearing-impaired visitors are not supported. This makes hearing-impaired visitors to go through unpalatable experiences and eventually dissatisfy with their visit to the museum. This experience is making learning impossible for hearing-impaired visitors at the museum sites. Therefore, this study explored engagement Mobile Augmented Reality elements that are needed for the design of an efficient museum Mobile Augmented Reality application for hearing impaired visitors’. The study employed both systematic literature review and expert opinion methods. The findings revealed that there are eleven major elements of engagement Mobile Augmented Reality needed for the design of an efficient museum Mobile Augmented Reality app for hearing impaired visitors’. These eleven elements include Aesthetics, Curiosity, Usability, Interaction, Motivation, Satisfaction, Self-Efficacy, Perceived Control, Enjoyment, Focused Attention and Interest. This paper argues that for an efficient and engaged Mobile Augmented Reality app for the hearing-impaired community most especially hearing-impaired visitors to museum sites these eleven elements are critical. This finding will help Mobile Augmented Reality designers and developers on how to design an efficient and engaged Mobile Augmented Reality app for the hearing-impaired community at large and museum hearing-impaired visitors’ specifically.


. D. W. F. Van Krevelen, and R. Poelman, “A survey of augmented reality technologies, applications and limitations”. International Journal of Virtual Reality, 9(2), 1, 2010.

. R. Lindgren, M. Tscholl, S. Wang, and E. Johnson, “Enhancing learning and engagement through embodied interaction within a mixed reality simulation”. Computers & Education, 95, 174-187, 2016.

. M. B. Ibáñez, A. Di Serio, D. Villarán, and C. D. Kloos, “Experimenting with electromagnetism using augmented reality: Impact on flow student experience and educational effectiveness”, Computers & Education, 71, 1-13, 2014.

. A. Di Serio, M. B. Ibáñez, and C. D. Kloos, “Impact of an augmented reality system on students' motivation for a visual art course”. Computers & Education, 68, 586-596, 2013,

. C. Dede, “Immersive interfaces for engagement and learning. science, 323(5910), 66-69, 2009.

. S. Dow, M. Mehta, E. Harmon, B. MacIntyre, and M. Mateas, “Presence and engagement in an interactive drama”. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 1475-1484). ACM, 2007.

. J. Goss, E. K. Kollmann, C. Reich, and S. Iacovelli, “Understanding the Multilingualism and Communication of Museum Visitors who are d/Deaf or Hard of Hearing”. Museums & Social Issues, 10(1), 52-65, 2015.

. I. Kožuh, M. Hintermair, A. Holzinger, Z. Volčič, and M. Debevc, “Enhancing universal access: deaf and hard of hearing people on social networking sites”. Universal Access in the Information Society, 14(4), 537-545, 2015.

. P. Zazove, H. E. Meador, B. D. Reed, and D. W. Gorenflo, “Deaf Persons' English Reading Levels and Associations with Epidemiological, Educational, and Cultural Factors”, Journal of health communication, 18(7), 760-772, 2013.

. M. Kenteris, D. Gavalas, and D. Economou, “Mytilene E-guide: a multiplatform mobile application tourist guide exemplar”, Multimedia Tools and Applications, 54(2), 241-262, 2011.

. H. F. Lin, and C. H. Chen, “Design and application of augmented reality query-answering system in mobile phone information navigation”. Expert Systems With Applications, 42(2), 810-820, 2015.

. N. Linge, K. Booth, and D. Parsons, “Practicalities of developing and deploying a handheld multimedia guide for museum visitors”, Place-Based Spaces for Networked Learning, 160, 2016.

. S. C. Wu, “u-Museum and u-learning: On Development of National Palace Museum’s Mobile Apps. perspective, 9(4), 2016.

. M. K. Othman, N. E. Young, and S. Aman, “Viewing Islamic Art Museum Exhibits on the SmartPhone: Re-examining Visitors’ Experiences”. Journal of Cognitive Sciences & Human Development, 1(1), 102-118, 2015.

. E. Sakkopoulos, M. Paschou, Y. Panagis, D. Kanellopoulos, G. Eftaxias, and A. Tsakalidis, “e-souvenir appification: QoS web based media delivery for museum apps” Electronic Commerce Research, 15(1), 5-24, 2015.

. Y. L. Chang, H. T. Hou, C. Y. Pan, Y. T. Sung, and K. E. Chang, “Apply an Augmented Reality in a Mobile Guidance to Increase Sense of Place for Heritage Places”. Educational Technology & Society, 18(2), 166-178, 2015

. M. Hatala, and R. Wakkary, “Ontology-based user modeling in an augmented audio reality system for museums”. User Modeling and User-Adapted Interaction, 15(3-4), 339-380, 2005.

. M. Pérez-Sanagustín, D. Parra, R. Verdugo, G. García-Galleguillos and M. Nussbaum. Using QR codes to increase user engagement in museum-like spaces. Computers in Human Behavior, 60, 73-85. 2016.

. C. L. Lin, “Augmented Reality Interaction Design and User Experience of Hand Puppet Historical Museum in Taiwan”. International Journal of Humanities and Cultural Studies (IJHCS) ISSN 2356-5926, 3(2), 1044-1057, 2016.

. K. Seppälä, O. Heimo, T. Korkalainen, J. Pääkylä, J. Latvala, S. Helle, and L. Viinikkala, “Examining User Experience in an Augmented Reality Adventure Game: Case”, 2016.

. S.C Loy, F. Zhao and E. Jun, E. The Use of Digital Technologies to Enhance User Experience at Gansu Provincial Museum. Digital Presentation and Preservation of Cultural and Scientific Heritage, (V), 257-263. 2015.

. S. C. Loy, L. Ta, and F. Zhao, “Emergence of a Digital Mobile Museum–Reaching Underserved Audience”. Digital Presentation and Preservation of Cultural and Scientific Heritage, (V), 243-255, 2015.

. I. Rubino, C. Barberis, L. Di Chio, J. Xhembulla, and G. Malnati, “Enhancing a Museum Mobile Application through User Experience Design: A Comparative Analysis”, 2014.

. M. E. Cho, H. H. Choi, and M. J. Kim, “A Usability Study On Museum Installations Emphasizing Interaction Design for User Experience”, Korean Institute of Interior Design Journal, 22(5), 302-310, 2013.

. S. His and H. Fait. RFID enhances visitors' museum experience at the Exploratorium. Communications of the ACM, 48(9), 60-65. 2005

. S. Hsi. The Electronic Guidebook: A study of user experiences using mobile web content in a museum setting. In Wireless and Mobile Technologies in Education, 2002. Proceedings. IEEE International Workshop on (pp. 48-54). IEEE. 2002

. S. His. A study of user experiences mediated by nomadic web content in a museum. Journal of Computer Assisted Learning, 19(3), 308-319. 2003

. A. Ribeiro, and A. R. da Silva, “Survey on cross-platforms and languages for mobile apps”, In Quality of Information and Communications Technology (QUATIC), 2012 Eighth International Conference on the (pp. 255-260). IEEE, 2012.

. G. Kearsley, and B. Shneiderman, “Engagement Theory: A Framework for Technology-Based Teaching and Learning”. Educational technology, 38(5), 20-23, 1998.

. P. Chapman, S. Selvarajah, J. and Webster, “Engagement in multimedia training systems”. In Proceedingsof the Systems Sciences, 1999. HICSS-32. Proceedings of the32nd Annual Hawaii International Conference on (5-8 Jan1999), 9. DOI=, 772808, 1999

. H. L. O'Brien, and E. G. Toms, “The development and evaluation of a survey to measure user engagement”. Journal of the American Society for Information Science and Technology, 61(1), 50-69, 2010

. H. Du, Venkatakrishnan, G. M. Youngblood, A. Ram, and P. Pirolli, “A Group-Based Mobile Application to Increase Adherence in Exercise and Nutrition Programs: A Factorial Design Feasibility Study”. JMIR mHealth and uHealth, 4(1), 2016.

. A. Weston, L. Morrison, L. Yardley, and M. Van Kleek, “Measurements of engagement in mobile behavioural interventions?”, 2015.

. J. Huizenga, W. Admiraal, S. Akkerman, and G. T. Dam, “Mobile game‐based learning in secondary education: engagement, motivation and learning in a mobile city game”. Journal of Computer Assisted Learning, 25(4), 332-344, 2009.

. H. Pavliscsak, J. R. Little, R. K. Poropatich, F. L. McVeigh, J. Tong, J. S. Tillman, and S. J. Fonda, “Assessment of patient engagement with a mobile application among service members in transition”. Journal of the American Medical Informatics Association, 23(1), 110-118, 2016.

. L. Kosinski, C. Baum, M. Sorensen, J. Rosenberg, D. Sales, B. Blumenstein, and M. Haverty, “P-208 Project Sonar: Improvement in Patient Engagement Rates Using a Mobile Application Platform. Inflammatory Bowel Diseases, 22, S72, 2016.

. Y. H. Kim, D. J. Kim, and K. Wachter, “A study of mobile user engagement (MoEN): Engagement motivations, perceived value, satisfaction, and continued engagement intention”, Decision Support Systems, 56, 361-370, 2013.

. S. R. Harper, and S. J. Quaye, “Beyond sameness, with engagement and outcomes for all”. Student engagement in higher education, 1-15, 2009.

. J. A. Fredricks, P. C. Blumenfeld, and A. H. Paris, “School engagement: Potential of the concept, state of the evidence. Review of educational research, 74(1), 59-109, 2004.

. J. Patel, J. Clawson, A. Voida, and K. Lyons, “Mobiphos: A study of user engagement with a mobile collocated–synchronous photo sharing application”. International Journal of Human-Computer Studies, 67(12), 1048-1059, 2009.

. D. S. Carter, “Mobile Application Design to Encourage Civic Engagement” (Doctoral dissertation, Pennsylvania State University), 2014.

. K. Han, P. C. Shih, M. B. Rosson, and J. M. Carroll, “Enhancing community awareness of and participation in local heritage with a mobile application. In Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing (pp. 1144-1155). ACM, 2014.

. M. De Marsico, C. Galdi, M. Nappi, and D. Riccio, “FIRME: face and iris recognition for mobile engagement. Image and Vision Computing, 32(12), 1161-1172, 2014.

. P. Denny, “The effect of virtual achievements on student engagement”. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 763-772). ACM, 2013.

. S. Barello, S. Triberti, G. Graffigna, C. Libreri, S. Serino, J. Hibbard, and G. Riva, “eHealth for Patient Engagement: a systematic review”, Frontiers in psychology, 6 2015.

. T. Jung, M. C. tom Dieck, H. Lee, and N. Chung, “Effects of Virtual Reality and Augmented Reality on Visitor Experiences in Museum”, In Information and Communication Technologies in Tourism 2016 (pp. 621-635). Springer International Publishing, 2016.

. S. Rattanarungrot, M. White, and B. Jackson, “The application of service orientation on a mobile AR platform? a museum scenario”, Digital Heritage (Vol. 1, pp. 329-332). IEEE, 2015

. U. C. Pendit, U. C. and S. B. Zaibon. Enjoyable Informal Learning at Cultural Heritage Site using Mobile Augmented Reality: A Conceptual Model. Journal of Advances in Science and Technology, 2(3), 93-106. 2014.

. Z. Yovcheva, D. Buhalis, and C.Gatzidis, Engineering augmented tourism experiences (pp. 24-35). Springer Berlin Heidelberg, 2013.

. E. Wachelka, Techcooltour: Discover Ancient Empires with Augmented Reality. Available:, 2013.

. B. Seo, K. Kim, and J. I Park,”Augmented Reality-Based On-Site Tour Guide: A Study in Gyeongbokgung”, Computer Vision - ACCV 2010 Workshops, R. Koch and F. Huang, Eds. Springer Berlin Heidelberg, p. 276–285, 2011.

. M. E. C. Santos, A. Chen, T. Y. Taketomi, G. Yamamoto, J. Miyazaki, and H. Kato, “Augmented reality learning experiences:Survey of prototype design and evaluation”. IEEE Transactions on learning technologies, 7(1), 38-56, 2014.

. A. Angelopoulou, D. Economou V. Bouki, A. Psarrou, L. Jin, C. Pitchard and F. Kolyda, “Mobile Augmented Reality for Cultural Heritage”, In 4th International ICST Conference Mobilware, p. 15-22, 2011.

. B. Seo, K. Kim, and J. I Park,”Augmented Reality-Based On-Site Tour Guide: A Study in Gyeongbokgung”, Computer Vision - ACCV 2010 Workshops, R. Koch and F. Huang, Eds. Springer Berlin Heidelberg, p. 276–285, 2011.

. D. Marimon, C. Sarasua, P. Carrasco, R. Álvarez, J. Montesa, T. Adamek, and P. Gascó, P, “MobiAR: tourist experiences through mobile augmented reality”, Telefonica Research and Development, Barcelona, Spain, 2010.

. A. Damala, P. Cubaud, A. Bationo, P. Houlier, and I. Marchal, Bridging the gap between the digital and the physical: design and evaluation of a mobile augmented reality guide for the museum visit. In Proceedings of the 3rd international conference on Digital Interactive Media in Entertainment and Arts (pp. 120-127). ACM, 2008.

. A. Huang and S. Wang “Promote Learner Autonomy and Create a Language Community with Social Apps. From , 2015.

. A. Nachairit, and N. SrisawasdI, “Using Mobile Augmented Reality for Chemistry Learning of Acid-base Titration: Correlation between Motivation and Perception”. 2015.

. M. Boberg, E. Karapanos, J. Holopainen, and A. Lucero, “PLEXQ: Towards a Playful Experiences Questionnaire”, In Proceedings of the 2015 Annual Symposium on Computer-Human Interaction in Play (pp. 381-391). ACM, 2015.

. C. Pribeanu, C. “Extending and Refining Usability Heuristics to Better Address User Centered Design Issues in the Development and Evaluation of Municipal Websites”. Informatica Economica, 18(1), 83, 2014.

. A. Di Serio, M. B. Ibáñez and C. D. Kloos. Impact of an augmented reality system on students' motivation for a visual art course. Computers & Education , 68: 586-596. 2013

. J. Sauro, “Measuring usability with the system usability scale (SUS). Verfügbar unter: http://www. measuringusability. com/sus. Php. 2011.

. J. Hector, and R. Payel, “Analysis of Four Usability Evaluation Methods Applied to Augmented Reality Applications “ 2014.

. S. Nilsson, and B. Johansson, “Fun and usable: augmented reality instructions in a hospital setting”. In Proceedings of the 19th Australasian conference on Computer-Human Interaction: Entertaining user interfaces (pp. 123-130). ACM, 2007.

. A. C. Haugstvedt, “Accessing Cultural Heritage Resources on a Mobile Augmented Reality Platform: A Study on Technology Acceptance”, 2012.

. M. Alqahtani, and H. Mohammad, “Mobile Applications' Impact on Student Performance and Satisfaction”, Turkish Online Journal of Educational Technology-TOJET, 14(4), 102-112, 2012.

. D. Permadi, and A. Rafi, A. “Developing a Conceptual Model of User Engagement for Mobile-Based Augmented Reality Games”. Jurnal Teknologi, 77(29), 2015.

. T. L. Chou, and L. J. Chanlin, “Location-Based Learning through Augmented Reality. Journal of Educational Computing Research, 51(3), 355-368, 2014.

. Y. Y. Mun, and Y. Hwang, “Predicting the use of web-based information systems: self-efficacy, enjoyment, learning goal orientation, and the technology acceptance model. International journal of human-computer studies, 59(4), 431-449, 2003.

. R. Wojciechowski, W. Cellary, “Evaluation of learners’ attitude toward learning in ARIES augmented reality environments”, Computers & Education, 68, 570-585, 2013.

. D. M. Bressler, and A. M. Bodzin, “A mixed methods assessment of students' flow experiences during a mobile augmented reality science game”. Journal of Computer Assisted Learning, 29(6), 505-517, 2013.

. J. Ferrer-Torregrosa, J. Torralba, M. A. Jimenez, S. García, and J. M. Barcia, “ARBOOK: development and assessment of a tool based on augmented reality for anatomy”. Journal of Science Education and Technology, 24(1), 119-124, 2015

. M. K. Othman, H. Petrie, and C. Power, “Engaging visitors in museums with technology: scales for the measurement of visitor and multimedia guide experience”. In IFIP Conference on Human-Computer Interaction (pp. 92-99). Springer Berlin Heidelberg, 2011.

. B. D. Harper, and K. L. Norman, “Improving user satisfaction: The questionnaire for user interaction satisfaction version 5.5”. In Proceedings of the 1st Annual Mid-Atlantic Human Factors Conference (pp. 224-228), 1993.

. V. Gopalan, A. N. Zulkifli, N. F. Faisal, A. A. Mohamed, R. C. Mat, J. Aida, and A. Z. Saidin, “Evaluation of e-STAR: an enhanced science textbook using Augmented Reality among lower secondary school students”. Jurnal Teknologi. 2015.

. L. C. Beaudin, “Computer self-efficacy and classroom practice: what is the correlation?” (Doctoral dissertation, Lethbridge, Alta.: University of Lethbridge, Faculty of Education, 1998.

. A. Knezek, “Self-Efficacy with Computers (Anxiety Scale). Retrieved from: Texas Center for Educational Technology”, 1997.

. U. C. Pendit, S. B. Zaibon, J. Aida, and J. A. A. Bakar, “Conceptual Model of Mobile Augmented Reality for Cultural Heritage Site towards Enjoyable Informal Learning Aspect”, Jurnal Teknologi, 77(29), 2015.

. S. B. Zaibon, U. C. Pendit, and J. A. A. Bakar, “User requirements on mobile AR for cultural heritage site towards enjoyable informal learning”. In Multimedia and Broadcasting (APMediaCast), 2015 Asia Pacific Conference on (pp. 1-7). IEEE, 2015.

. G. Schraw, R. Bruning and C. Svoboda, C. Sources of situational interest. Journal of Literacy Research, 27(1), 1-17. 1995.




How to Cite

Baker, E. J., Abu Bakar, J. A., & Zulkifli, A. N. (2017). Mobile Augmented Reality Elements for Museum Hearing Impaired Visitors’ Engagement. Journal of Telecommunication, Electronic and Computer Engineering (JTEC), 9(2-12), 171–178. Retrieved from

Most read articles by the same author(s)