Mobile Video Streaming Applications: A Systematic Review of Test Metrics in Usability Evaluation
Keywords:
Mobile Video Streaming Applications, Performance Evaluation, Systematic Review, Usability,Abstract
In evaluating the usability of mobile video streaming applications, the performance of the applications comes into focus. This is because the performance of mobile streaming applications affects their usability. From this study, video streaming and video quality are identified as the two most evaluated elements in the usability test of mobile video streaming applications. These elements are affected by several related factors that are peculiar to the mobile platforms and domains. These in turn affect the usability of the applications. In mobile platforms, bandwidth is low and network connections are unstable; this is coupled with the limitations caused by the smallness of the screen sizes of the mobile devices. Furthermore, startup delays, jitter, latency and rebuffering are the determining factors for the performance of mobile video streaming. On the other hand, video quality is determined by frame rate, bit rate, and resolution. These factors present themselves due to the mobile context of mobile streaming applications. They combine to influence the performance of the applications as well as their usability. Therefore, in considering the usability of these set of applications, these factors (metrics) are important as they determine the performance of the applications and by and large also affect the usability of the applications. Other factors identified in the study that affect the usability of mobile streaming applications include: functionality, social context and user interface and appearance. On the whole, this paper presents the results of a systematic review of test metrics in the usability evaluation of mobile video streaming applications. The systematic review approach used include: defining the search strategy, selection of primary studies, the extraction of data, and the implementation of a synthesis strategy. Using this methodology, 238 studies were found; however, only 51 relevant studies were eventually selected for the review. The study reveals that time taken for video streaming and the video quality were the two most popular metrics used in the usability test and evaluation of mobile video streaming applications. Besides, most of the studies concentrated on the usability of mobile TV as users switch from traditional TV to mobile TV.References
H. Khalid, E. Shihab, M. Nagappan, A. Hassan, "What Do Mobile App
Users Complain About? A Study on Free iOS Apps", Software, IEEE, 1,
H. Yuwen, M. Kunstner, S. Gudumasu, R. Eun-Seok, Y. Yan, X. Xiaoyu, "Power aware HEVC streaming for mobile, Visual Communications and Image Processing (VCIP)", 2013, pp.1-5.
H. Yajun, Z. Wenan, D. Yu, "Research on the User Behavior-Based QoE Evaluation Method for HTTP Mobile Streaming", 2014 9th Int'l Conf.
on Broadband & Wireless Comput., Comm. & Appls., 2014, pp.47-51.
W. Song, D. W. Tjondronegoro, and M. Docherty, "Understanding user experience of mobile video: framework, measurement, and
optimization", INTECH Open Access Publisher, 2012.
S. Singh, O. Oyman, A. Papathanassiou, D. Chatterjee, J. G. Andrews, "Video capacity and QoE enhancements over LTE", 2012 IEEE Int'l Conf. on Comm., 2012, pp.7071-7076.
K. D. Singh, Y. Hadjadj-Aoul, G. Rubino, "Quality of experience estimation for adaptive HTTP/TCP video streaming using H.264/AVC", Consumer Comms. & Networking Conf. (CCNC), 2012, pp.127-131.
M. Ramadan, L. El Zein, Z. Dawy, "Implementation and evaluation of cooperative video streaming for mobile devices", IEEE 19th Int'l Symp. on Personal, Indoor & Mobile Radio Comms., 2008, pp.1-5.
A. Kuwadekar, K. Al-Begain, "User Centric Quality of Experience Testing for Video on Demand over IMS", 1st Int'l Conf. on Computational Intelligence, Comm. Systems & Networks, 2009, pp.497-504.
D. Kovachev, C. Yiwei, R. Klamma, "Cloud Services for Improved User Experience", IEEE 7th Int'l Symp. on Sharing Mobile Videos, Service Oriented Sys. Eng. (SOSE), 2013, pp.298-303.
H. Jun, S. Wei, "Towards Smart Routing: Exploiting User Context for Video Delivery in Mobile Networks", IEEE 11th Int'l Conf. on Mobile
Ad Hoc & Sensor Systems (MASS), 2014, pp.227-231.
K. Jahon, C. Kwangsue, "MARC: Adaptive Rate Control scheme for improving the QoE of streaming services in mobile broadband networks", Int'l Symp. on Comms. & Info. Technologies (ISCIT), 2010, pp.105-110.
S. Ickin, M. Fiedler, K. Wac, "Demonstrating the staling events with instantaneous total power consumption in smartphone-based live video streaming", Sustainable Internet & ICT for Sustainability, 2012, pp.1-4.
Z. Hussain, W. Slany, "Analyzing real mobile web usage of a multimedia streaming application through log files", Int'l conf' on Multimedia info. retrieval, Philadelphia, Pennsylvania, USA, 2010.
C. Huifang, X. Yu, X. Lei, "End-to-end qual. adaptation scheme based on QoE pred. for video streaming serv. in LTE netw.", 11th Int'l Symp. on Model. & Optim. in Mobile, Ad Hoc & Wireless Netw., 2013, pp.627-633.
H. Herman, A. A. Rahman, Y. A. Syahbana, K. A. Bakar, "Nonlinearity Modeling of QoE for Video Streaming over Wireless & Mobile Netw.", 2nd Int'l Conf. on Intelligent. Sys. Modeling & Simulation., 2011, pp.313-317.
D. Ghadiyaram, A. C. Bovik, H. Yeganeh, R. Kordasiewicz, M. Gallant, "Study of the effects of stalling events on the quality of experience of mobile streaming videos", IEEE Global Conf. on Signal & Info. Processing, 2014, pp.989-993.
A. Díaz, P. Merino, F. Javier Rivas, "QoS analysis of video streaming service in live cellular networks", Computer Communications, 33, 2010, pp.322-335.
A. Devlic, P. Lungaro, P. Kamaraju, Z. Segall, K. Tollmar, "Energy Consumption Reduction via Context-Aware Mobile Video Prefetching", IEEE Int'l Symposium on Multimedia (ISM), 2012, p.261-265.
L. Chun-Han, H. Pi-Cheng, H. Cheng-Kang, "Dynamic Backlight Scaling Optimization: A Cloud-Based Energy-Saving Service for Mobile Streaming Applications", IEEE Trans. on Com , vol. 63, 2014, pp.335-348.
X. Changgiao, G. Jianfeng, C. Vuanlong, J. Shijie, "Trends of
multimedia streaming development in Cognitive Mobile Internet", Int'l Conf. on Advanced Intelligence and Awareness Internet, 2011, pp.3-6.
F. Bo, G. Kunzmann, M. Wetterwald, D. Corujo, R. Costa, "QoE-aware traffic management for mobile video delivery", 2013 IEEE Int'l Conf. on Communications (ICC), 2013, pp.652-656.
T. N. Smyth, S. Kumar, I. Medhi, K. Toyama, "Where there's a will there's a way: mobile media sharing in urban India", SIGCHI Conference on Human Factors in Computing Systems, Atlanta, Georgia, USA, 2010.
Y. Abe, R. Geambasu, K. Joshi, H. Andrea, S. Lagar-Cavilla, "vTube: efficient streaming of virtual appliances over last-mile networks", 4th annual Symposium on Cloud Computing, Santa Clara, California, 2013.
S. S. Krishnan, R. K. Sitaraman, "Video stream quality impacts viewer behavior: inferring causality using quasi-experimental designs", ACM conference on Internet measurement, Boston, Massachusetts, USA,2012.
E. Kaasinen, M. Kulju, T. Kivinen, V. Oksman, "User acceptance of mobile TV services", 11th Int'l Conf. on Human-Computer Interaction with Mobile Devices and Services, Bonn, Germany, 2009.
M. Z. Shafiq, J. Erman, L. Ji, A. X. Liu, J. Pang, J. Wang, "Understanding the impact of network dynamics on mobile video user engagement", ACM int'l conf. on measurement & modeling of comp. sys, Texas, 2014.
M. M. Saleemi, J. Bjrkqvist, J. Lilius, "System architecture and interactivity model for mobile TV applications", 3rd int'l conf. on Digital Interactive Media in Entertainment and Arts, Athens, Greece, 2008.
K. Wac, S. Ickin, J.-H. Hong, L. Janowski, M. Fiedler, A. K. Dey, "Studying the experience of mobile appls used in different contexts of daily life", 1st SIGCOMM workshop on Meas. up the stack, Toronto,
C. Riede, A. Al-Hezmi, T. Magedanz, "Session and media signaling for IPTV via IMS", 1st int'l conf. on MOBILe Wireless MiddleWARE,
Operating Systems, and Applications, Innsbruck, Austria, 2007.[30] Y. Cui, J. Chipchase, Y. Jung, "Personal TV: a qualitative study of
mobile TV users", 5th European conf. on Interactive TV: a shared
experience, Amsterdam, The Netherlands, 2007.
S. J, Rvinen, J. Peltola, J. Lahti, A. Sachinopoulou, "Multimedia service creation platform for mobile experience sharing", 8th Int'l Conf. on Mobile & Ubiquitous Multimedia, Cambridge, United Kingdom, 2009.
P. Vidales, F. Steuer, N. Kirschnick, M. W, ltermann, "Mobisense testbed: merging user perception & net. perf.", 4th Int'l Conf. on Testbeds & research infrastructures for the dev. of netw. & communities, Innsbruck, 2008.
A. Kaheel, M. El-Saban, M. Refaat, M. Ezz, "Mobicast: a system for collaborative event casting using mobile phones", 8th Int'l Conf. on Mobile & Ubiquitous Multimedia, Cambridge, United Kingdom, 2009.
W. Liu, T. Mei, Y. Zhang, J. Li, S. Li, "Listen, look, and gotcha: instant video search with mobile phones by layered audio-video indexing", 21st ACM int'l conf. on Multimedia, Barcelona, Spain, 2013.
A. Gro, B. Baumann, J. Bross, C. Meinel, "Distribution to multiple platforms based on one video lecture archive", 37th annual ACM SIGUCCS fall conf. on comm. & collaboration, St. Louis, Missouri, USA, 2009.
M. LaRosa, D. Poole, R. Schusteritsch, "Designing and deploying usetube, google's global user experience observation and recording system", Extended Abstracts on Human Factors in Computing Sys., Boston, 2009.
H. Knoche, J. D. McCarthy, "Design requirements for mobile TV", 7th int'l conf' on Human computer interaction with mobile devices & services, Salzburg, Austria, 2005.
K. O'Hara, A. S. Mitchell, A. Vorbau, "Consuming video on mobile devices", SIGCHI Conf. on Human Factors in Computing Systems, San Jose, California, USA, 2007.
M. Davies, A. Dantcheva, P. Frhlich, "Comparing access methods and quality of 3g mobile video streaming services", Extended Abstracts on Human Factors in Computing Systems, Florence, Italy, 2008.
A. Finamore, M. Mellia, M. M. Munaf, R. Torres, S. G. Rao, "YouTube
everywhere: impact of device and infrastructure synergies on user
experience", ACM SIGCOMM conf. on Internet meas. conf., Berlin,
S.-H. Shen, A. Akella, "An information-aware QoE-centric mobile video cache", 19th annual int'l conf. on Mobile computing & networking, Miami, Florida, USA, 2013.
Y. Wu, T. Mei, N. Yu, S. Li, "Accelerometer-based single-handed video browsing on mobile devices: design and user studies", 4th Int'l Conf. on Internet Multimedia Computing and Service, Wuhan, China, 2012.
W. Song, D. Tjondronegoro, "A survey on usage of mobile video in Australia", 22nd Conf. of the CHI Special Interest Group of Australia on Computer-Human Interaction, Brisbane, Australia, 2010.
S. Buchinger, S. Kriglstein, H. Hlavacs, "A comprehensive view on user studies: survey and open issues for mobile TV", 7th European conference on European interactive television conf., Leuven, Belgium, 2009.
O. B. Maia, H. C. Yehia, L. d. Errico, "A concise review of the quality of experience assessment for video streaming", Computer
Communications, vol.57, 2015, pp.1-12..
H. Yoon, J. Kim, R. Hsieh, "Peer-assisted video on-demand streaming system in practical WiFi-based mobile opportunistic networks", J. Network & Computer Applications, vol. 37, 2014, pp.33-44..
H. Viswanathan, D. D. Vleeschauwer, A. Beck, S. Benno, R. B. Miller, G. Li, "Mobile Video Optim. at the Base Station: Adapt. Guarant. Bit Rate for HTTP Adaptive Streaming", Bell Labs Tech. J., vol.18, 2013, pp.159–174.
M. Shiddiqi, H. Pratama, H. T. Ciptaningtyas, "A Video Streaming Application Using Mobile Media Application Programming Interface", TELKOMNIKA Telecomm., Computing, Electronics & Control, vol. 8, 2010.
L. Yao, L. Fei, G. Lei, S. Bo, C. Songqing, L. Yingjie, "Measurement
and Analysis of an Internet Streaming Service to Mobile Devices", IEEE
Trans on Parallel & Distributed Sys, vol.24, 2013, pp.2240-2250.
S.-M. Seo, Y.-H. Choi, "Development of android app for smooth multimedia streaming service via portable media file format", Int'l J. Multimedia and Ubiquitous Engineering, vol.9, 2014, pp.197-208.
Z. Xin-chen, "Scalable video transcoding method for H.264 over wireless network", Int'l Conf. on Future Info. Tech. & Manag. Eng. (FITME), 2010, pp.378-381.
K. Evensen, T. Kupka, H. Riiser, P. Ni, R. Eg, C. Griwodz, "Adaptive
media streaming to mobile devices: challenges, enhancements, and recommendations", Adv. MultiMedia, vol.10, 2014.
A. Hussain, E.O.C. Mkpojiogu, Z. Hussain, "Usability evaluation of a
web-based health awareness portal on Smartphone devices using SO 9241-11 model". Jurnal Teknologi, vol.77, no.4, 2015, pp.1-5.
A. Hussain, E.O.C. Mkpojiogu, "An application of ISO/IEC 25010 standard in the quality-in-use assessment of an online health awareness system". Jurnal Teknologi, vol.77, no.5, 2015, pp.9-13.
A. Hussain, E.O.C. Mkpojiogu, E.O.C. "The effect of responsive web design on the user experience with laptop and Smartphone devices". Jurnal Teknologi, vol.77, no.4, 2015, pp.41-47.
A. Hussain, E.O.C. Mkpojiogu, F. Hassan. "Systematic review of mobile learning applications for children". Proceedings of the 2nd International Conference on Information and Communication Technology for Transformation (IC-ICT4T’16), 5-7 April 2016, Kota Kinabalu, Sabah, Malaysia.
A. Hussain, E.O.C. Mkpojiogu. "Usability evaluation techniques in mobile commerce applications: a systematic review". Proceedings of the 1st International Soft Science Conference (ISSC’16), 11-13 April 2016, Langkawi Island, Malaysia.
A. Hussain, E.O.C. Mkpojiogu. "Perceived usefulness, perceived ease of use, and perceived enjoyment as drivers for the user acceptance of interactive mobile maps". Proceedings of the 1st International Soft Science Conference (ISSC’16), 11-13 April 2016, Malaysia.
E.O.C. Mkpojiogu, N.L. Hashim, R. Adamu. "Observed demographic differentials in user perceived satisfaction on the usability of mobile banking applications". 8th Knowledge Management International Conference (KMICe’16), Chiang Mai, Thailand, 29-30 August 2016.
A. Hussain, E.O.C. Mkpojiogu. "A systematic review on usability evaluation methods in m-commerce apps". Journal of Technology, Electronics & Computer Engineering (JTEC), 2016.
A. Hussain, E.O.C. Mkpojiogu. "Antecedents to user adoption of interactive mobile maps". Journal of Technology, Electronics &
Computer Engineering (JTEC), 2016.
Downloads
Published
How to Cite
Issue
Section
License
TRANSFER OF COPYRIGHT AGREEMENT
The manuscript is herewith submitted for publication in the Journal of Telecommunication, Electronic and Computer Engineering (JTEC). It has not been published before, and it is not under consideration for publication in any other journals. It contains no material that is scandalous, obscene, libelous or otherwise contrary to law. When the manuscript is accepted for publication, I, as the author, hereby agree to transfer to JTEC, all rights including those pertaining to electronic forms and transmissions, under existing copyright laws, except for the following, which the author(s) specifically retain(s):
- All proprietary right other than copyright, such as patent rights
- The right to make further copies of all or part of the published article for my use in classroom teaching
- The right to reuse all or part of this manuscript in a compilation of my own works or in a textbook of which I am the author; and
- The right to make copies of the published work for internal distribution within the institution that employs me
I agree that copies made under these circumstances will continue to carry the copyright notice that appears in the original published work. I agree to inform my co-authors, if any, of the above terms. I certify that I have obtained written permission for the use of text, tables, and/or illustrations from any copyrighted source(s), and I agree to supply such written permission(s) to JTEC upon request.
