Multi-Task Learning Approach for Natural Images' Quality Assessment

Authors

  • Redzuan Abdul Manap Centre for Telecommunication Research and Innovation (CeTRI), Faculty of Electronic and Computer Engineering, Universiti Teknikal Malaysia Melaka (UTeM), Melaka, Malaysia. CISTIB, Department of Electronic and Electrical Engineering, University of Sheffield, Sheffield, UK.
  • Alejandro F. Frangi CISTIB, Department of Electronic and Electrical Engineering, University of Sheffield, Sheffield, UK
  • Ling Shao School of Computing Sciences, University of East Anglia, Norwich, UK
  • Abdul Majid Darsono Centre for Telecommunication Research and Innovation (CeTRI), Faculty of Electronic and Computer Engineering, Universiti Teknikal Malaysia Melaka (UTeM), Melaka, Malaysia

Keywords:

Blind Image Quality Assessment, Multi-Task Learning, Spatial Domain Image Features, Trace-Norm Regularization,

Abstract

Blind image quality assessment (BIQA) is a method to predict the quality of a natural image without the presence of a reference image. Current BIQA models typically learn their prediction separately for different image distortions, ignoring the relationship between the learning tasks. As a result, a BIQA model may has great prediction performance for natural images affected by one particular type of distortion but is less effective when tested on others. In this paper, we propose to address this limitation by training our BIQA model simultaneously under different distortion conditions using multi-task learning (MTL) technique. Given a set of training images, our Multi-Task Learning based Image Quality assessment (MTL-IQ) model first extracts spatial domain BIQA features. The features are then used as an input to a trace-norm regularisation based MTL framework to learn prediction models for different distortion classes simultaneously. For a test image of a known distortion, MTL-IQ selects a specific trained model to predict the image’s quality score. For a test image of an unknown distortion, MTLIQ first estimates the amount of each distortion present in the image using a support vector classifier. The probability estimates are then used to weigh the image prediction scores from different trained models. The weighted scores are then pooled to obtain the final image quality score. Experimental results on standard image quality assessment (IQA) databases show that MTL-IQ is highly correlated with human perceptual measures of image quality. It also obtained higher prediction performance in both overall and individual distortion cases compared to current BIQA models.

References

R. A. Manap, L. Shao, and A. F. Frangi, “Non-parametric quality assessment of natural images,” IEEE Multimedia, vol. 23, no. 4, pp. 22- 30, Oct. 2016.

B. Girod, “What’s wrong with mean-squared error?” in Digital Images and Human Vision, A. B. Watson, Ed. Cambridge, MA: MIT Press, 1993, pp. 207-220.

Z. Wang and A. C. Bovik, “Mean squared error: Love it or leave it? A new look at signal fidelity measures,” IEEE Signal Processing Magazine, vol. 26, no. 1, pp. 98-117, Jan. 2009.

D. M. Chandler and S. S. Hemami, “VSNR: A wavelet-based visual signal-to-noise ratio for natural images,” IEEE Trans. Image Processing, vol. 16, no. 9, pp. 2284-2298, Sept. 2007.

E. C. Larson and D. M. Chandler, “Most apparent distortion: Fullreference image quality assessment and the role of strategy,” Journal of Electronic Imaging, vol. 19, no. 1, pp. 011006.1-011006.21, Jan. 2010.

Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: From error visibility to structural similarity,” IEEE Trans. Image Processing, vol. 13, no. 4, pp. 600-612, Apr. 2004.

L. Zhang, X. Mou, and D. Zhang, “FSIM: A feature similarity index for image quality assessment,” IEEE Trans. Image Processing, vol. 20, no. 8, pp. 2378-2386, Aug. 2011.

H. R. Sheikh and A. C. Bovik, “Image information and visual quality,” IEEE Trans. Image Processing, vol. 15, no. 2, pp. 430-444, Feb. 2006.

C. Charrier, O. l’ezoray, and G. Lebrun, “Machine learning to design full-reference image quality assessment algorithm,” Signal Processing: Image Communication, vol. 27, no. 3, pp. 209-219, Mar. 2012.

R. A. Manap, L. Shao and A. F. Frangi, “A non-parametric framework for no-reference image quality assessment,” in Proc. of IEEE Conference on Signal and Information Processing, Orlando, Florida, Dec. 2015, pp. 562-566.

S. A. Golestaneh and D. M. Chandler, “No-reference quality assessment of JPEG images via a quality relevance map,” IEEE Signal Processing Letters, vol. 21, no. 2, pp. 155-158, Feb. 2014.

J. Zhang and T. M. Le, “A new no-reference quality metric for JPEG2000 images,” IEEE Trans. Consumer Electronics, vol. 56, no. 2, pp. 743-750, May 2010.

G. Yammine, E. Wige, and A. Kaup, “A no-reference blocking artefacts visibility estimator in images,” in Proc. of IEEE Conference on Image Processing, Hong Kong, Sept. 2010, pp. 2497-2500.

E. Cohen and Y. Yitzhaky, “No-reference assessment of blur and noise impacts on image quality,” Signal, Image and Video Processing, vol. 4, no. 3, pp. 289-302, Sep. 2010.

H. R. Sheikh, Z. Wang, L. Cormack, and A. C. Bovik, LIVE Image Quality Assessment Database Release 2 [Online]. Available: http://live.ece.utexas.edu/research/quality

A. K. Moorthy and A. C. Bovik, “A two-step framework for constructing blind image quality indices,” IEEE Signal Process. Letters, vol. 17, no. 5, pp. 513-516, May 2010.

M. A. Saad, A. C. Bovik, and C. Charrier, “Blind image quality assessment: a natural scene statistics approach in the DCT domain,” IEEE Trans. Image Processing, vol. 21, no. 8, pp. 3339-3352, Aug. 2012.

A. Mittal, A. K. Moorthy, and A. C. Bovik, “No-reference image quality assessment in the spatial domain,” IEEE Trans. Image Processing, vol. 21, no. 12, pp. 4695-4708, Dec. 2012.

W. Xue, X. Mou, L. Zhang, A. C. Bovik, and X. Feng, “Blind image quality assessment using joint statistics of gradient magnitude and Laplacian features,” IEEE Trans. Image Processing, vol. 23, no. 11, pp. 4850-4862, Nov. 2014.

R. A. Manap, L. Shao and A. F. Frangi, “PATCH-IQ: a patch based learning framework for blind image quality assessment,” Information Sciences, vol. 420, no. 11, pp. 329-344, Aug. 2017.

Q. Wang, J. Chu, L. Xu and Q. Chen, “A new blind image quality framework based on natural color statistic,” Neurocomputing, vol. 172, no. 3, pp. 1798-1810, 2016.

G. Zhai, X. Wu, X. Yang, W. Lin, and W. Zhang, “A psychovisual quality metric in free-energy principle,” IEEE Trans. Image Processing, vol. 21, no. 1, pp. 41-52, Jan. 2012.

K. Gu, G. Zhai, X. Yang, and W. Zhang, “Using free energy principle for blind image quality assessment,” IEEE Trans. Multimedia, vol. 17, no. 1, pp. 50-63, Jan. 2015.

P. Ye, J. Kumar, L. Kang, and D. Doermann, “Unsupervised feature learning framework for no-reference image quality assessment,” in Proc. of IEEE Conference on Computer Vision and Pattern Recognition, Providence, Rhode Island, June 2012, pp. 1098-1105.

L Kang, P. Ye, Y. Li, and D. Doermann, “Convolutional neural networks for no-reference image quality assessment,” in Proc. of IEEE Conference on Computer Vision and Pattern Recognition, Columbus, Ohio, June 2014, pp. 1733-1740.

J. Chen, L. Tang, J. Liu, and J. Ye, “A convex formulation for learning shared structures from multiple tasks,” IEEE Trans. Pattern Analysis Machine Intelligence, vol. 35, no. 5, pp. 1025-1038, May 2013.

D. Zhang and D. Shen, “Multi-modal multi-task learning for joint prediction of multiple regression and classification variables in Alzheimer’s disease,” NeuroImage, vol. 59, no. 2, pp. 895-907, Jan. 2012.

S. Bickel, J. Bogojeska, T. Lengauer, and T. Scheffer, “Multi-task learning for HIV therapy screening,” in Proc. of Conference on Machine Learning, Helsinki, Finland, July 2008, pp. 56-63.

J. Zhou, J. Chen, and J. Ye, “Multi-task learning: theory, algorithms and applications,” SDM Tutorial, SIAM Conference on Data Mining, Anaheim, CA, Apr. 2012, pp. 1-83.

L. Xu, J. Li, W. Lin, Y. Zhang, L. Ma, Y. Fang, Y. Zhang, and Y. Yan, “Multi-task rank learning for image quality assessment,” in Proc. of IEEE Conference on Acoustics, Speech and Signal Processing, Brisbane, Australia, Apr. 2015, pp. 1339-1343.

M. Fazel, H. Hindi, and S. P. Boyd, “A rank minimization heuristic with application to minimum order system approximation,” in Proc. of American Control Conference, Arlington, VA, June 2001, pp. 4734- 4739.

S. Ji and J. Ye, “An accelerated gradient method for trace norm minimization,” in Proc. of Conference on Machine Learning, Montreal, Canada, June 2009, pp. 457-464.

C. J. C. Burges, “A tutorial on support vector machines for pattern recognition,” Journal on Data Mining and Knowledge Discovery, vol. 2, no. 2, pp. 121 – 167, 1998.

J. Zhou, J. Chen, and J. Ye, MALSAR: Multi-task learning via structural regularization, Arizona State University, 2012 [Online]. Available: http://www.public.asu.edu/~jye02/software/MALSAR

C. –C. Chang and C. –J. Lin, LIBSVM: A library for support vector machines [Online]. Available: https://www.csie.ntu.edu.tw/~cjlin/libsvm

Downloads

Published

2018-07-04

How to Cite

Abdul Manap, R., F. Frangi, A., Shao, L., & Darsono, A. M. (2018). Multi-Task Learning Approach for Natural Images’ Quality Assessment. Journal of Telecommunication, Electronic and Computer Engineering (JTEC), 10(2-5), 1–7. Retrieved from https://jtec.utem.edu.my/jtec/article/view/4340