Analysis of SURF and SIFT Representations to Recognize Food Objects
Keywords:
Bag of Features, Food Recognition, Image Classification, Local Features,Abstract
The social media services such as Facebook, Instagram and Twitter has attracted millions of food photos to be uploaded every day since its inception. Automatic analysis on food images are beneficial from health, cultural and marketing aspects. Hence, recognizing food objects using image processing and machine learning techniques has become emerging research topic. However, to represent the key features of foods has become a hassle from the immaturity of current feature representation techniques in handling the complex appearances, high deformation and large variation of foods. To employ many kinds of feature types are also infeasible as it inquire much pre-processing and computational resources for segmentation, feature representation and classification. Motivated from these drawbacks, we proposed the integration on two kinds of local feature namely Speeded-Up Robust Feature (SURF) and Scale Invariant Feature Transform (SIFT) to represent the features large variation food objects. Local invariant features have shown to be successful in describing object appearances for image classification tasks. Such features are robust towards occlusion and clutter and are also invariant against scale and orientation changes. This makes them suitable for classification tasks with little inter-class similarity and large intra-class difference. The Bag of Features (BOF) approach is employed to enhance the discriminative ability of the local features. Experimental results demonstrate impressive overall recognition at 82.38% classification accuracy from the local feature integration based on the challenging UEC-Food100 dataset. Then, we provide depth analysis on SURF and SIFT implementation to highlight the problems towards recognizing foods that need to be rectified in the future research.References
H. H. Fanyu Kong Hollie A. Raynor, Jindong Tan, “DietCam: Multi-view regular shape food recognition with a camera phone,” Pervasive and Mobile Computing, vol. 19, no. C, pp. 108–121, 2015.
J. T. Fanyu Kong, “DietCam: Automatic dietary assessment with mobile camera phones,” Pervasive and Mobile Computing, vol. 8, pp. 147–163, 2012.
M. Bosch, F. Zhu, N. Khanna, C. J. Boushey, and E. J. Delp, “Combining Global And Local Features For Food Identification In Dietary Assessment Video and Image Processing Lab ( VIPER ) School of Electrical and Computer Engineering Department of Foods and Nutrition,” pp. 1789–1792, 2011.
H. Kagaya and K. Aizawa, “New Trends in Image Analysis and Processing -- ICIAP 2015 Workshops,” vol. 9281, pp. 350–357, 2015.
Z. Z. Duc Thanh Nguyen Philip O. Ogunbona, Yasmine Probst ,Wanqing Li, “Food image classification using local appearance and global structural information,” Neurocomputing, vol. 140, pp. 242–251, 2014.
G. Csurka, C. Dance, L. Fan, J. Willamowski, and Cedric Bray, “Visual categorization with bag of keypoints,” International Workhop on Statistical Learning in Computer Vision, pp. 1–22, 2004.
S. Keypoints and D. G. Lowe, “Distinctive Image Features from,” International Journal in Computer Vision, vol. 60, no. 2, pp. 91–110, 2004.
U. L. Altintakan and A. Yazici, “Towards effective image classification using class-specific codebooks and distinctive local features,” IEEE Transaction on Multimedia, vol. 17, no. 3, pp. 323–332, 2015.
V. C. Luciano Oliveira Gustavo Neves, Talmai Oliveira, EduardoJorge, Miguel Lizarraga c, “A mobile, lightweight, poll-based food identification system,” Pattern Recognition, vol. 47, pp. 1941–1952, 2014.
L. G. MariosM. Anthimopoulos Luca Scarnato, Peter Diem, Stavroula G.Mougiakakou, “A Food Recognition System for Diabetic Patients Based on an Optimized Bag-of-Features Model,” IEEE Journal on Biomedical and Health Informatics, vol. 18, no. 4, pp. 1261–1271, 2014.
P. Pouladzadeh, S. Shirmohammadi, S. Member, and R. Al-maghrabi, “Measuring Calorie and Nutrition From Food Image,” vol. 63, no. 8, pp. 1947–1956, 2014.
F. Kong, “Automatic Food Intake Assessment Using Camera Phones,” 2012.
M. B. Fengqing Zhu InsooWoo, Sung Ye Kim, Carol J. Boushey, David S. Ebert, Edward J. Delp, “The Use of Mobile Devices in Aiding Dietary Assessment and Evaluation,” IEEE Journal of Selected Topics in Signal Processing, vol. 4, no. 4, pp. 756–766, 2010.
Y. K. Keiji Yanai, “Twitter Food PhotoMining and Analysis for One Hundred Kinds of Foods,” 2014.
S. V. B. P. Parisa Pouladzadeh Pallavi Kuhad, Abdulsalam Yassine, Shervin Shirmohammadi, “A virtualization mechanism for real-time multimedia-assisted mobile food recognition application in cloud computing,” Cluster Computing, vol. 18, no. 3, pp. 1099–1110, 2015.
Y. Kawano and K. Yanai, “FoodCam: A real-time food recognition system on a smartphone,” Multimedia Tools Application, vol. 74, no. 14, pp. 5263–5287, 2015.
G. M. Farinella, D. Allegra, M. Moltisanti, F. Stanco, and S. Battiato, “Retrieval and classification of food images,” Comput. Biol. Med., vol. 77, pp. 23–39, 2016.
L. Bossard, M. Guillaumin, and L. Van Gool, “Food-101 - Mining discriminative components with random forests,” Lecture Notes Computing Science (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 8694 LNCS, no. PART 6, pp. 446–461, 2014.
J. T. Fanyu Kong, “DietCam: Regular Shape Food Recognition with a Camera Phone,” in International Conference on Body Sensor Networks, 2011.
H. Kagaya, “Food Detection and Recognition Using Convolutional Neural Network,” no. 3, pp. 1085–1088, 2014.
L. H. Ruihan Xu Shuqiang Jiang, ShuangWang, Xinhang Song, Ramesh Jain, “Geolocalized Modeling for Dish Recognition,” IEEE Transaction on Multimedia, vol. 17, no. 8, pp. 1187–1199, 2015.
M. Giovanni Maria Farinella and S. Battiato, “Classifying Food Images Represented As Bag Of Textons Giovanni Maria Farinella Department of Mathematics and Computer Science Image Processing Laboratory - University of Catania,” pp. 5212–5216, 2014.
D. A. Lisin, M. A. Mattar, M. B. Blaschko, E. G. Learned-Miller, and M. C. Benfield, “Combining Local and Global Image Features for Object Class Recognition,” 2005 IEEE Computer Society Conference on Computer Vision Pattern Recognition Workshop, vol. 3, pp. 47–47, 2005.
T. Joutou and K. Yanai, “A food image recognition system with Multiple Kernel Learning,” in 2009 16th IEEE International Conference on Image Processing (ICIP), 2009, pp. 285–288.
H. Hoashi, T. Joutou, and K. Yanai, “Image Recognition of 85 Food Categories by Feature Fusion,” 2010.
Y. Matsuda, H. Hoashi, and K. Yanai, “Recognition of multiple-food images by detecting candidate regions,” in Proceedings - IEEE International Conference on Multimedia and Exposition, 2012, pp. 25–30.
K. Y. Yoshiyuki Kawano, “FoodCam: A real-time food recognition system on a smartphone,” Multimedia Tools Application, vol. 74, no. 14, pp. 5263–5287, 2015.
Y. Kawano and K. Yanai, “Rapid Mobile Object Recognition Using Fisher Vector,” 2013 2nd IAPR Asian Conference Pattern Recognition, pp. 476–480, 2013.
Y. Li, S. Wang, Q. Tian, and X. Ding, “Feature representation for statistical-learning-based object detection: A review,” Pattern Recognition, vol. 48, no. 11, pp. 3542–3559, 2015.
G. M. Farinella, D. Allegra, and F. Stanco, “A Benchmark Dataset to Study the Representation of Food Images.”
J. Cui, M. Cui, B. Xiao, and G. Li, “Compact and discriminative representation of Bag-of-Features,” Neurocomputing, vol. 169, pp. 55–67, 2015.
H. Bay, T. Tuytelaars, and L. Van Gool, “SURF: Speeded up robust features,” Lect. Notes Comput. Sci. (including Subser. Lecture Notes Artificial Intelligent and Lecture Notes Bioinformatics), vol. 3951 LNCS, pp. 404–417, 2006.
H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool, “Speeded-Up Robust Features (SURF),” Computer Vision and Image Understanding, vol. 110, no. 3, pp. 346–359, 2008.
C. Hiba, Z. Hamid, and A. Omar, “Bag of Features Model Using the New Approaches: A Comprehensive Study,” International Journal Advances Computer Science and Applications, vol. 1, no. 7, pp. 226–234, 2016.
N. Ali, K. B. Bajwa, R. Sablatnig, S. A. Chatzichristofis, Z. Iqbal, M. Rashid, and H. A. Habib, “A novel image retrieval based on visual words integration of SIFT and SURF,” PLoS One, vol. 11, no. 6, pp. 1–20, 2016.
J. Yu, Z. Qin, T. Wan, and X. Zhang, “Feature integration analysis of bag-of-features model for image retrieval,” Neurocomputing, vol. 120, pp. 355–364, 2013.
C. Morikawa, H. Sugiyama, and K. Aizawa, “Food region segmentation in meal images using touch points,” pp. 7–12, 2012.
P. Pouladzadeh, A. Yassine, and S. Shirmohammadi, “New Trends in Image Analysis and Processing -- ICIAP 2015 Workshops,” vol. 9281, pp. 441–448, 2015.
P. Pouladzadeh, S. Shirmohammadi, A. Bakirov, A. Bulut, and A. Yassine, “Cloud-based SVM for food categorization,” Multimedia Tools and Applications, pp. 5243–5260, 2014.
K. Li, F. Wang, and L. Zhang, “A new algorithm for image recognition and classification based on improved Bag of Features algorithm,” Opt. - International Journal on Light Electron Optics, vol. 127, no. 11, pp. 4736–4740, 2016.
Downloads
Published
How to Cite
Issue
Section
License
TRANSFER OF COPYRIGHT AGREEMENT
The manuscript is herewith submitted for publication in the Journal of Telecommunication, Electronic and Computer Engineering (JTEC). It has not been published before, and it is not under consideration for publication in any other journals. It contains no material that is scandalous, obscene, libelous or otherwise contrary to law. When the manuscript is accepted for publication, I, as the author, hereby agree to transfer to JTEC, all rights including those pertaining to electronic forms and transmissions, under existing copyright laws, except for the following, which the author(s) specifically retain(s):
- All proprietary right other than copyright, such as patent rights
- The right to make further copies of all or part of the published article for my use in classroom teaching
- The right to reuse all or part of this manuscript in a compilation of my own works or in a textbook of which I am the author; and
- The right to make copies of the published work for internal distribution within the institution that employs me
I agree that copies made under these circumstances will continue to carry the copyright notice that appears in the original published work. I agree to inform my co-authors, if any, of the above terms. I certify that I have obtained written permission for the use of text, tables, and/or illustrations from any copyrighted source(s), and I agree to supply such written permission(s) to JTEC upon request.
