Autonomous Fruit Harvester with Machine Vision

Authors

  • Kathleen Anne M. Almendral Electronics and Communications Engineering Department of De La Salle University, Manila, 2401 Taft Ave., Malate, Manila 1004, Philippines.
  • Rona Mae G. Babaran Electronics and Communications Engineering Department of De La Salle University, Manila, 2401 Taft Ave., Malate, Manila 1004, Philippines.
  • Bryan Jones C. Carzon Electronics and Communications Engineering Department of De La Salle University, Manila, 2401 Taft Ave., Malate, Manila 1004, Philippines.
  • Karl Patrick K. Cu Electronics and Communications Engineering Department of De La Salle University, Manila, 2401 Taft Ave., Malate, Manila 1004, Philippines.
  • Jasmine M. Lalanto Electronics and Communications Engineering Department of De La Salle University, Manila, 2401 Taft Ave., Malate, Manila 1004, Philippines.
  • Alexander C. Abad Electronics and Communications Engineering Department of De La Salle University, Manila, 2401 Taft Ave., Malate, Manila 1004, Philippines.

Keywords:

Fruit Harvester, Machine Vision, Robotic Arm, ZED Stereo Camera,

Abstract

This study presents an autonomous fruit harvester with a machine vision capable of detecting and picking or cutting an orange fruit from a tree. The system of is composed of a six-degrees of freedom (6-DOF) robotic arm mounted on a four-wheeled electric kart. The kart uses ZED stereo camera for depth estimation of a target. It can also be used to detect trees using the green detection algorithm. Image processing is done using Microsoft Visual Studio and OpenCV library. The x & y coordinates and distance of the tree are passed on to Arduino microcontroller as inputs to motor control of the wheels. When the kart is less than 65cm to the tree, the kart stops and the robotic arm system takes over to search and harvest orange fruits. The robotic arm has a webcam and ultrasonic sensor attached at its end-effector. The webcam is used for orange fruit detection while ultrasonic sensor is used to provide feedback on the distance of the orange fruit to end-effector. Multiple fruit harvesting is successfully done. The success rate of harvesting and putting fruit into the basket is 80% and 85% for the gripper end-effector and cutter end-effector respectively.

References

S. Blackmore, W. Stout, M. Wang, B. Runov, “Robotic agriculture – the future of agricultural mechanisation? 5th European Conference on Precision Agriculture. ed. J. Stafford, V. The Netherlands, Wageningen Academic Publishers. 2005. pp: 621-628.

B. Tanuja Priya, B.N.S. Murthy, S. Vasanthu, S. Jaganadh, "ROBOTICS: THE FUTURE FOR FARM MECHANIZATION", International Journal of Research in Engineering and Applied Sciences, VOLUME 5, ISSUE 5 (May, 2015).

B. Sani, "The Role of Robotics at the Future of Modern Farming" , 2012 International Conference on Control, Robotics and Cybernetics (ICCRC2012).

S. Hayashi, K. Ganno, Y.Ishhi and T. Ota, “Robotic harvesting system for egg plants,” Japan Agril. Res. Quart, 2002, 36(3): 163 – 168.

S. Hayashi, T. Ota, K. Kubota, K. Ganno and N. Konda, “Robotic harvesting technology for fruit vegetables in protected horticultural production,” In: Proc. 5th symposium on information and technology for sustainable fruit and vegetable production, 12-16, Montpellier, France, 2005, Pp. 227-235.

S. S. Mehta, "Vision-based control for autonomous robotic citrus harvesting", Master of Science Thesis, University of Florida, 2007.

S. Bargoti and J. Underwood, "Deep Fruit Detection in Orchards", Online]. Available: https://arxiv.org/abs/1610.03677, [Accessed: April, 17, 2017].

K. Kapach, E. Barnea, R. Mairon, Y. Edan, and O. Ben-Shahar, “Computer Vision for Fruit Harvesting Robots - state of the Art and Challenges Ahead,” International Journal of Computational Vision and Robotics, vol. 3, no. 1-2, pp. 4–34, apr 2012.

A. Payne, K. Walsh, P. Subedi, and D. Jarvis, “Estimating mango crop yield using image analysis using fruit at ’stone hardening’ stage and night time imaging,” Computers and Electronics in Agriculture, vol. 100, pp. 160–167, jan 2014.

Q. Wang, S. Nuske, M. Bergerman, and S. Singh, “Automated Crop Yield Estimation for Apple Orchards,” in Experimental Robotics, ser. Springer Tracts in Advanced Robotics. Springer International Publishing, 2013, vol. 88, pp. 745–758.

S. Nuske, K. Wilshusen, S. Achar, L. Yoder, and S. Singh, “Automated visual yield estimation in vineyards,” Journal of Field Robotics, vol. 31, no. 5, pp. 837–860, sep 2014.

W. C. Seng and S. H. Mirisaee, “A new method for fruits recognition system,” in 2009 International Conference on Electrical Engineering and Informatics, vol. 1, pp. 130–134, IEEE, 2009.

J. Lv, D. Zhao, W. Ji, Y. Chen, H. Shen, "Design and Research on Vision System of Apple Harvesting Robot", Proceedings of the International Conference on Intelligent Human-Machine Systems and Cybernetics; Zhejiang, China. 2011; pp. 177–180.

D. Font, T. Pallejà, M. Tresanchez, D. Runcan, J. Moreno, D. Martínez, J. Palacín, A proposal for automatic fruit harvesting by combining a low cost stereovision camera and a robotic arm, Sensors, 14 (2014) 11557-11579.

P. C. Carando, S. A. Dela Cruz, N. J. Lorena, J. P. Zabat, and A. C. Abad, "Wireless Controlled Robotic Arm for Handling Hazardous Chemicals," 4th DLSU Innovation and Technology (DITECH) Fair, DITECH 2016, De La Salle University, Manila, Nov. 24-25, 2016.

“US-100 Ultrasonic Sonar Module”, [Online]. Available: http://www.egizmo.com/KIT/images/ultrasonicsonar/ultrasonic%20sonar%20mod ule%201r0.pdf, [Accessed: April, 17, 2017].

“Eroding and Dilating, OpenCV Tutorial”, [Online]. Available: http://docs.opencv.org/2.4/doc/tutorials/imgproc/erosion_dilatation/er osion_dilatation.html [Accessed: April, 17, 2017].

G. Bradski and A. Kaehler, “Learning OpenCV”, OReilly Media, Inc, 2008.

G. Chan, “Using MDD10A with ARDUINO UNO”, Tutorials by Cytron Technologies April 5, 2015, [Online]. Available: http://tutorial.cytron.com.my/2015/04/05/using-mdd10a-witharduino-uno/ [Accessed: April, 17, 2017].

Downloads

Published

2018-02-05

How to Cite

Almendral, K. A. M., Babaran, R. M. G., Carzon, B. J. C., Cu, K. P. K., Lalanto, J. M., & Abad, A. C. (2018). Autonomous Fruit Harvester with Machine Vision. Journal of Telecommunication, Electronic and Computer Engineering (JTEC), 10(1-6), 79–86. Retrieved from https://jtec.utem.edu.my/jtec/article/view/3671

Issue

Vol. 10 No. 1-6: Breakthrough To Excellence in Communication and Computer Engineering III

Section

Articles

License

TRANSFER OF COPYRIGHT AGREEMENT

The manuscript is herewith submitted for publication in the Journal of Telecommunication, Electronic and Computer Engineering (JTEC). It has not been published before, and it is not under consideration for publication in any other journals. It contains no material that is scandalous, obscene, libelous or otherwise contrary to law. When the manuscript is accepted for publication, I, as the author, hereby agree to transfer to JTEC, all rights including those pertaining to electronic forms and transmissions, under existing copyright laws, except for the following, which the author(s) specifically retain(s):

  1. All proprietary right other than copyright, such as patent rights
  2. The right to make further copies of all or part of the published article for my use in classroom teaching
  3. The right to reuse all or part of this manuscript in a compilation of my own works or in a textbook of which I am the author; and
  4. The right to make copies of the published work for internal distribution within the institution that employs me

I agree that copies made under these circumstances will continue to carry the copyright notice that appears in the original published work. I agree to inform my co-authors, if any, of the above terms. I certify that I have obtained written permission for the use of text, tables, and/or illustrations from any copyrighted source(s), and I agree to supply such written permission(s) to JTEC upon request.