Real-Time Kinect Fingers Tracking and Recognition for Servo Gripper Controller
Keywords:Finger Tracking, Gesture Recognition, Kinect, Real-Time, Servo Gripper Control,
AbstractThe conventional method to control something like machines such as remote controllers or wearable sensors have its limitation and cannot cater the high demands in some scenarios. To overcome this situation, a real-time Kinect finger tracking and recognition is developed to control a servo gripper via a single board microcontroller, Arduino. This paper presents hand and finger tracking method that can determine the number of fingers and also the angle of the finger’s position. The hand gesture is captured by the Kinect and go through a series of image processing and finally, the information will send to the Arduino. The image processing method includes a detection of hand using depth sensor on the Kinect, then finds the finger and calculates the angle of the finger’s position. The convex hull algorithm is used to represent the region of the hand. The fingertips are recognized by calculating the angle of the fingertip and each angle is compared with threshold angle. The orientation and position of the fingers are obtained by finding the middle line of the finger and compares to the vertical line of the middle finger, then calculates the angle. The result shows that the number of fingers appeared in the display can be recognized and the orientation and position of each finger can be determined. A gripper also able to react simultaneously (open and close) based on the detected finger.
C. A. Burande, R. M. Tugnayat, and N. K. Choudhary, “Advanced recognition techniques for human computer interaction,” in Proc. of The 2nd International Conference on Computer and Automation Engineering (ICCAE), 2010, pp. 480-483.
Bulling and K. Kunze, “Eyewear computers for human-computer interaction,” in Interactions, pp. 70-73, 2016. DOI=http://dx.doi.org/10.1145/2912886.
M. M. F. M. Fareed, Q. I. Akram, S. B. A. Anees and A. H. Fakih, “Gesture based wireless single-armed robot in cartesian 3D space using Kinect,” in Proc. of The Fifth International Conference on Communication Systems and Network Technologies, 2015, pp.1210- 1215
C. Li, C.Yang, P. Liang, A. Cangelosi and J. Wan, “Development of Kinect based teleoperation of Nao robot,” in Proc. of International Conference on Advanced Robotics and Mechatronics (ICARM), 2016, pp.133-138.
R. Afthoni, A. Rizae and E. Susanto, “Proportional derivative control based robot arm system using Microsoft Kinect,” in Proc. of International Conference on Robotics, Biomimetics, Intelligent Computational Systems (ROBIONETICS), Yogyakarta, Indonesia, 2013, pp. 24-29.
M. M. Ali, H. Liu, N. Stoll, K. Thuro, “Intelligent arm manipulation system in Life Science Labs using H20 mobile robot and Kinect sensor,” in Proc. of IEEE 8th International Conference on Intelligent Systems,2016, pp.382-387.
M. V. Liarokapis, P. K. Artemiadis, and K. J. Kyriakopoulos, “Mapping human to robot motion with functional anthropomorphism for teleoperation and telemanipulation with robot arm hand systems,” in Proc. of IEEE International Conference on Intelligent Robots and Systems (IROS), 2013, pp. 2075-2075.
J. Shin and C. M. Kim, “Non-touch character input system based on hand tapping gestures using Kinect sensor,” in IEEE Access Open Access Journal, 2017. Doi: 10.1109/ACCESS.2017.2703783.
Z. Lai, Z. Yao, C. Wang, H. Liang, H. Chen and W. Xia, “Fingertips detection and hand gesture recognition based on Discrete Curve Evolution with a Kinect sensor,” in Proc. of The International Conference on Visual Communications and Image Processing (VCIP), 2016.
PrimeSense, “OpenNI NiTE126.96.36.199,” 2012. [Online]. Available:http://openni.ru/files/nite
W. Yan, H. Chuanyan, Y. Guanghui, and W. Changbo, “A robust method of detecting hand gestures using depth sensors,” in Proc. of IEEE International Workshop on Haptic Audio Visual Environments and Games (HAVE), 2012, pp. 72-77.
R. Hartanto, A. Susanto, and P. I. Santosa, “Real time hand gesture movements tracking and recognizing system,” in Electrical Power, Electronics, Communications, Controls and Informatics Seminar (EECCIS), 2014, pp. 137-141
Arduino, “Arduino Uno Rev3,”. 2015. [Online]. Available:http://www.arduino.cc/en/Main/ArduinoBoardUno
How to Cite
TRANSFER OF COPYRIGHT AGREEMENT
The manuscript is herewith submitted for publication in the Journal of Telecommunication, Electronic and Computer Engineering (JTEC). It has not been published before, and it is not under consideration for publication in any other journals. It contains no material that is scandalous, obscene, libelous or otherwise contrary to law. When the manuscript is accepted for publication, I, as the author, hereby agree to transfer to JTEC, all rights including those pertaining to electronic forms and transmissions, under existing copyright laws, except for the following, which the author(s) specifically retain(s):
- All proprietary right other than copyright, such as patent rights
- The right to make further copies of all or part of the published article for my use in classroom teaching
- The right to reuse all or part of this manuscript in a compilation of my own works or in a textbook of which I am the author; and
- The right to make copies of the published work for internal distribution within the institution that employs me
I agree that copies made under these circumstances will continue to carry the copyright notice that appears in the original published work. I agree to inform my co-authors, if any, of the above terms. I certify that I have obtained written permission for the use of text, tables, and/or illustrations from any copyrighted source(s), and I agree to supply such written permission(s) to JTEC upon request.