ASL Finger Spelling Recognition System for Interactive Learning and Education Purpose

Abstract

The learning platforms for sign language are very limited. Images and videos are usually used to deliver the process of sign language learning. However, sign language is complicated and hard to practice. Thus, a vision-based American Sign Language (ASL) finger spelling recognition system is developed and presented in this paper. The aim of this project is to help users to learn and practice the sign language. ASL is the most common used sign language around the world. Five ASL finger spellings are used as the data set in this project which are 'A', 'E', 'I', 'O' and 'U'. MATLAB software was used to design the graphical user interface (GUI) of the system. The GUI is divided into 3 parts which are Learn, Test and Quiz. The input image was captured by webcam and then converted into HSV color space for segmentation. Recognition process was done by computing 2D correlation coefficient between input testing images and database training images. The accuracy and processing time of 500 testing images with 100 training images are 97.8% and 0.9434 seconds respectively. The challenges of vision-based hand gesture recognition system such as the complexity background and background color were analyzed and they are presented in this paper.