Viseme Recognition using lip curvature and Neural Networks to detect Bangla Vowels

Authors

  • Nahid Akhter Department of Computer Science and Engineering BRAC University Dhaka-1212, Bangladesh
  • Amitabha Chakrabarty Department of Computer Science and Engineering BRAC University Dhaka-1212, Bangladesh

Keywords:

Active Appearance Model, Artificial Neural Networks, Bangla Viseme Recognition, Lip Reading, Speech Recognition,

Abstract

Automatic Speech Recognition plays an important role in human-computer interaction, which can be applied in various vital applications like crime-fighting and helping the hearing-impaired. This paper provides a new method for recognition of Bengali visemes based on a combination of image-based lip segmentation techniques, use of curvature of the both inner and outer lips as well as neural networks. The method is divided into three steps. First step is a lip segmentation step that uses a combination of red exclusion method, HSV space and CIE spaces to produce illumination invariant images. Next, inner and outer lips are extracted separately using a new technique for curve-fitting. Second step is the feature extraction step, which makes use of quadratic curve-coefficients of the inner and outer lip contours. Finally, viseme recognition is done using a Neural Network. A dataset was created with 171 lip images of Bangla Visemes being spoken by different speakers and under different lighting conditions. The proposed method gave a viseme recognition result of 87.3%. Due to the use of non-iterative method as opposed to conventional methods, the algorithm was found to be faster in detecting lip contours.

Downloads

Published

2017-12-30

How to Cite

Akhter, N., & Chakrabarty, A. (2017). Viseme Recognition using lip curvature and Neural Networks to detect Bangla Vowels. Journal of Telecommunication, Electronic and Computer Engineering (JTEC), 9(4), 7–11. Retrieved from https://jtec.utem.edu.my/jtec/article/view/1586

Issue

Vol. 9 No. 4: October - December 2017

Section

Articles

License

TRANSFER OF COPYRIGHT AGREEMENT

The manuscript is herewith submitted for publication in the Journal of Telecommunication, Electronic and Computer Engineering (JTEC). It has not been published before, and it is not under consideration for publication in any other journals. It contains no material that is scandalous, obscene, libelous or otherwise contrary to law. When the manuscript is accepted for publication, I, as the author, hereby agree to transfer to JTEC, all rights including those pertaining to electronic forms and transmissions, under existing copyright laws, except for the following, which the author(s) specifically retain(s):

  1. All proprietary right other than copyright, such as patent rights
  2. The right to make further copies of all or part of the published article for my use in classroom teaching
  3. The right to reuse all or part of this manuscript in a compilation of my own works or in a textbook of which I am the author; and
  4. The right to make copies of the published work for internal distribution within the institution that employs me

I agree that copies made under these circumstances will continue to carry the copyright notice that appears in the original published work. I agree to inform my co-authors, if any, of the above terms. I certify that I have obtained written permission for the use of text, tables, and/or illustrations from any copyrighted source(s), and I agree to supply such written permission(s) to JTEC upon request.