RECOGNITION OF TELUGU

CHARACTERS USING
DYNAMIC HAND GESTURES

TEAM MEMBERS:
Under the esteemed guidance of
Dr. Songhita Misra , M. Tech., Ph.D K. Pavani (16A91A0427)
RVS.Yaswanth(16A91A0447)
Y.V.Lalitha (16A91A0449)
Sk Sabbhir (16A91A0453)
P. Hemanth (16A91A0445)
3/5/2020 1 Aditya Engineering College
 OUTLINE
 Introduction
 Motivation
 Literature Survey
 Proposed Methodology
 Database Preparation
 Results and Discussion
 Conclusion and Future Scope
 References

3/5/2020 2 Aditya Engineering College

 INTRODUCTION

 The aim of gesture recognition is to create a

system which can identify specific human
gestures and use them to convey information
or for device control .

 It is a challenging task to develop a virtual

text entry interface covering a large number
of gesture based characters[1].

 Our objective is to recognize the dynamic

hand gesture images based on orientations
and other trajectory features extracted from
the input video stream recorded in stable
lighting and simple background conditions.

3
3/5/2020 Aditya Engineering college
 STAGES OF DHGR SYSTEM
On recognition
of gesture it is Virtual text-entry Gesture is
used in the made using the
applications
interface system input device

Gesture Input Device

Recognition Detection
Training model
for each
gesture is
developed On detecting
which is used the input
during the device, it is
testing phase
Gesture in the Input Device tracked till the
using testing Phase tracking last frame
classifiers

Multiple Feature
features Gesture Spotting After tracking
extracted
Extraction trajectory
using the points are
shape and obtained,
geometrical as unwanted
attributes of strokes are
the gesture
Fig 1: 2 Stages of a DHGR system removed
3/5/2020 4 Aditya Engineerng College
 MOTIVATION
 Gesture is a form of non-verbal communication in which visible body
communicates particular messages.
 Gesture based virtual text entry interface is to ease the human computer
interaction(HCI) while using computational or operational devices such as
computer, mobile, and remote[2].
 The use of such devices in the present world has become a part and parcel
of life.
 Gesture based recognition systems aim to interact with virtual objects
without the use of mouse and text entry interface such as mechanical
keyboards, smart phones and tablets keyboards.
 No such system has been developed for Telugu alphabets

3/5/2020 5 Aditya Engineering College

 LITERATURE SURVEY:
Author and Title Gesture System attributes
year
J. Davis and Visual gesture Left, right, up, Color-marker on finger tip;
M. Shah recognition down, grab, rotate, trajectory based
[1994][3] and stop.
S.M Siam et Human Computer Click movement, Two markers : red and
al. [2016] [4] Interaction Using right click, left green mounted on the
Marker Based Hand click, double click, finger tips; Tracked and
Gesture Recognition going forward and classified using kalman
backward, zoom in filter.
and out.
J. Singha and Hand gesture 40 Gestures Single red-marker on finger
R. H. Laskar recognition using (alphabets, tip; Trajectory features
[2016] [5] two-level speed numbers, arithmetic
normalization, operators)
feature selection
and classifier
fusion.
3/5/2020 8 Aditya Engineering College
 LITERATURE SURVEY:
Authors and Title Gestures System attributes
year
Dynamic hand Kanade-Lucas-Tomasi
gesture 40 gestures (alphabets, feature tracker; trajectory
Singha et.al. recognition numbers, arithmetic features; ANN,SVM and
[2018] [6] using vision operators) kNN model.
based approach
for HCI
Bhuyan et. al A novel set of Arabic numbers Skin-color cues in HSV
[2014] [7] features for and YCbCr color spaces;
continuous hand Trajectory features;
gesture Conditional random
recognition fields(CRF)
Misra et. al Vision based 58 gestures(English Single red color marker
[2017] [8] hand gesture alphabets, arabic on finger tip; Trajectory
recognition numbers, Arithmetic features
operators and other 18
ASCII characters)
3/5/2020 9 Aditya Engineering College
 LITERATURE SURVEY:
Authors and Year Title Gesture System Attributes
Mahmoud Elmezian A Hidden Recognizes both HMM usingLR,LRB
et.al[2008][9] MarkovModel Based isolated and
Continuous Gesture continuous gestures
Recognition System for Arabic
for Hand Motio numbers(0-9)
Trajectory
Zhihan Lv Preprint Touch Less To develop Augumented reality
et.al[2015][10] Interactive touchless, interactive games
Agumented Reality and augumented
Game on Vision reality games
Based Wearable
Device
Serban Oprisescu Automatic Static Static HGR relying Decision tree
et.al[2012][11] Hand Gesture on both depth and
Recognition using intensity information
TOF Cameras by TOF cameras.

3/5/2020 10 Aditya Engineering College

 LITERATURE SURVEY: EXISTING
METHODS

 Many researchers have proposed numerous methods for hand gesture

recognition system.
 1) Contact based system
i) Data Gloves
ii) Magnetic tracker, etc.
 2) Vision-based systems
i)Red-marker and Bare-hand Fig.1: Contact based hand gesture
ii)2D and 3D

3/5/2020 12 Aditya Engineering College

 PROPOSED METHOD
 Any vision based hand gesture recognition system starts with the image
processing steps which are image/video acquisition, hand/marker detection
for gesture spotting, followed by trajectory acquisition.

 One of the objective of the study is to collect Telugu alphabet database as

there is no publicly available database for this.

 Different combinations of gestures are developed and stored in training

database.

3/5/2020 13 Aditya Engineering College

 Extraction of
red color
component
from RGB

Subtract the Removal of

Image/Video
Input frame gray frame impulse noise
acquisition
from the red using 3x3
frame median filter

Converting
input RGB
frame to gray Conversion to
binary image

Figure 1: Block diagram of red-color marker detection

3/5/2020 14 Aditya Engineering College

 Max edge distance
Final gesture

Start end distance

Figure 2: Ramer-Douglas-Peucker algorithm process

3/5/2020 15 Aditya Engineering College

 PROPOSED METHOD
(Contd..)
 The video is taken as the input frame once the red marker has
been detected from the background.
 The next step is to track the detected part of the image from the
first frame till the last frame of the video, this will provide the
required gesture trajectory.
Ramer-Douglas-Peucker algorithm can be used to reduce the
noise from the gesture trajectories.
Once all the smoothened trajectory points are obtained, the next
step is spotting of self-co-articulation strokes.
This provides the information whether the gesture is single
stroked or multiple stroked.

3/5/2020 16 Aditya Engineering College

 DATABASE PREPARATION
The database for the proposed system required 150 recordings
of each 5 gestures.

So, we are developing our own database with 150 recordings per
character.

The database is recorded from different sessions and from multiple persons.

However, the presence of bright red color in the background has been
avoided due to red color segmentation process.

In future, gestures will be captured in uncontrolled environment such

static and dynamic complex environment, and under varying
illumination conditions.

3/5/2020 17 Aditya Engineering College

3/5/2020 Aditya Engineering College 15
 RESULTS AND DISCUSSION

3/5/2020 18 Aditya Engineering College

 RESULTS AND DISCUSSION
Time complexity of classification tools in secs
3.5

2.5

Time complexity in secs

1.5

0.5

0
Naïve Bayes (Distribution and Kernal) K-NN

3/5/2020 19 Aditya Engineering College

 Key Observations:
 The system has been developed using two classification tools, Naïve Bayes,
and K-NN.
 Overall accuracy of Naïve Bayes is 89%.
 Overall accuracy of k-NN for k=2 is 93%.
 Accuracy of k=2 is maximum as compared to k=3 to 10.
 Accuracy of K-NN is observed to be higher than Naïve Bayes.
 Time-complexity of K-NN is much lower than Naïve Bayes system.

3/5/2020 20 Aditya Engineering College

CONCLUSION AND FUTURE SCOPE
 In this study, we have developed a system to recognize 5 Telugu script
characters, using red color marker and trajectory based features.
 In this study, the trajectory features applied are location feature, velocity,
ratio, number of curves, and motion chain code feature.
 Only gestures with no self Co-articulation are recognized.
 Two classification techniques such as Naïve Bayes and K-NN has been
applied.
 The maximum accuracy is achieved by using K-NN which is 93% accuracy.
 In future we will focus on the development of more Telugu based
characters recognition system.
 Such kind of virtual text-entry interface systems can be used by elderly
people, physically disable people to control home appliances without any
physical contact.

3/5/2020 21 Aditya Engineering College

 REFERENCES:
1.Joyeeta Singha, R.H.Laskar(2015),ANN based Hand Gesture Recognition
using self co-articulated set of features.

2.. Misra, Joyeeta singha, R.H.Laskar.(2017). Vision based hand gesture

recognition.

3. Davis, j., & Shah, M. (1994). Visual gesture recognition. IEE proceedings-
Vision, Image And Signal Processing, 141(2), 101-106.

4. Siam, S.M., Sakel, J.A., & abir, M. (2016). Human Computer Interaction
using Marker based Hand Gesture Recognition. arXiv preprintarXiv:1606.07247.

5. Singha, J., & Laskar, R. H. (2016). Hand gesture recognition using two- level
speed normalization, feature selection and classifier fusion.

3/5/2020 22 Aditya Engineering College

 REFERENCES:
6. Joyeeta Singha, Amarjit Roy, R.H. Laskar(2018), Dynamic hand gesture
recognition using vision based approach for human computer interaction

7. Bhuyan, M.k., kumar, D.A., Mac Dorman, kF., & Iwahori, Y.(2014). A novel
set of features for continuous hand gesture recognition. Journal on ultimodal User
Interfaces, 8(4), 333-343.

8. Songhita Misra, R.H.Laskar(2019), A Novel approach towards pattern and

speed invariant Holistic analysis of Dynamic Gesture Recognition system.

9. Joyeeta Singha, R.H.Laskar(2016), Recognition of global hand gesture using

self co-articulation information and classifier fusion.

10. Songhita Misra, R.H.Laskar(2019), Integrated features and GMM based

hand detector applied to character recognition system under practical conditions.

3/5/2020 23 Aditya Engineering College

 REFERENCES:
11.Mahmoud Elmezian, Ayoub Al-Hamadi(2008), A Hidden Marov Model
based Continuous Gesture Recognition System for Hand Motion
Trajectory.

12.Zhihan Lv. Alaa Halawani.(2015), Preprint Touch Less Interactive

Agumented Reality Game on Vision Based Wearable Device.

13.Serban Oprisescu, Christoph Rasche, Bochao Su(2012), Automatic

Static Hand Gesture Recognition using TOF Cameras

3/5/2020 24 Aditya Engineering College

 THANK YOU

3/5/2020 25 Aditya Engineering College