SIGN LANGUAGE

RECOGNITION SYSTEM

PROJECT PHASE I REPORT

Submitted by

NANDHANA P G (714020104053)

PAUL DEEPAK S (714020104060)

SIDDARTH S (714020104085)

SNEKA T (714020104087)

in partial fulfillment for the award of the degree of

BACHELOR OF ENGINEERING

IN

COMPUTER SCIENCE AND ENGINEERING

SRI SHAKTHI INSTITUTE OF ENGINEERING

AND TECHNOLOGY (AUTONOMOUS),

COIMBATORE 641 062

Autonomous Institution,

Accredited by NAAC with “A” Grade

OCTOBER 2023
 BONAFIDE CERTIFICATE

ii
 BONAFIDE CERTIFICATE

Certified that this project report “SIGN LANGUAGE RECOGNITION SYSTEM” is

the bonafide work of “NANDHANA PG (714020104053), PAUL DEEPAK S

(714020104060), SIDDARTH S (714020104085) AND SNEKA T (714020104087)”,

who carried out the work under my supervision.

SIGNATURE SIGNATURE

Dr. K.E. KANNAMMAL Dr. K.E. KANNAMMAL

Professor and Head, Professor and Head,

Department of CSE, Department of CSE,

Sri Shakthi Institute of Sri Shakthi Institute of

Engineering and Technology, Engineering and Technology,

Coimbatore- 641 062. Coimbatore- 641 062.

Submitted for the project work viva-voce Examination held on………………

INTERNAL EXAMINER EXTERNAL EXAMINER

iii
 ACKNOWLEDGEMENT

iv
 ACKNOWLEDGEMENT

First and foremost, I would like to thank God Almighty for giving me the strength.
Without his blessings this achievement would not have been possible.

We express our deepest gratitude to our Chairman Dr. S. Thangavelu for his
continuous encouragement and support throughout our course of study.

We are thankful to our Secretary Er. T. Dheepan for his unwavering support during
the entire course of this project work.

We are also thankful to our Joint Secretary Mr. T. Sheelan for his support during
the entire course of this project work.

We are highly indebted to Principal Dr. A. R. Ravi Kumar for his support during the
tenure of the project.

We are deeply indebted to our Head of the Department, Computer Science and
Engineering, Dr. K.E. Kannammal, for providing us with the necessary facilities.

It’s a great pleasure to thank our Project Guide Dr. K. E. Kannammal, for her
valuable technical suggestions and continuous guidance throughout this project work.

We solemnly extend our thanks to all the teachers and non-teaching staff of our
department, family and friends for their valuable support.

NANDHANA P G
PAUL DEEPAK S
SIDDARTH S
SNEKA T

v
 ABSTRACT

vi
 ABSTRACT

Sign language recognition systems play a pivotal role in bridging communication barriers

between the deaf and hearing communities. This paper presents an overview of a state-of-the-

art sign language recognition system designed to interpret and translate sign language gestures

into text or speech. The system utilizes computer vision techniques, machine learning

algorithms, and deep neural networks to analyze and classify a wide range of sign language

gestures accurately and in real-time. The core components include hand gesture tracking,

feature extraction, and classification. Extensive datasets of sign language gestures were used

for model training, ensuring robust and accurate recognition performance. The system's

applications extend to aiding deaf individuals in daily communication, educational settings, and

facilitating inclusive access to information. Moreover, it offers the potential for innovative and

interactive sign language applications in various domains. This abstract provides an insight into

the key elements of the sign language recognition system, highlighting its importance in

promoting accessibility and communication inclusivity.

vii
 LIST OF FIGURES

viii
 LIST OF FIGURES

FIGURE NO. TITLE PAGE NO.

1.1 Lighting Conditions And Background 3

1.2 Hand Gesture Recognition Flowchart 4
2.1 The effect of self-shadowing (A) 9
and cast shadowing
3.3 System Implementation 22
3.4 Receptive Field Mapping 26
3.6 Architecture Diagram 28
4.1 Convolution Neural Network 43
4.2 Types of Pooling 44

4.3 Connected Layer 45

4.4 Gray Scale Image 46

4.5 Gaussian Blur Image 46

4.6 CNN Workflow 48

4.7 Pictorial Representation of The 51
Working

ix
 LIST OF ABBREVIATIONS

x
 LIST OF ABBREVIATIONS

ADA GRAD Adaptive Gradient Algorithm

API Application Programming Interface

CMOS Complementary Metal Oxide Semiconductor

CNN Convolutional Neural Network

DMA Direct Memory Access

HCI Human-Computer Interaction

KFM Kohohen Feature Map

PDAs Personal Digital Assistants

ReLu Rectified Linear Unit

RMSProp Root Mean Square Propagation

xi
 TABLE OF CONTENTS

xii
 TABLE OF CONTENTS

CHAPTER NO TITLE PAGE NO

vii
ABSTRACT

LIST OF FIGURES ix

LIST OF ABBREVIATIONS xi

1. INTRODUCTION 1

1.1 Digital Image Processing 2

1.2 Biometrics 2

1.3 Hand Gesture Detection and Recognition 2

2
1.3.1 Detection
4
1.3.2 Recognition
5
1.4 Motivation
5
1.5 Scope

1.6 Software Tools 6

2. LITERATURE REVIEW 7

2.1 Lighting 8

2.2 Camera Orientations and Distance 10

2.3 Background Selection 10

2.4 Different Recognition Approaches 10

xiii
 2.4.1 Pen Based Gesture Recognition 10

2.4.2 Tracker Based Gesture Recognition 11

2.4.3 Data Gloves 11

2.4.4 Body Suits 13

2.4.5 Head and Face Gestures 14

2.4.6 Head and Arm Gestures 15

2.5 Body Gestures 16

2.6 Vision Based Gesture Recognition 17

3. PROPOSED METHODOLOGY 20
3.1 Existing System 20
3.2 Proposed System 20
3.3 Proposed plan of work 22
3.4 Convolutional Neural Network 23
3.5 System Architecture 26
3.6 Architecture Diagram 28
3.7 Python-3.7.9 29

3.7.1 NumPy 31

3.7.2 OpenCV 34

3.7.3 Matplotlib 36

3.7.4 TensorFlow 36

3.7.5 Keras 38

xiv
 40
3.7.6 Tflearn
41
3.7.7 Sklearn
4. DESIGN AND IMPLEMENTATION 43
4.1 Algorithm Used
43
4.2 Gesture Classification 46
4.3 Working 50

5. CONCLUSION AND FUTURE ENHANCEMENT 52

5.1 Conclusion 52
51
5.2 Future enhancement

6 APPENDIX 55

7 REFERENCES 68

xv