Volume 10, Issue 4, April – 2025 International Journal of Innovative Science and Research Technology

ISSN No:-2456-2165 https://doi.org/10.38124/ijisrt/25apr2374

Enhancing Model Accuracy for Keypoint-Based

Sign Language Recognition using Optimized
Neural Network Architectures
Kailash Kumar Bharaskar1; Dharmendra Gupta2; Vivek Kumar Gupta3;
Rachit Pandya4; Rachit Jain5
1,2,3
Department of Computer Science, Medi-Caps University, Indore,India
4,5
Student, Department of Computer Science, Medi-Caps University, Indore,India

Publication Date: 2025/05/17

Abstract: Sign language is a vital means of communication for millions, yet technological barriers still limit accessibility.
To address this, we analyzed the existing deep learning model and identified key areas for enhancement. We expanded the
neural network to improve learning capacity, replaced ReLU with LeakyReLU to avoid inactive neurons, and added batch
normalization to maintain gradient stability throughout training. To reduce overfitting while preserving performance, we
fine-tuned the dropout layer. We also improved preprocessing to filter out background noise, enhancing the system’s
ability to accurately track gestures. The training process was accelerated using early stopping and model checkpointing in
order to save the best-performing version possible without incurring unnecessary computation. The final leg was
converting the model to run in TensorFlow Lite, so that it would be able to run efficiently on mobile and edge devices and
hence making its real-world deployment possible. The results were demonstrative; greatly improved accuracy, enhanced
stability, and decent real-time performance. With confusion matrices and ROC curves backing it, the improvement is
measurable. But more importantly, this project is about inclusivity—what it means to bring people into technology more
finely on behalf of the community.

How to Cite: Kailash Kumar Bharaskar; Dharmendra Gupta; Vivek Kumar Gupta; Rachit Pandya; Rachit Jain (2025) Enhancing
Model Accuracy for Keypoint-Based Sign Language Recognition using Optimized Neural Network Architectures International
Journal of Innovative Science and Research Technology, 10(4), 4049-4055.https://doi.org/10.38124/ijisrt/25apr2374

I. INTRODUCTION not only faster—but also more accurate and reliable in the
real world. This research mainly aims to improve the
The sign language allows the deaf and hard-of-hearing accuracy and robustness of a keypoint-based sign language
community to convey complex thoughts, feelings, and recognition model.
concepts. But the lack of being familiar with sign language
leaves obstacles to communication in everyday life.  The Following Research Questions Guide Our Study:
Automated Sign Language Recognition SLR Automated
(SLR) is a significant development that utilizes machine  How can neural network optimization techniques improve
learning to convert visual gestures into text or speech, the accuracy of keypoint-based SLR models?
facilitating smoother communication. The adoption of deep  What impact do modifications such as batch
learning-based approaches has further augmented SLR normalization, LeakyReLU activations, and adjusted
systems, drastically enhancing their precision, speed, and dropout rates have on the model’s stability and
overall robustness. Deep learning models are very good at generalization capabilities?
identifying complicated patterns. The traditional SLR task
based on vision typically leverages CNNs and RNNs. But II. LITERATURE REVIEW
there’s an alternative — keypoint-based models. Rather
than analyzing entire images, they follow hand joint In the last 10 years, sign language recognition (SLR)
movements which makes them lightweight and quick. They has evolved significantly. Researchers progressed from
don’t require as much computing power and are better traditional methods — in which features such as those based
equipped to handle background noise, making them ideal for on hand shape, movement, and timing were manually
real-time usage. That said, they present challenges. extracted from the data — to deep learning techniques, in
Overfitting. Poor generalization. Our research addresses which patterns in the data were automatically learned. Early
these issues directly, we observed that this direct deployment systems had their moments, but they struggled with the
of architectural improvements would yield models that were complexity and variety of sign language as it is used in the

IJISRT25APR2374 www.ijisrt.com 4049

Volume 10, Issue 4, April – 2025 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165 https://doi.org/10.38124/ijisrt/25apr2374
real world. Gestures change. Context matters. And hand-
rolled features could only do so much. That’s where deep
learning came into the picture, providing more flexible and
scalable solutions.

The advent of deep learning has revolutionized the

field of sign language recognition (SLR) enabling models to
recognize gestures with remarkable accuracy. The CNNs are
particularly good at extracting visual details — but they have
their costs. They require enormous datasets and lots of
computing power, making real-time use more difficult,
especially on mobile devices. That’s where the keypoint-
based models play a role. Rather than full images, they report
only positions of hands and joints. It is quicker, lighter and
more adaptable. These models are better at handling different
lighting, backgrounds and occlusions — and since they are
less resource-heavy, they’re a practical fit for real-world use,
including on mobile and edge devices.

In a bid to comprehend the temporal nature of gestures

in a sign language, Recurrent Neural Networks (RNNs),
mostly Long Short Term Memory (LSTM) networks, have
become popular. They’re good at modeling sequences —
how one sign progresses into another. Such approaches are
ideal for continuous signing recognition, i.e., when there is a
context. But they have challenges, too. RNNs are slow to
train and have difficulty dealing with long sequences
because of the vanishing gradient problem. Without those
long frames, they may miss some important details, or lose
track of earlier gestures.

Fig 2 Model Architecture Diagram

Keypoint-based approaches take a more nuanced and

targeted approach. Rather than analyzing a whole image, they
track discrete points — such as joints in the hands and body
— to determine what gesture is being made.Think of it like
connecting the dots. Tools like OpenPose and MediaPipe
make this easier by pulling out those keypoints accurately,
even when the background is messy or the lighting isn’t
perfect. The big win? Less data to deal with. That means
faster, lighter models that still perform well. And because
they don’t need a ton of computing power, they’re a great fit
for real-world use—especially on phones and small devices
where speed and simplicity matter.

Fig 1 RNN Model Architecture for SLR

IJISRT25APR2374 www.ijisrt.com 4050

Volume 10, Issue 4, April – 2025 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165 https://doi.org/10.38124/ijisrt/25apr2374

Fig 3 System Architecture for SLR

Sign language recognition (SLR) has really grown over training time. It was a good starting point, but clearly had
the years. It started with simple machine learning models that room for improvement. In order to provide some
used handcrafted features and basic classifiers like SVMs. performance improvements several important enhancements
They got the job done—sort of—but struggled with the were made:
complexity of real-world gestures. Then deep learning came
along and changed the game. Early CNNs helped improve Increased Learning Power: Having more neurons in the
accuracy, but what really pushed things forward were dense layers could allow the model to learn more
techniques like dropout, batch normalization, and smarter comprehensive features of the gesture. LeakyReLU
activation functions. These made models not just better—but Activation: Traditional ReLU activation was swapped to
more stable, and actually usable outside the lab. In our work, LeakyReLU, to mitigate the "dying neurons", encourage
we’ve built on all of that. We didn’t just copy what came smoother gradient flow, and boost stability during training. In
before—we dug into it. Looked at what worked. What didn’t. each layer image, we implement Input Normalization which
Then focused on fine-tuning key parts of the model—like allows the training process work faster and better(Batch
choosing the right activation functions, applying better Normalization). Optimized Dropout: After tuning dropouts
normalization, and adjusting dropout levels. The result? A several results were obtained which clearly measured the
leaner, smarter network that learns faster and holds up better balance between overfitting and learning. Enhanced Data
in the real world. Processing: By adopting a more flexible data-loading
approach, we could better handle inconsistencies, and feature
III. RELATED WORKS standardization helped conform the features before learning.
Lean Deployment: The model was converted to TensorFlow
 Data Labelling and Structuring Lite leading to a decrease in size without sacrificing accuracy
The first version of the sign language recognition allowing the model to run on mobile and embedded device.
system was built using TensorFlow and Keras, with a focus
on keeping things simple and beginner-friendly. It worked by  Training
tracking 21 key points on the hand—so, 42 values for x and y Optimizer & Learning Rate: Adam optimizer (LR =
coordinates—and running them through a few dense layers. 0.001) based on the learning adaptable to data. Loss
ReLU was used for activation, and dropout helped reduce function: sparse categorical cross-entropy for multi-class
overfitting. The final softmax layer handled the actual gesture classification. Regularization: Dropout (35%) to prevent
classification, sorting each input into a specific sign language overfitting. Instability in training backpropagation of batch
category. Training was done using sparse categorical cross- normalization. Use Early Stopping to avoid computation
entropy with the Adam optimizer. It did the job—but just time. Tracked. Metrics: Loss, accuracy and gradient flow
barely. The model struggled with more complex patterns, through TensorBoard. Training Workflow: (Initialization:
overfitted easily, and wasn’t exactly efficient when it came to His initialization for stable weight distribution. Forward Pass:

IJISRT25APR2374 www.ijisrt.com 4051

Volume 10, Issue 4, April – 2025 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165 https://doi.org/10.38124/ijisrt/25apr2374
Processing keypoint data through the network. Loss ability to identify the essential characteristics necessary
Computation: Sparse categorical cross-entropy. Backward to make accurate predictions.
Pass & Weight Updates: Gradient updates via Adam  Good Generalization: The model was consistently
optimizer. Validation: Performance evaluation of epoch). performing well on the validation set, indicating that it
was not overfitting the training set and it could also
IV. RESULTS recognize new gestures which weren’t part of the training
dataset.
 The New Sign Language Recognition Model Improved  Slow Decreasing of Loss: The training and validation
Accuracy from 90.60% to 98.09% Proving the losses go down slowly over time which indicates that we
Classification Performance of Hand Gestures is Superior. were learning well. This was further supported by
Some Significant Observations from this Training techniques like dropout and batch normalization, which
Process Are: helped keep overfitting in check.
 Performance Comparison: Given in Fig 5
 Training Accuracy Improvement: Throughout the training
process, we observed that the model kept learning the
distinguishing patterns from the target data, displaying its

Fig 4 Confusion Matrix

IJISRT25APR2374 www.ijisrt.com 4052

Volume 10, Issue 4, April – 2025 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165 https://doi.org/10.38124/ijisrt/25apr2374

Table 1 Performance Comparison

IJISRT25APR2374 www.ijisrt.com 4053

Volume 10, Issue 4, April – 2025 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165 https://doi.org/10.38124/ijisrt/25apr2374

Fig 5 Epoch Diagram of Enhanced Model

FUTURE SCOPE
V. LIMITATIONS
 Future Research Should Address Several Critical Areas
 Although the Findings are Promising But Still We Have to Further Enhance Sign Language Recognition Systems:
Number of Limitations Which are Given Below:
 Continuous Sign Language Recognition: Developing
 Limited Dataset Diversity: The training dataset may not models that can handle continuous, fluid signing rather
represent the full spectrum of sign language variations than isolated gestures.
found globally.  Multi-Modal Data Integration: Combining keypoint data
 Isolated Gesture Focus: The current system is designed with other sensory inputs (e.g., video, audio, depth) for
for isolated gesture recognition, and additional work is richer feature representation.
needed to handle continuous signing.  Real-Time Deployment: Further optimizing the model for
 Real-World Variability: Factors such as occlusions, real-time applications on mobile and embedded devices
lighting variations, and background clutter in real-world through advanced compression techniques.
environments can affect performance.  Robustness to Environmental Variability: Enhancing the
 Computational Trade-Offs: Despite optimizations, further system’s resilience to changes in lighting, background,
work is needed to reduce model size and latency for and occlusions using adaptive learning techniques.
deployment on ultra-low-power devices.  Ethical and Regulatory Considerations: Establishing
frameworks for data privacy, user consent, and fairness to
VI. CONCLUSION guide the development and deployment of SLR systems.

In this work, we set out to build a smarter deep learning REFERENCES

model for keypoint-based sign language recognition—and it
paid off. By improving the network design, tightening up [1]. Goodfellow, I., Bengio, Y., & Courville, A. (2016).
how we trained it, and cleaning the data that went in, we saw Deep Learning. MIT Press.
a clear jump in performance over the original version. We [2]. Simonyan, K., & Zisserman, A. (2014). Very Deep
didn’t just guess—we ran experiments, adjusted the settings, Convolutional Networks for Large-Scale Image
and visualized the results to see what really worked. Simple Recognition. arXiv preprint.
tweaks like adding batch normalization, switching to [3]. Vaswani, A., et al. (2017). Attention is All You Need.
LeakyReLU, and fine-tuning dropout made a big difference Advances in Neural Information Processing Systems.\
in both accuracy and training stability. [4]. Abeer et al., "Deep Learning for Sign Language
Recognition: Current Techniques, Benchmarks, and
But this isn’t just about numbers on a chart. It’s about Open Issues," ResearchGate, 2021.ResearchGate
people. By making it easier for machines to understand sign [5]. SLR model-https://github.com/CodingSamrat/Sign-
language, we’re helping close the gap between deaf and Language-Recognition
hearing communities. It’s a step toward better
communication. And a more inclusive world.

IJISRT25APR2374 www.ijisrt.com 4054

Volume 10, Issue 4, April – 2025 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165 https://doi.org/10.38124/ijisrt/25apr2374
APPENDIX

 Workflow Diagram
The below diagram shows how the enhanced model works (see fig 7.).

Fig 6 Workflow Diagram of Model

 Data Preprocessing  Input(shape=(42,)),

 Dropout(0.2),
 Import numpy as np  Dense(24, activation='linear'),
 # using np.genfromtxt for robust data loading  LeakyReLU(alpha=0.1),
 Dataset = '/path/to/keypoint.csv'  BatchNormalization(),
 X_dataset = np.genfromtxt (dataset, delimiter=',',  Dropout(0.35),
dtype='float32', usecols=list (range (1, 43)))  Dense(12, activation='linear'),
 y_dataset = np.loadtxt(dataset, delimiter=',', dtype='int32',  LeakyReLU(alpha=0.1),
usecols=(0))  Dense(NUM_CLASSES, activation='softmax')
 # normalize keypoints
 X_dataset = (X_dataset - np.mean(X_dataset, axis=0)) /  Training Configuration
np.std(X_dataset, axis=0)
 Model. compile (optimizer='Adam',
 Model Architecture  loss='sparse_ categorical_ cross entropy',
 metrics=['accuracy'])
 import tensorflow as tf from tensorflow.keras.layers  History = model. Fit(Train, train,
 import Dense, Dropout, BatchNormalization,  validation split=0.25,
LeakyReLU, Input from tensorflow.keras.models import  epochs=50,
Sequential  batch size=32,
 model = Sequential([  callbacks=[early stopping, model checkpoint])

IJISRT25APR2374 www.ijisrt.com 4055