© June 2024| IJIRT | Volume 11 Issue 1 | ISSN:

2349-6002

Dynamic hand gesture detector using python and open

CV

Lakshya Gaur1, Kapil Tomar2, Dr. Naveen Tyagi3

1
M.Tech Scholar, Department of Computer Science and Engineering, MIT, Bulandshahr
2
Assistant Professor, Department of Computer Science and Engineering, MIT, Bulandshahr
3
Professor, Department of Computer Science and Engineering, MIT, Bulandshahr

Abstract- Hand Gesture Recognition and Image

verbal cues, computers can discern user intent and
Overlay Using OpenCV and MediaPipe
This research paper presents a method for real-time
respond in a manner that feels more natural and
hand gesture recognition and image overlay using intuitive.
OpenCV and MediaPipe. The system captures live video The evolution of gesture-based interfaces has been
feed from a webcam, detects hand gestures, and propelled by advancements in a myriad of
overlays corresponding images based on the detected technologies, including computer vision, machine
gestures. The implementation leverages the capabilities learning, sensor technology, and augmented reality.
of MediaPipe for hand tracking and OpenCV for image
processing and display. The proposed method is
The main objectives of this project are:
efficient and runs in real-time, providing immediate
 To develop a real-time hand gesture recognition
feedback on detected gestures. This paper discusses the
system model, the underlying algorithm, and the results
system.
obtained from the implementation.  To overlay images based on detected gestures.
 To provide a seamless and responsive user
1. INTRODUCTION experience.

Hand gesture recognition is a crucial technology in 2. SYSTEM MODEL

the field of human-computer interaction. It allows
users to interact with digital devices using 2.1 The system comprises the following components:
natural hand movements, enhancing the user  Webcam: Captures live video feed.
experience. This project focuses on creating a hand  OpenCV: Handles video capture, image
gesture recognition system that identifies specific processing, and display.
gestures and overlays images accordingly. The  MediaPipe: Detects and tracks hand landmarks.
system uses Python, OpenCV, and MediaPipe,  Gesture Recognition: Identifies specific hand
leveraging their powerful image processing and gestures.
machine learning capabilities. The journey  Image Overlay: Overlays corresponding images
towards gesture-based interaction based on detected gestures.
represents a departure from the conventional notion
2.2 The flow of the system is as follows:
of computer interfaces as passive tools controlled
1. Capture live video feed using OpenCV.
solely through manual manipulation. Instead, it
2. Process each frame to detect hand landmarks
seeks to imbue technology with a deeper
using MediaPipe.
understanding of human intention and expression,
3. Identify gestures based on the positions of the
enabling seamless interaction that mirrors the
landmarks.
fluidity of human communication. At its core,
4. Overlay images on the video feed based on
gesture-based interaction draws inspiration from the
detected gestures.
rich tapestry of human movement, encompassing
5. Display the processed video feed in real-time.
gestures, postures, facial expressions, and even
subtle nuances of body language. By decoding
and interpreting these non-

IJIRT 165596 INTERNATIONAL JOURNAL OF INNOVATIVE RESEARCH IN TECHNOLOGY 1

 © June 2024| IJIRT | Volume 11 Issue 1 | ISSN:
2349-6002
3. ALGORITHM hands.process(rgb) if results.multi_hand_landmarks:
for hand_landmarks in
3.1 Initialization results.multi_hand_landmarks: lm_list = [] for lm
in hand_landmarks.landmark:
 Import necessary libraries. lm_list.append(lm) finger_fold_status = [] for tip in
 Initialize video capture using OpenCV. Finger_tips: x, y = int(lm_list[tip].x * w),
 Load images for overlay. int(lm_list[tip].y * h) cv2.circle(img, (x, y), 13, (0,
 Initialize MediaPipe for hand detection. 255, 0), cv2.FILLED) if lm_list[tip].x < lm_list[tip -
3].x: cv2.circle(img, (x, y), 13, (0, 0, 255),
3.2 Frame Processing cv2.FILLED) finger_fold_status.append(True) else:
For each frame captured from the webcam: finger_fold_status.append(False) if
1. Flip the frame horizontally for a mirror effect. all(finger_fold_status): if lm_list[thumb_tip].y <
2. Convert the frame to RGB format. lm_list[thumb_tip - 1].y < lm_list[thumb_tip - 2].y: h,
3. Detect hand landmarks using MediaPipe. w, c = like_img.shape img[0:h, 0:w] = like_img else:
4. Identify gestures by analyzing the positions of h, w, c = dislike_img.shape img[0:h, 0:w] =
landmarks. dislike_img if lm_list[Finger_tips[0]].x
5. Overlay images based on detected gestures. > lm_list[Finger_tips[0] - 1].x: h, w, c
6. Display the processed frame. = neutral_img.shape img[0:h, 0:w]
= neutral_img if lm_list[Finger_tips[0]].x <
3.3 Gesture Detection lm_list[Finger_tips[0] - 3].x: if
The algorithm identifies gestures by analyzing the lm_list[thumb_tip].x < lm_list[thumb_tip - 2].x: h,
positions of finger tips and thumb: w, c = dybala_img.shape img[0:h, 0:w] =
 LIKE: Thumb up gesture. dybala_img draw.draw_landmarks(img,
 DISLIKE: Thumb down gesture. hand_landmarks,
 NEUTRAL: No specific gesture. mpHands.HAND_CONNECTIONS)
 DYBALA CELEBRATION: Specific finger cv2.imshow('Hand Gesture Recognition', img) if
arrangement. cv2.waitKey(1) == ord('q'): break cap.release()
cv2.destroyAllWindows()
4. CODE IMPLEMENTATION
5. CONCLUSION
4.1 Initialization
python This project demonstrates an efficient and real-time
Copy code hand gesture recognition system using OpenCV and
import cv2 import numpy as np import mediapipe as MediaPipe. By leveraging these libraries, we
mp cap = cv2.VideoCapture(0) mpHands = achieved accurate hand tracking and gesture
mp.solutions.hands hands = mpHands.Hands() draw recognition, providing immediate visual feedback
= mp.solutions.drawing_utils # Load images for through image overlay. The system can be further
overlay like_img = cv2.imread('like.jpg') dislike_img enhanced with additional gestures and applications,
= cv2.imread('dislike.jpg') neutral_img making it a versatile tool for human-computer
= cv2.imread('neutral.jpg') dybala_img = interaction.
cv2.imread('dybala.jpg') Finger_tips = [8, 12, 16, 20]
thumb_tip = 4 REFERENCE

4.2 Frame Processing 1. OpenCV: Open Source Computer Vision

python Library. Available at: https://opencv.org/
Copy code 2. MediaPipe: Cross-platform Framework for
while True: success, img = cap.read() img = Building Multimodal Applied ML Pipelines.
cv2.flip(img, 1) h, w, c = img.shape rgb = Available at: https://mediapipe.dev/
cv2.cvtColor(img, cv2.COLOR_BGR2RGB) results = 3. Python: The Python Programming Language.
Available at: https://www.python.org/

IJIRT 165596 INTERNATIONAL JOURNAL OF INNOVATIVE RESEARCH IN TECHNOLOGY 2

 © June 2024| IJIRT | Volume 11 Issue 1 | ISSN:
2349-6002
Images to Include
1. Development Environment:
 Screenshot of the code editor with the project
open.
 Photo of the setup with the webcam.
Hand Detection Process:
 Raw webcam feed before processing.
 Hand landmarks detection with landmarks
highlighted.
Gesture Recognition Output:
 Examples of each gesture ("LIKE", "DISLIKE",
"NEUTRAL", "DYBALA CELEBRATION")
with the overlay image.
By incorporating these images and code snippets, this
thesis will effectively illustrate the project's
implementation and results.

IJIRT 165596 INTERNATIONAL JOURNAL OF INNOVATIVE RESEARCH IN TECHNOLOGY 3