Taiz University
Faculty of Engineering & IT
MRE department
AUTOMATIC INSPECTION SYSTEM FOR
DEFECTIVE WAFER BASED ON COMPUTER VISION
IN PRODUCTION LINE
DONE BY
MOHAMMED AHMED MOHAMMED MHYOOB
AMRAN NAJI MOHAMED MAHYOOB
MOHAMMED ABDULJALIL QASSIM ABDELLAH
MAZEN GHALIB ABDULRHMAN MOHSEN
ABDULMAGEED ADNAN MOHAMMED ABDULLAH
SUPERVISOR
PHD. NASHWAN ALKHOLIDI
CHAPTER 1: INTRODUCTION
Overview
ABSTRACT
Problem statement
The importance of the problem
Objective
main objective
sub-objectives
Hypotheses
Limits
deep learning algorithms
Python language
food product, which is Wafer
CHABTER 2: LITERATURE REVIEW
Pulivarthi S R et al (2010)
M.Z. Abdullah et al (2011)
Arjenaki et [Link] (2013)
A T Fadhil et al (2020)
Inspection
Inspection types
Manual visual inspection
Automated visual inspection
Semi-automated visual inspection
Chapter 3 : Deep Learning And
Neural
Networks
Deep learning
:
is a subset of machine learning that focuses on artificial
neural networks and algorithms inspired by the structure
and function of the human brain, Deep learning has
achieved remarkable success in various fields, including
computer vision, natural language processing, speech
recognition. It has been applied in tasks like image
classification, object detection, language translation and
.autonomous driving
Convolutional Neural
Network :
CNN
Feature Learning
CLASSIFICATION
INPUT HIDDEN OUTPUT INPUT
LAYER CONVOLUT
LAYER LAYER LAYER IONAL
LAYER
CNN architecture :
Convolutional Layer and Feature
Extraction :
Pooling Layers :
Flatten :
Fully Connected Layer
:
Input
layer
Hidden
layer
Output
layer
Hidden Layer and The
Feedforward Process :
:Error Function
is a mathematical function used to measure the difference
between the predicted output and the true or desired output
Optimization
:Optimization refers to the process of finding the best
set of model parameters that minimize the error
function. The goal of it is to improve the performance
of the model
CHAPTER 4:
SOFTWARE AND
HARDWRE
COMPONENTS
overview
this chapter discusses the requirements were used to in this project,
these include a few combinations of software and hardware
components are necessary for training of models, tools for preparing
of data and Arduino microcontroller implementation.
Software Components
Libraries Environment Tool
Tensorflow Jupyter Notebook LabelImg
Keras Google Colaboratory
OpenCV Pycharm
Programming Language
Numpy Arduino IDE
Python
PROJECT PROTOTYPE
Prototype Hardware
•Arduino Uno
•DC motor
•Servo Motor
•Camera
•Laptop Proprieties
Arduino Uno
DC motor
Servo Motor
Camera
Laptop Proprieties
Chapter 5: Methodology
1. Dataset
2. YOLO model
3. Classification models( keras
models )
4. Train modeals
Dataset
Data Source:
The data adopted in the project is obtained
from a Weafer Factory
Generating annotation files :
This for only object detection models
LabelImg tool
YOLO model
YOLO (You Only Look Once) is a popular object detection
algorithm that uses neural networks to predict bounding
boxes and class probabilities for objects in an image.
Anchor Boxes
Anchor Box Regression
Non-Maximum Suppression
YOLOv5
The YOLOv5 architecture is
made up of three main parts
which include the Backbone,
Neck, and Head.
C3
SPPF
concat
Classification models
MobileNetV2
VGG16
MobileNetV2:
Introduced in
2018
Inverted Residual block
Depthwise layer
VGG16:
Introduced in 2014
Train models
training of classification training
of YOLO model
models Learn_rate=0.01
Learn_rate=0.0001 Epochs= 100
Epochs= 100 Bitch_size=32
Bitch_size=16
Img_width=640
Img_width=224
Img_height=640
Img_height=224
CHABTER 6:
RESULTS AND
Discussion
Confusion Matrices
A confusion matrix is used to evaluate
the performance of a machine learning
or deep learning model on a
classification problem. It is a useful tool
for understanding how well the models
are able to distinguish between
different classes.
[Link] Positive (TP)
2. True Negative (TN)
3. False Positive (FP)
4. False Negative (FN)
Accuracy
Precision
Recall
F1-Score
Classification models result analysis
After the model is trained with 100 epochs for perfecting
learning the shape of the weafer's normal and defective, it is
ready to test with testing data. three different keras models
(Mobilenetv2, Inceptionv3, VGG16) provide different
acceptable accuracy using the evaluate() method that gave
the accuracies of 99%, 98% and 97% for Mobilenetv2,
Inceptionv3 and VGG16, respectively
Also, confusion matrix was used to
measure the performance of these models
Object detection model Result Analysis
After training YOLOv5 model for 100 epochs
the model is evaluated on a separate testing dataset to
.assess its performance
The evaluation typically involves measuring metrics such
as mean average precision (mAP), which quantifies the
accuracy of object detection, and average recall, which
measures the model's ability to detect objects across
.different scales and aspect ratios
Results Of Yolov5 On Testing Dataset
Confusion matrix of YOLOv5
Performance curves of YOLOv5
,Precision-Recall curve )a(
,Precision-confidence curve )b(
,F1- confidence curve )c(
.Recall- confidence curve )d(
Results of detection of YOLOv5 in testing dataset
Discussion
Tree different CNN models (Mobilenetv2,
Inceptionv3, VGG16) have been used in this
implemented approach which provided four
different accuracies, precision, recall, and f1 score
the best performing algorithm is Mobilenetv2 is 97%
accuracy
And the YOLOV5 achieves high detection accuracy
93% mAP
In real time test we found that YOLOv5 models were
more accurate than CNN models, but they were also
slower
CONCLUSION
Computer vision systems, also known as
intelligent machine vision, play a crucial role in
detecting manufacturing errors and ensuring
high-quality products are delivered to customers
The proposed machine vision scenario involves
capturing a high-definition image of the finished
product using a camera installed at the end of
the processing line
The image identification algorithm uses a
database that includes labeled sample images of
Future work
Our project aims to build an
automated inspection model for
wafer products using deep learning
algorithms
Firstly, we can focus on collecting
more data
Lastly, we can consider utilizing
