ARTIFICIAL INTELLIGENCE

MANUFACTURING ME6324
FINAL REVIEW

FRAMEWORKS USED:
TOOLS: FEED FORWARD NEURAL NETWORK AND IMAGE AUGMENTATION
LANGUAGES : PYTHON
RIDHI PUPPALA (ME15B133) VISHAL CHANDRAHAS (ME15B148)
 COLLECTION OF DATA
• Due to the novelty of the chosen problem statement there is no available
data both in research journals and online platform
• The image data set has been generated by taking sample images of
fasteners and other mechanical components using smart phone cameras
and the data is populated using image augmentation techniques for
training and testing over Neural Networks
• Images were taken in different perspectives, lighting conditions and
orientation of the components to simulate the dynamic orientations and
lighting conditions of components moving on a conveyor belt

Hex Bolt Countersunk Bolt Hex Nut Bearing T nut

 AUGMENTATION OF IMAGE DATA
• Tensor Flow and OpenCV(Python) were used to perform image
augmentation
• Augmentation involves a combination of affine transformations
and addition of white and gaussian noise, blurring and other CV
operations
• Images are resized from 4160 X 3120 to 100 X 100 pixels
• Then following Augmentation Operations are applied:
• Scaling
• Translation
• Rotation by coarse and fine angles
• Flipping
• Adding Salt and Pepper Noise
• Lighting Condition (Add Gaussian Noise)
• Perspective Transform
 Resized
Images
(100X100)

Rotation Transformation through multiples of 30°

Flip Up, Down,

and Transpose
Transformation
Perspective Transformation to
account for image distortion
Gaussian Noise for simulating
during movement on conveyors
different Lightning conditions
 TRAINING OF THE NEURAL NETWORK
• The neural network propagates through a weighted transformation with a bias represented as:
y = Wx + b ,where W is weight matrix while x is an array of input data with 784 X 1 dimension

• Back Propagation algorithm is

implemented for training the
chosen feed forward neural
network
• The algorithm calculates the new
weights based on the errors in
each of the layers by calculating
the gradients in weights and biases
• Then the new weights and biases are
updated after the back propagation
of error signals
Summary of
Single
perceptron NN
CUSTOM DEFINED PYTHON FUNCTIONS

Fig. Mean Squared Error (MSE) is chosen

as LOSS function for tuning alpha
 FEEDFORWARD FUNCTION
• Feedforward function is the heart of the neural network
• It takes input images, weight and bias as numpy arrays
• It operates over the network generated using makeNN() function
• H is activation while A is pre activation function for the neurons
 OPTIMIZATION AND MAIN FUNCTION
Main function: Loads weights and bias from training
sessions and runs feedforward to predict the label of the
test data and compute accuracy based on correct labels

Optimization Function: Performs several

iterations(epochs) of feedforward, back
propagation and weights update step
to train the network with the input data
 PYTHON NUMPY VS TENSORFLOW APPROACH

Comparision of the training, testing, data processing time for two different approaches

Operation Python Numpy Tensor Flow

Training ~350 seconds ~15 seconds Conditions
~ 30 seconds ~ 5 seconds 300 Iterations (Epochs)
Data Processing 300 Hidden Layer Neurons
Testing & 784 input layer Neurons
~ 5 seconds < 2 seconds 5 classes (5 output layers)
Implementation
Learning rate (alpha) = 0.005
No. of Lines of Code 180 30
TENSOR FLOW CODE

• Open source machine learning framework

• Flexible architecture allows easy
deployment of computation across variety
of platforms with inter-operability
• CPUs, GPUs, TPUs ,desktops, clusters of
servers, mobile and edge devices
• Inbuilt functions with highly optimized
performance and less computation time
• Save and Restore sessions of training allow
us to keep training and testing at any time

After rigorous tuning of the parameters,

the following were chosen:
• Learning Rate = 0.005
• No. of hidden layers = 1
• Hidden layer Neuron = 300
• Iteration (Epochs) = 300
• Input dataset size = 1500
• Input layer Neurons = 784
PERFORMANCE STATISTICS

Epochs: 100
Alpha: Variable
HLN : 300
Train Time : ~2.8s

Epochs: 100
Alpha: Variable
HLN : 500
Train Time : ~4.5s
EFFECT OF HLN AFTER SATURATION

Epochs: 200
Alpha: Variable
HLN : 500
Train Time : ~9s

Epochs: 200
Alpha: Variable
HLN : 300
Train Time : ~6s
REDUNDANCY OF HIGHER HLN AFTER SATURATION

Epochs: 300
Alpha: Variable
HLN : 300
Train Time : ~10s

Epochs: 300
Alpha: Variable
HLN : 500
Train Time : ~16s
 EFFECT OF TUNING PARAMETERS ON PERFORMANCE
Parameters : Learning Rate(alpha), HLN, ILN, No . Of classes, Train Dataset Size, Epochs
HLN
Epochs • Testing accuracy improves, but
• Testing accuracy improves, but saturates after a certain point
saturates after a certain point • Increases train time
• Increases train time proportionally • Capable of handling higher
volume of data
No. of Classes ILN
• Improves testing and training accuracy • Better prediction against higher variation
• Increases train time and volume of input data • Requires more data for better performance
• Requires more number of hidden layers and • Increases train time exponentially
efficient probability distribution function • Requires more number of hidden layers

Learning Rate (alpha)

• Training accuracy and prediction are highly sensitive to alpha
• Small αlpha implies slower gradient descent, large alpha leads to overshoot
of minimum LOSS and divergence, the cost function chosen was MSE
• Training started from a relatively large alpha and then alpha was
decreased during the training to allow more fine-grained weight updates.
 CROPPING AND GRAYSCALE CONVERSION
Why Cropping ? Why Grayscale Conversion ?
• Presence of too many white pixels in train • Feedforward network operates over
and test images single layer of perceptron
• Resizing of image from (4160X3120) to • RGB images have 3 channels which
(28,28) suppresses the features of the are a 3D matrix with depth
images making it difficult for the network • Input layer of this network has 784
to train and predict over the dataset neurons for each of the pixels
• When resized all the images except Hex • Each pixels is fed in as a feature
bolt shrinks into a blob of grey pixels • CNNs can be used for operating over
higher volumes of colour image
dataset for feature extraction

Fig. Incorrect prediction of

T - nut as a counter sunk
bolt when not cropped
 DISCUSSIONS ON PERFORMANCE RESULTS
• Initialization of arrays to array of zeros led to degraded training and prediction
• randn() function generates a random array over gaussian distribution
• Prediction accuracy is not consistent for a for a few images, while it is consistent around 90% for
large test image datasets
• Classification can be improved with higher volume of train data, better computational resources
(GPUs), using CNNs for convoluting over test data for specific feature extraction
CLASSIFICATION OF FADED & CROPPED IMAGES

Faded test images

Test images with the component slightly cropped out

RESULTS WITH HLN = 100
DEGRADED PERFORMANCE WITH LESS EPOCHS
CONTOUR DETECTION & CLASSIFICATION
 CONTOUR DETECTION - FEATURE EXTRACTION

• Contour Detection can be implemented post

classification by running specific OpenCV scripts to
search for features (feature extraction)
• Features can help identify the sub categories of the
family of components (ex: types of bearings, fasteners)
• Canny Edge detection & contour detection can be
deployed under calibrated environments for extracting
simple dimensional information like bore, OD, thread
length, flange dimensions and other dimensions
 SCHEMATIC OF AN INDUSTRIAL SORTING SYSTEM

Training Data
Computer and UI

Training Neural
Training Data Processing,
Network
Compression and Labelling
(CPU + GPU) PLC system for
automated tossing
Dynamic Training

Feed Forward NN
Classification &
Data Processing Classifier operating
Feature Extraction
over Trained Model
High Quality
Video Input
Record dimensional,
component and
feature information
 OUTCOMES
• Feed forward Neural Network Classifier for predicting the type of component
• Populated higher volumes of data through Image Augmentation
• Developed codes for Python Numpy and Tensor Flow approach
• Comparision of custom defined Python functions and Tensor Flow
• Possible methods for extracting features and dimensional information
• Proposed schematic for a complete setup of Industrial sorting system

LEARNING OUTCOMES
• Tensor Flow & Python (Numpy)
• Image Augmentation
• Structure of Neural Networks
• OpenCV Python