Scribd
Upload
52 views

Deep Learning 2017 Lecture5CNN

The document summarizes a technical seminar presentation on convolutional neural networks (CNNs). It begins with an introduction and literature review on CNNs. It then discusses the basics of CNNs, including how they use fewer parameters than fully connected networks by sharing weights across filters and using convolutional and pooling layers. CNNs are trained by learning the weights of the filters via backpropagation. The document provides examples of convolutions and max pooling. In summary, it outlines the key concepts of CNNs for image classification and recognition tasks.

Uploaded by

RHYTHM BHATNAGAR
Copyright
© © All Rights Reserved
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
52 views

Deep Learning 2017 Lecture5CNN

The document summarizes a technical seminar presentation on convolutional neural networks (CNNs). It begins with an introduction and literature review on CNNs. It then discusses the basics of CNNs, including how they use fewer parameters than fully connected networks by sharing weights across filters and using convolutional and pooling layers. CNNs are trained by learning the weights of the filters via backpropagation. The document provides examples of convolutions and max pooling. In summary, it outlines the key concepts of CNNs for image classification and recognition tasks.

Uploaded by

RHYTHM BHATNAGAR
Copyright
© © All Rights Reserved
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 30

|| Jai Sri Gurudev ||

SJB Institute of Technology


#67, BGS Health & Education City, Dr. Vishnuvardhan Road, Kengeri, 
       Bengaluru – 560060, KARNATAKA, INDIA

Technical Seminar –01


Convolutional Neural Network (CNN)
Presented By

Student’s name       [USN] Under the guidance of


Rishabh Sharma 1JB18CS117 Dr. Krishna A N
Head of Department
Computer Science and Engineering
S J B INSTITUTE OF TECHNOLOGY
TABLE OF CONTENTS

INTRODUCTION

LITERATURE SURVEY

MOTIVATION AND PROBLEM STATEMENT

BASICS OF CNN

TRAINING OF CNN

APPLICATION OF CNN
CONVOLUTIONAL NEURAL NETWORK (CNN)
• We know it is good to learn a small model.
• From this fully connected model, do we really need all the edges?
• Can some of these be shared?
LITERATURE SURVEY
Author Title Year Technique Learning

Sakshi Indolia, Anil Conceptual 2018 CNN(Convolutional neural About Convolutional


kumar swami,pooja as understanding of cnn network ) neural network and its
applications

Neha sharma,anuj An analysis of CNN for 2020 Convolutional neural A Detailed Study on
Mishra , vibhor jain image classification network Convolutional neural
network and its Types.

Liang Gonog1,2 and A Review: Convolutional 2021 Convolutional neural Basics of CNN, Training of
Yimin Zhou1 neural network network CNN, Usage of the CNN
CONSIDER LEARNING AN IMAGE:

• Some patterns are much smaller than the whole image


Can represent a
small region with
fewer parameters

“beak” detector
SAME PATTERN APPEARS IN DIFFERENT PLACES:
THEY CAN BE COMPRESSED!
WHAT ABOUT TRAINING A LOT OF SUCH “SMALL”
DETECTORS
AND EACH DETECTOR MUST “MOVE AROUND”.

“upper-left beak”
detector

They can be compressed


to the same parameters.

“middle beak”
detector
A CONVOLUTIONAL LAYER
A CNN is a neural network with some convolutional layers
(and some other layers). A convolutional layer has a number
of filters that does convolutional operation.

Beak detector

A filter
CONVOLUTION
These are the network
parameters to be learned.

1 -1 -1
1 0 0 0 0 1 -1 1 -1 Filter 1
0 1 0 0 1 0 -1 -1 1
0 0 1 1 0 0
1 0 0 0 1 0 -1 1 -1
-1 1 -1 Filter 2
0 1 0 0 1 0
0 0 1 0 1 0 -1 1 -1

…
…
6 x 6 image
Each filter detects a
small pattern (3 x 3).
1 -1 -1
CONVOLUTION -1 1 -1 Filter 1
-1 -1 1
stride=1

1 0 0 0 0 1 Dot
product
0 1 0 0 1 0 3 -1
0 0 1 1 0 0
1 0 0 0 1 0
0 1 0 0 1 0
0 0 1 0 1 0

6 x 6 image
1 -1 -1
CONVOLUTION -1 1 -1 Filter 1
-1 -1 1
If stride=2

1 0 0 0 0 1
0 1 0 0 1 0 3 -3
0 0 1 1 0 0
1 0 0 0 1 0
0 1 0 0 1 0
0 0 1 0 1 0

6 x 6 image
1 -1 -1
CONVOLUTION -1 1 -1 Filter 1
-1 -1 1
stride=1

1 0 0 0 0 1
0 1 0 0 1 0 3 -1 -3 -1
0 0 1 1 0 0
1 0 0 0 1 0 -3 1 0 -3
0 1 0 0 1 0
0 0 1 0 1 0 -3 -3 0 1

6 x 6 image 3 -2 -2 -1
-1 1 -1
CONVOLUTION -1 1 -1 Filter 2
-1 1 -1
stride=1
Repeat this for each filter
1 0 0 0 0 1
0 1 0 0 1 0 3 -1 -3 -1
-1 -1 -1 -1
0 0 1 1 0 0
1 0 0 0 1 0 -3 1 0 -3
-1 -1 -2 1
0 1 0 0 1 0 Feature
0 0 1 0 1 0 -3 -3 Map0 1
-1 -1 -2 1
6 x 6 image 3 -2 -2 -1
-1 0 -4 3
Two 4 x 4 images
Forming 2 x 4 x 4 matrix
COLOR IMAGE: RGB 3 CHANNELS

11 -1-1 -1-1 -1-1 11 -1-1


1 -1 -1 -1 1 -1
-1-1 11 -1-1 -1-1-1 111 -1-1-1 Filter 2
-1 1 -1 Filter 1 -1 1 -1
-1-1 -1-1 11 -1 1 -1
-1 -1 1 -1 1 -1
Color image
1 0 0 0 0 1
1 0 0 0 0 1
0 11 00 00 01 00 1
0 1 0 0 1 0
0 00 11 01 00 10 0
0 0 1 1 0 0
1 00 00 10 11 00 0
1 0 0 0 1 0
0 11 00 00 01 10 0
0 1 0 0 1 0
0 00 11 00 01 10 0
0 0 1 0 1 0
0 0 1 0 1 0
Convolution v.s. Fully Connected

1 0 0 0 0 1 1 -1 -1 -1 1 -1
0 1 0 0 1 0 -1 1 -1 -1 1 -1
0 0 1 1 0 0 -1 -1 1 -1 1 -1
1 0 0 0 1 0
0 1 0 0 1 0
0 0 1 0 1 0
convolution
image

x1
1 0 0 0 0 1
0 1 0 0 1 0 x2
Fully- 0 0 1 1 0 0
1 0 0 0 1 0
connected
…
…

…
…
0 1 0 0 1 0
0 0 1 0 1 0
x36
1 -1 -1 Filter 1 1 1
-1 1 -1 2 0
-1 -1 1 3 0
4 0 3
:

…
1 0 0 0 0 1
0 1 0 0 1 0 0
0 0 1 1 0 0 8 1
1 0 0 0 1 0 9 0
0 1 0 0 1 0 10: 0

…
0 0 1 0 1 0
13 0
6 x 6 image
14 0
fewer parameters! 15 1 Only connect to 9
16 1 inputs, not fully
connected
…
THE WHOLE CNN

cat dog ……
Convolution

Max Pooling
Can repeat
Fully Connected many
Feedforward network
Convolution times

Max Pooling

Flattened
MAX POOLING

1 -1 -1 -1 1 -1
-1 1 -1 Filter 1 -1 1 -1 Filter 2
-1 -1 1 -1 1 -1

3 -1 -3 -1 -1 -1 -1 -1

-3 1 0 -3 -1 -1 -2 1

-3 -3 0 1 -1 -1 -2 1

3 -2 -2 -1 -1 0 -4 3
WHY POOLING

bird
bird

• Subsampling pixels will not change the object


Subsampling

We can subsample the pixels to make image smaller


fewer parameters to characterize the image
A CNN COMPRESSES A FULLY CONNECTED
NETWORK IN TWO WAYS:

• Reducing number of connections


• Shared weights on the edges
• Max pooling further reduces the complexity
MAX POOLING

New image
1 0 0 0 0 1 but smaller
0 1 0 0 1 0 Conv
3 0
0 0 1 1 0 0 -1 1
1 0 0 0 1 0
0 1 0 0 1 0 Max 3 1
0 3
0 0 1 0 1 0 Pooling
2 x 2 image
6 x 6 image
Each filter
is a channel
THE WHOLE CNN

3 0
-1 1 Convolution

3 1
0 3
Max Pooling
Can repeat
A new image
many
Convolution times
Smaller than the original
image
The number of channels Max Pooling

is the number of filters


THE WHOLE CNN

cat dog ……
Convolution

Max Pooling

Fully Connected A new image


Feedforward network
Convolution

Max Pooling

Flattened A new image


3
FLATTENING
0

1
3 0
-1 1 3

30 1 -1
3 Flattened
Fully Connected
1 Feedforward network

3
Only modified the network structure and input
CNN in Keras format (vector -> 3-D tensor)

input

Convolution
1 -1 -1
-1 1 -1
-1 1 -1
-1 1 -1 … There are
-1 -1 1 25 3x3
-1 1 -1 … Max Pooling
filters.
Input_shape = ( 28 , 28 , 1)

28 x 28 pixels 1: black/white, 3: RGB Convolution

3 -1 3 Max Pooling

-3 1
Only modified the network structure and input
CNN in Keras format (vector -> 3-D array)

Input
1 x 28 x 28

Convolution
How many parameters for
each filter? 9 25 x 26 x 26

Max Pooling
25 x 13 x 13

Convolution
How many parameters 225=
for each filter? 50 x 11 x 11
25x9
Max Pooling
50 x 5 x 5
ALPHAGO

Next move
Neural
(19 x 19
Network positions)

19 x 19 matrix
Fully-connected feedforward network
Black: 1
can be used
white: -1
none: 0 But CNN performs much better
ALPHAGO’S POLICY NETWORK

The following is quotation from their Nature article:


Note: AlphaGo does not use Max Pooling.
CNN IN SPEECH RECOGNITION

The filters move in the


CNN frequency direction.
Frequency

Image Time
Spectrogram
CNN IN TEXT CLASSIFICATION

Source of image: http://citeseerx.ist.psu.ed


u/viewdoc/download?doi=10.1.1.703.6858
&rep=rep1&type=pdf
THANK YOU

You might also like