Computer Vision and Image processing

Chapter 3
Woldia University
IOT
March 2021
Outline
• Image Filtering
• Image Thresholding
Global thresholding,
Adaptive thresholding, and
Otsu’s binarization
• Transformation
• Concept of Blurring and Edge detection
 Question: Noise reduction

Give a camera and a still scene, how can you reduce noise?
Take lots of images and average them!

 What's the next best thing?

 Image Filtering

Filtering is a technique for modifying or enhancing an image.

E.g. you can filter an image to emphasize certain features or remove other
features.
Filtering is a neighborhood operation, in which the value of any given pixel
in the output image is determined by applying some algorithm to the values
of the pixels in the neighborhood of the corresponding input pixel.
A pixel's neighborhood is some set of pixels, defined by their locations
relative to that pixel.
Image Filtering
 Applications of Filtering

Enhance an image, e.g., denoise, resize.

Extract information, e.g., texture, edges.
Detect patterns, e.g., template matching.
 Noise reduction

Simplest thing: replace each pixel by the average of its neighbors.

This assumes that neighboring pixels are similar, and the noise to be
independent from pixel to pixel.
 Moving Average in 2D

Smoothing process over an image using averages.

 Linear and Nonlinear filters

Linear filtering is filtering in which the value of an output pixel is a linear

combination of the values of the pixels in the input pixel's neighborhood.
Filters that do not satisfy this property are non-linear.
 Linear and Nonlinear filters

What is Convolution?
Linear filtering of an image is accomplished through an operation called
convolution.
Convolution is a neighborhood operation in which each output pixel is the
weighted sum of neighboring input pixels.
The matrix of weights is called the convolution kernel, also known as the
filter. A convolution kernel is a correlation kernel that has been rotated
180 degrees.
 Linear and Nonlinear filters

Convolution
For example, suppose the image is
A = [ 17 24 1 8 15
23 5 7 14 16
4 6 13 20 22
10 12 19 21 3
11 18 25 2 9]

and the convolution kernel is

h=[8 1 6
3 5 7
4 9 2]
 Linear and Nonlinear filters

The following figure shows how to compute the (2,4) output pixel using
these steps:
1. Rotate the convolution kernel 180 degrees about its center element.
2. Slide the center element of the convolution kernel so that it lies on top of
the (2,4) element of A.
3. Multiply each weight in the rotated convolution kernel by the pixel of A
underneath.
4. Sum the individual products from step 3.
 Linear and Nonlinear filters

Hence the (2,4) output pixel is:

 Computing the (2,4) Output of Convolution.

 Linear and Nonlinear filters

The Median Filter is a non-linear digital filtering technique, often used to

remove noise from an image or signal. Such noise reduction is a typical pre-
processing step to improve the results of later processing (for example, edge
detection on an image). Median filtering is very widely used in digital
image processing because, under certain conditions, it preserves edges
while removing noise, also having applications in signal processing.
In application, the median filter removes outliers and shot noise that is
independent of magnitude, while mean filter serves as smoothing purpose.
 Linear and Nonlinear filters

Nonlinear filters are those for which the linearity relationship breaks
down.
Consider two signals A and B, for linear filter such as mean filter Fm,
you have:
Fm (A + λB) = Fm (A) + λFm (B),
but the equation is not satisfied for a nonlinear filter such as the median
filter.
 Linear and Nonlinear filters

Example:
 Linear and Nonlinear filters

To understand the linearity property more easily. Let us consider the above
diagram, here we have 2 sequences namely Xn and Yn. when we add both
the sequence we get Xn+Yn whose amplitude value are represented with
blue color. when any system which satisfy this condition then it is called
linear.
In case of mean filter, mean value for sequence Xn is 1+1+3/3=5/3. Mean
value for sequence Yn is 1+2+0/3=1. Mean value for Xn+Yn is
2+3+3/3=8/3.
 Linear and Nonlinear filters

mean(Xn)+mean(Yn)=mean(Xn+Yn)
5/3+1=8/3
hence we called mean filter as linear filter.
 Linear and Nonlinear filters

In case of median filter, if we calculate median value for sequence Xn, we get
1 (arrange the sequence in ascending order and then find the middle value).
similarly median value of sequence Yn is 1 .
median value of sequence Xn+Yn is 3.
median(Xn) + median(Yn) ≠ median(Xn+Yn).
1+1 ≠ 3
hence we call median filter as non-linear filter
 Median filter

It smoothens the image by utilising neighbouring of pixels.

It is non-linear filter.
It removes black and white noise from image.
 Median filter

Questions: Compute median value of marked pixels.

18 22 33 25
34 128 24 172
22 19 32 31
Write the value in ascending order.
Marked value should be in mid place or at centre. Use a 3x3 filter.
 Median filter

Median value for 128.

18 22 33
34 128 24
22 19 32
18 19 22 22 24 32 33 34 128
Remove one value from the left and one from right till last.
 Median filter

18 22 33
34 24 24
22 19 32
 Median filter

Median value for 24:

22 33 25
24 24 172
19 32 31

19 22 24 24 25 31 32 33 172
 Median filter

22 33 25
24 25 172
19 32 31
Finally we will have an output image as bellow,
18 22 33 25
34 24 25 172
22 19 32 31
 Image Thresholding

Goals: You will learn;

Global thresholding, Adaptive thresholding, Otsu’s binarization etc..
Functions such as, cv2.threshold, cv2.adaptiveThreshold etc
 Simple Thresholding

If pixel value is greater than a threshold value, it is assigned one value (may
be white), else it is assigned another value (may be black).
The function used is “cv2.threshold’’.
First argument is the source image, which should be a grayscale image.
Second argument is the threshold value which is used to classify the pixel
values.
Third argument is the maxVal which represents the value to be given if pixel
value is more than (sometimes less than) the threshold value.
 Simple Thresholding

OpenCV provides different styles of thresholding and it is decided by the

fourth parameter of the function. Different types are:
o cv2.THRESH_BINARY
o cv2.THRESH_BINARY_INV
o cv2.THRESH_TRUNC
o cv2.THRESH_TOZERO
o cv2.THRESH_TOZERO_INV
Simple Thresholding
Simple Thresholding
 Adaptive Thresholding

In Simple Thresholding, we used a global value as threshold value.

But it may not be good in all the conditions where image has different
lighting conditions in different areas.
In that case, we go for adaptive thresholding.
 Adaptive Thresholding

In this, the algorithm calculate the threshold for a small regions of the image.
So we get different thresholds for different regions of the same image and it
gives us better results for images with varying illumination.
It has three ‘special’ input parameters and only one output argument.
 Adaptive Thresholding

Adaptive Method - It decides how thresholding value is calculated.

o cv2.ADAPTIVE_THRESH_MEAN_C : threshold value is the mean of
neighborhood area.
o cv2.ADAPTIVE_THRESH_GAUSSIAN_C : threshold value is the weighted
sum of neighborhood values where weights are a gaussian window.
Block Size - It decides the size of neighborhood area.
C - It is just a constant which is subtracted from the mean or weighted mean
calculated.
Global thresholding vs Adaptive thresholding
 Global thresholding vs Adaptive thresholding

 Result
 Otsu’s Binarization

In global thresholding, we used an arbitrary value for threshold value.

So, how can we know a value we selected is good or not?
Answer is, trial and error method.
But consider a bimodal image (In simple words, bimodal image is an image
whose histogram has two peaks).
Otsu’s Binarization
 Otsu’s Binarization

For that image, we can approximately take a value in the middle of those
peaks as threshold value.
That is what Otsu binarization does.
So in simple words, it automatically calculates a threshold value from image
histogram for a bimodal image. (For images which are not bimodal,
binarization won’t be accurate.)
 Otsu’s Binarization

For this, our cv2.threshold() function is used, but pass an extra flag,
cv2.THRESH_OTSU.
For threshold value, simply pass zero.
Then the algorithm finds the optimal threshold value and returns you as the
second output, retVal.
If Otsu thresholding is not used, retVal is same as the threshold value you
used.
Otsu’s Binarization
Otsu’s Binarization
 Geometric Transformations of Images

 Goals:
 Learn to apply different geometric transformation to images like translation, rotation,
scaling, reflection, affine transformation etc.
 Using a function: cv2.getPerspectiveTransform
 Transformations

 OpenCV provides two transformation functions “cv2.warpAffine” and

“cv2.warpPerspective” with which you can have all kinds of transformations.
 cv2.warpAffine takes a 2x3 transformation matrix while cv2.warpPerspective takes
a 3x3 transformation matrix as input.
 2D Transformations

Two dimensional (2D)

point Transformations is a word that
1. Translation
2. Scaling describes the movement of a point or an
3. Rotation object around the cartesian plane.
4. Reflection
 Translation

Suppose a 2D point is translated/moved by an amount a in the x-direction

and b in the y-direction then the matrix used to represent this is:
 Translation

 Translation is the shifting of object’s location.

 If you know the shift in (x,y) direction, let it be (tx,ty), you can create the transformation
matrix M as follows:

 You can take make it into a Numpy array of type “np.float32” and pass it into
“cv2.warpAffine()” function.
 Let you see the bellow example for shifting the image by (35,50)
Translation
Translation
 Rotation

Suppose a 2D point is rotated about (0,0) anti-clockwise by an angle Ɵ

then the matrix used to represent this is:
 Rotation

Rotation of an image for an angle Ɵ is achieved by the transformation

matrix of the form:

To find this transformation matrix, OpenCV provides a function,

“cv2.getRotationMatrix2D”. Check below example which rotates the
image by 90 degree with respect to center without any scaling.
Rotation
Rotation
 Scaling

 Suppose a 2D point is scaled by an amount s in x-direction and t in the y-direction then

the matrix used to represent this is:
 Scaling

 Scaling is just resizing of the image.

 OpenCV comes with a function cv2.resize() for this purpose.
 The size of the image can be specified manually, or you can specify the scaling factor.
 Different interpolation methods are used. Preferable interpolation methods are
“cv2.INTER_AREA” for shrinking and “cv2.INTER_CUBIC” (slow) &
“cv2.INTER_LINEAR” for zooming.
 By default, interpolation method used is “cv2.INTER_LINEAR” for all resizing
purposes. See the below code:
Scaling
 Reflection

 Suppose a 2D point is reflected about the y-axis then the matrix used to represent this is:
 Applying transformations to 2D points

 All of the transformation matrices are 3x3.

 Transformation of 2D points are done by matrix multiplication.
 Applying transformations to 2D points

When transforming a 2D point it should be converted into a 3D column

vector.
Q. What should (2,6) be converted to?
 Applying transformations to 2D points

 Q1. Using the transformation matrix, translate the point (2,6) by 4 in the x-direction and
by -3 in the y-direction.
 Applying transformations to 2D points

 Q1. Using the transformation matrix, translate the point (2,6) by 4 in the x-direction and
by -3 in the y-direction.
 Applying transformations to 2D points

 Q2. Using a 3D transformation matrix, rotate the point (2,6) by 90 degrees anti-
clockwise about (0,0).
 Applying transformations to 2D points

 Q2. Using a 3D transformation matrix, rotate the point (2,6) by 90 degrees anti-
clockwise about (0,0).
 Applying transformations to 2D points

 Q3. Using a 3D transformation matrix, firstly, translate the point (2,6) by 4 in the x-
direction and -3 in the y-direction and then rotate it anti-clockwise by 90 degrees about
(0,0).
 Applying transformations to 2D points

 Q3. Using a 3D transformation matrix, firstly, translate the point (2,6) by 4 in the x-
direction and -3 in the y-direction and then rotate it anti-clockwise by 90 degrees about
(0,0).
 Applying transformations to 2D points

 Q4. The triangle Q is defined by the points a(2,6), b(2,10), and c(6,8).
Using a 3D combo matrix, apply the transformation to Q which:
firstly, translates Q by 4 in the x direction and -3 in the y direction and then rotates it anti-
clockwise by 90 degrees about (0,0).
 Applying transformations to 2D points

 Q4. The triangle Q is defined by the points a(2,6), b(2,10), and c(6,8).
Using a 3D combo matrix, apply the transformation to Q which:
firstly, translates Q by 4 in the x direction and -3 in the y direction and then rotates it anti-
clockwise by 90 degrees about (0,0).
 Applying transformations to 2D points

 Q4. The triangle Q is defined by the points a(2,6), b(2,10), and c(6,8).
Using a 3D combo matrix, apply the transformation to Q which:
firstly, translates Q by 4 in the x direction and -3 in the y direction and then rotates it anti-
clockwise by 90 degrees about (0,0).
 Scaling and reflection transformations

 Q5. Scale the line joining the points (2,6) and (6,8) by 2 in the x-direction and 0.5 in the
y-direction.
 Scaling and reflection transformations

 Q5. Scale the line joining the points (2,6) and (6,8) by 2 in the x-direction and 0.5 in the
y-direction.
 Scaling and reflection transformations

 Q6. Using a 3D transformation matrix reflect the point (2,6) in the x-axis.
 Affine Transformation

Affine transformation is a linear mapping method that preserves points,

straight lines, and planes.
In affine transformation, all parallel lines in the original image will still be
parallel in the output image.
The affine transformation technique is typically used to correct for
geometric distortions or deformations that occur with non-ideal camera
angles.
 Affine Transformation

 E.g. satellite imagery uses affine transformations to correct for wide angle lens distortion,
panorama stitching, and image registration.
 Image registration is an image processing technique used to align multiple scenes into a
single integrated image. It helps overcome issues such as image rotation, scale, and skew
that are common when overlaying images.
 Image registration is often used in medical and satellite imagery to align images from
different camera sources. Digital cameras use image registration to align and connect
adjacent images into a single panoramic image.
 Affine Transformation

Panorama image stitching

Registering aerial photos using point mapping
 Affine Transformation

Automatic registration using feature matching.

 Affine Transformation

To find the transformation matrix, we need three points from input image
and their corresponding locations in output image.
“cv2.getAffineTransform” will create a 2x3 matrix which is to be passed
to “cv2.warpAffine”
Affine Transformation
Affine Transformation
 //End…//
References..
“Digital Image Processing, Rafael C. Gonzalez”
“Computer Vision: Algorithms and Applications”, Richard Szeliski