*

CPCS432 Al
Lecture 8
Image Segmentation
Computer Vision
Applying Deep Learning Algorithms
for Image Segmentation

Dr. Arwa Basbrain

1
 Image Segmentation

What is image segmentation?

Image segmentation is a computer vision technique that partitions a
Image Segmentation

digital image into discrete groups of pixels—image segments—to

inform object detection and related tasks. By parsing an image’s
complex visual data into specifically shaped segments, image
segmentation enables faster, more advanced image processing.

CPCS432 Lecture 8
Dr. Arwa Basbrain
 Image Segmentation

Image segmentation Main categories

Image segmentation methods can be classified according to the tasks they aim
Image Segmentation

to accomplish. These tasks generally fall into three main categories:

1. Semantic segmentation,
2. Instance segmentation
3. Panoptic segmentation.

CPCS432 Lecture 8
Dr. Arwa Basbrain
 Image segmentation methods

Semantic Segmentation
In semantic segmentation, each pixel is classified into a class label, so the entire image is
segmented into categories like "car," "road," "sky," etc. This method does not distinguish
between separate instances of the same object.
Applications: Used in autonomous driving to understand road scenes, in AR for scene
understanding, and in satellite imaging for land classification.
Example: Labelling each pixel in a street image as road, sidewalk, building, or vehicle.

CPCS432 Lecture 8 5/11/2024 8

 Image segmentation methods

Instance Segmentation
Instance segmentation is similar to semantic segmentation, but it also distinguishes between
different instances of the same class. For example, it would label two cars as separate objects
rather than combining them into one "car" category.
Applications: Instance segmentation is critical in robotics and autonomous vehicles for
object-level understanding and interaction, and in retail (e.g., counting items on a shelf).
Example: In a street scene, marking each car as a distinct entity, rather than a general “car”
label for all cars.

CPCS432 Lecture 8 5/11/2024 9

 Image segmentation methods

Instance Segmentation

CPCS432 Lecture 8 5/11/2024 10

 Image segmentation methods

Panoptic Segmentation
Panoptic segmentation combines semantic and
instance segmentation. Each pixel is assigned a
semantic label, and each object instance is
uniquely labelled. This approach is holistic,
covering both things (objects with instances,
like cars) and stuff (regions without distinct
instances, like roads).
Applications: Used in complex scenes where
object interactions need to be fully understood,
such as in autonomous driving and augmented
reality
Example: In a driving scene, labeling roads,
cars, buildings, and trees with distinct
boundaries and semantic meaning.
CPCS432 Lecture 8 5/11/2024 11
 Use cases for image segmentation
Image segmentation has become an essential tool in a variety of fields.

1. Medical imaging: Image segmentation has many applications across radiography,

magnetic resonance imagery (MRI), ultrasounds and computer tomography (CT),
Image Segmentation

aiding tasks like tumor detection, brain segmentation, disease diagnosis and surgical
planning.
2. Autonomous vehicles: Image segmentation allows self-driving cars to avoid
obstacles like pedestrians and other cars, as well as identify lanes and traffic signs. It
is similarly used to inform navigation in robotics.
3. Satellite imaging: Semantic and instance segmentation automate the identification of
different terrain and topographical features.
4. Smart cities: Image segmentation powers tasks like real-time traffic monitoring and
surveillance.
5. Manufacturing: in addition to powering robotics tasks, image segmentation powers
product sorting and the detection of defects.
6. Agriculture: image segmentation helps farmers estimate crop yields and detect
weeds for removal.
 Image segmentation techniques
Image Segmentation Techniques are methods used to divide an image into multiple parts or
regions, each representing meaningful sections or objects. These techniques are essential in
computer vision tasks where the goal is to isolate specific objects or regions of interest within an
image, enabling more focused analysis and interpretation. Here’s a comprehensive overview of the
main types of image segmentation techniques, along with examples and applications for each.

Traditional image segmentation techniques use information from a pixel’s colour values (and
related characteristics like brightness, contrast or intensity) for feature extraction, and can be
quickly trained with simple machine learning algorithms for tasks like semantic classification.

Deep learning image segmentation models

Trained on an annotated dataset of images, the neural networks of deep learning image
segmentation models discover underlying patterns in visual data and discern the salient features
most relevant to classification, detection and segmentation.

CPCS432 Lecture 8 5/11/2024 13

 Image segmentation techniques
Traditional techniques
Common traditional (or "classic") image segmentation techniques include:

1. Thresholding techniques is a simple yet effective segmentation technique that involves

converting an image into binary form (foreground and background) based on pixel intensity
values. Thresholding works well when there’s a clear distinction between object and background
intensities. Types:
• Applications:
Document processing (separating text from background)
Medical imaging (detecting regions of high contrast, such as bones in X-rays)

CPCS432 Lecture 8 5/11/2024 14

 Image segmentation techniques
Traditional techniques
Common traditional (or "classic") image segmentation techniques include:

• Edge detection: Edge detection methods identify the boundaries of objects or classes by
detecting discontinuities in brightness or contrast.
• Edge-based segmentation detects boundaries between objects in an image by identifying
changes in pixel intensity, often representing the edges of objects. This method is commonly
used when objects have well-defined boundaries.

CPCS432 Lecture 8 5/11/2024 15

 Image segmentation techniques
Traditional techniques
• Watersheds: Watershed algorithms transform images into grayscale, then generate a
topographical map in which each pixel’s “elevation” is determined by its brightness. Regions,
boundaries and objects can be inferred from where “valleys”, “ridges” and “catchment basins”
form.

• Region-based segmentation: Starting with one or more “seed pixels”, region-growing

algorithms group together neighbouring pixels with similar characteristics. Algorithms can be
agglomerative or divisive.

CPCS432 Lecture 8 5/11/2024 16

 Image segmentation techniques
Traditional techniques

• Clustering-based segmentation: An unsupervised learning method, clustering algorithms divide

visual data into clusters of pixels with similar values. A common variant is K-means clustering,
in which k is the number of clusters

CPCS432 Lecture 8 5/11/2024 17

 ditione a
Not
Image segmentation techniques
Deep learning models

to
pivedrecode
Prominent deep learning models used in
image segmentation include:
Fully Convolutional Networks (FCNs):
FCNs, often used for semantic
segmentation, are a type of convolutional e
neural network (CNN) with no fixed
layers. An encoder network passes visual
input data through convolutional layers to
extract features relevant to segmentation
or classification, and compresses (or
downsamples) this feature data to remove
non-essential information. This L

a
compressed data is then fed into decoder
layers, upsampling the extracted feature
data to reconstruct the input image with
segmentation masks.
CPCS432 Lecture 8 5/11/2024 18
 Image segmentation techniques
Deep learning models
Fully Convolutional Networks (FCNs):
Pixel-wise Prediction
The pixel-wise prediction block at the end of the
network represents the output of the model. This
output is a segmentation mask where each pixel is
assigned a class label based on the highest
prediction confidence.
For example, pixels that belong to the dog, cat, and
background regions will each have a unique label in
the output mask.
Output Resolution: The output segmentation map u
has the same spatial dimensions as the input image,
allowing each pixel to have a class label. The our wi
segmentation mask (e.g., dog in purple, cat in Lay
brown, background in green) shows the model’s
predictions for each pixel, effectively identifying
the regions corresponding to each object class.
CPCS432 Lecture 8 5/11/2024 19
 Image segmentation techniques
Deep learning models
Prominent deep learning models used in image segmentation include:

U-Nets: U-Nets modify FCN architecture to reduce data loss during downsampling with skip
connections, preserving greater detail by selectively bypassing some convolutional layers as
information and gradients move through the neural network. Its name is derived from the shape of
diagrams demonstrating the arrangement of its layers.

Deeplab: Like U-Nets, Deeplab is a modified FCN architecture. In addition to skip connections, it
uses diluted (or “atrous”) convolution to yield larger output maps without necessitating additional
computational power.

CPCS432 Lecture 8 5/11/2024 20

 Image segmentation techniques
Deep learning models

Mask R-CNNs: Mask R-CNNs are a leading model for instance segmentation. Mask R-CNNs
combine a region proposal network (RPN) that generates bounding boxes for each potential
instance with an FCN-based “mask head” that generates segmentation masks within each
confirmed bounding box.

CPCS432 Lecture 8 5/11/2024 21

 Image segmentation techniques
Deep learning models

Transformers: inspired by the success of transformer models like GPT and BLOOM in natural
- -

language processing, new models like Vision Transformer (ViT) using attention mechanisms in
place of convolutional layers have matched or exceeded CNN performance for computer vision
tasks.

CPCS432 Lecture 8 5/11/2024 22