CAB420 Assignment 1B Question 1: Template and Utilities Demo

Overview
This notebook provides a quick demo and overview of the provided utility functions to help with Assignment 1B, Question 1.

Please make sure you read the assignment brief on canvas, and check the FAQ for other information.

Note: File paths used in this template may need to change for your local machine. Please set these based on your local file system
structure.

Utility Functions
The following cell contains utility functions to:

Load the data

Split the training set into training and validation sets
Extract gallery and probe arrays from the validation set
Vectorise the data
Plot images
Resize all images
Convert images to grayscale
Build pairs or triplets of images for metric learning networks
Plot a CMC Curve

These are provided to assist you in developing your solution.

In [1]: !pip install opencv-python

Requirement already satisfied: opencv-python in c:\users\acer\anaconda3\lib\site-packages ([Link])

Requirement already satisfied: numpy>=1.17.3 in c:\users\acer\anaconda3\lib\site-packages (from opencv-python) (1.26.4)
In [2]: #
# Utility functions for CAB420, Assignment 1B, Q1
# Author: Simon Denman ([Link]@[Link])
#
import os
[Link]['TF_CPP_MIN_LOG_LEVEL'] = '3'

import [Link] as plt # for plotting

import numpy as np # for reshaping, array manipulation
import cv2 # for image loading and colour conversion
import tensorflow as tf # for bulk image resize
import glob
import random
import keras

# get the subject ID from the filename. A sample filename is 0001_c1s1_001051_00.jpg, the first
# four characters are the subject ID
# fn: the filename to parse
#
# returns: first four characters of the filename converted to an int
#
# Data Augmentation Functions
#import tensorflow as tf

def augment_image(image):
image = [Link].random_flip_left_right(image)
image = [Link].random_brightness(image, max_delta=0.2)
image = [Link].random_contrast(image, lower=0.8, upper=1.2)
image = [Link].random_crop(image, size=[60, 28, 3])
image = [Link](image, (64, 32))
return image

def get_subject_id_from_filename(fn):
return int(fn[0:4])

# load the images in a directory

# base_path: path to the data
#
# returns: numpy arrays of size (samples, width, height, channels), and size (samples) for
# images and thier labels
def load_directory(base_path):
 # find all images in the directory
files = [Link]([Link](base_path, '*.jpg'))
x = []
y = []

# loop through the images, loading them and extracting the subject ID
for f in files:
[Link]([Link]([Link](f), cv2.COLOR_BGR2RGB) / 255.0)
[Link](get_subject_id_from_filename([Link](f)))

return [Link](x), [Link](y)

# Split the training set into train and validation, with a disjoint set of IDs in each
# X: X data to split
# Y: Y data to split, corresponds to X
# id_cutoff: Any IDs equal or smaller than this will be train, the rest will be in validation
#
# returns: numpy arrays for the training and validation sets, which contain mutually exclusise sets
# of subjects
#
def split_validation(X, Y, id_cutoff = 250):

train_X = X[Y <= id_cutoff, :]

train_Y = Y[Y <= id_cutoff]

val_X = X[Y > id_cutoff, :]

val_Y = Y[Y > id_cutoff]

return train_X, train_Y, val_X, val_Y

# Extract gallery and probe sets from a given chunk of data. Gallery and probe will contain one sample each
# for each subject, with only the first and second images for each subject used, and the rest discarded.
# X: X data to split
# Y: Y data to split, corresponds to X
#
# returns: numpy arrays containing gallery and probe sets corresponding to the input data. Gallery and
# probe will both contain one image of each subject. Any subjects with fewer than two images
# will be discarded
#
def sample_gallery_and_probe(X, Y):

# storage
gallery_X = []
gallery_Y = []
 probe_X = []
probe_Y = []

# find unique IDs

unique_ids = [Link](Y)
# loop through unique IDs
for i in unique_ids:
# pull out samples for the current ID
samples = X[Y == i, :]
# need at least two samples for use the ID
if [Link][0] >= 2:
# first sample for a subject stored in the gallery
gallery_X.append(samples[0, :])
gallery_Y.append(i)
# second sample stored in the probe
probe_X.append(samples[1, :])
probe_Y.append(i)
# any other images for the subject are ignored

# return extracted data

return [Link](gallery_X), [Link](gallery_Y), [Link](probe_X), [Link](probe_Y)

# load the data

# base_path: path to the data, within the directory that this points to there should be a 'Training'
# and 'Testing' directory
#
# returns: loaded data
#
def load_data(base_path):

# load training data

train_X, train_Y = load_directory([Link](base_path, 'Training'))
# split validation data
train_X, train_Y, val_X, val_Y = split_validation(train_X, train_Y)

# load gallery data from the test set

gallery_X, gallery_Y = load_directory([Link](base_path, 'Testing', 'Gallery'))

# load probe data from the test set

probe_X, probe_Y = load_directory([Link](base_path, 'Testing', 'Probe'))

return train_X, train_Y, val_X, val_Y, gallery_X, gallery_Y, probe_X, probe_Y

# Plot some images and their labels. Will plot the first 50 samples in a 10x5 grid
# x: array of images, of shape (samples, width, height, channels)
# y: labels of the images
#
def plot_images(x, y):
fig = [Link](figsize=[15, 18])
for i in range(50):
ax = fig.add_subplot(5, 10, i + 1)
[Link](x[i,:], cmap=plt.get_cmap('Greys'))
ax.set_title(y[i])
[Link]('off')

# vectorise an array of images, such that the shape is changed from {samples, width, height, channels} to
# (samples, width * height * channels)
# images: array of images to vectorise
#
# returns: vectorised array of images
#
def vectorise(images):
# use numpy's reshape to vectorise the data
return [Link](images, [len(images), -1])

# Resize an array of images

# images: array of images, of shape (samples, width, height, channels)
# new_size: tuple of the new size, (new_width, new_height)
#
# returns: resized array of images, (samples, new_width, new_height, channels)
#
def resize(images, new_size):
# tensorflow has an image resize funtion that can do this in bulk
# note the conversion back to numpy after the resize
return [Link](images, new_size).numpy()

# Convert images to grayscale

# images: array of colour images to convert, of size (samples, width, height, 3)
#
# returns: array of converted images, of size (samples, width, height, 1)
#
def convert_to_grayscale(images):
# storage for converted images
gray = []
# loop through images
for i in range(len(images)):
# convert each image using openCV
[Link]([Link](images[i,:], cv2.COLOR_RGB2GRAY))
 # pack converted list as an array and return
return np.expand_dims([Link](gray), axis = -1)

# Create a batch of siamese data. Pairs will be evenly balanced, such that there is an
# equal number of positive and negative pairs
# imgs: images to use to generate data, of shape (samples, width, height, channels)
# labels: labels for the data, of shape (samples)
# batch_size: number of pairs to generate
#
# returns: image pairs and labels to indicate if the pairs are the same, or different
#
def get_siamese_data(imgs, labels, batch_size):

image_a = [Link]((batch_size, [Link](imgs)[1], [Link](imgs)[2], [Link](imgs)[3]));

image_b = [Link]((batch_size, [Link](imgs)[1], [Link](imgs)[2], [Link](imgs)[3]));
label = [Link](batch_size);

for i in range(batch_size):

if (i % 2 == 0):
idx1 = [Link](0, len(imgs) - 1)
idx2 = [Link](0, len(imgs) - 1)
l = 1
while (labels[idx1] != labels[idx2]):
idx2 = [Link](0, len(imgs) - 1)

else:
idx1 = [Link](0, len(imgs) - 1)
idx2 = [Link](0, len(imgs) - 1)
l = 0
while (labels[idx1] == labels[idx2]):
idx2 = [Link](0, len(imgs) - 1)

image_a[i, :, :, :] = imgs[idx1,:,:,:]
image_b[i, :, :, :] = imgs[idx2,:,:,:]
label[i] = l

return [image_a, image_b], label

# Plot the first 10 pairs of a batch, good sanity check for pair generation
# x: images in the pairs
# y: labels of the pairs
#
def plot_pairs(x, y):
 fig = [Link](figsize=[25, 6])
for i in range(10):
ax = fig.add_subplot(2, 10, i*2 + 1)
[Link](x[0][i,:], cmap=plt.get_cmap('Greys'))
ax.set_title('Pair ' + str(i) +'; Label: ' + str(y[i]))

ax = fig.add_subplot(2, 10, i*2 + 2)

[Link](x[1][i,:], cmap=plt.get_cmap('Greys'))
ax.set_title('Pair ' + str(i) +'; Label: ' + str(y[i]))

# Create a batch of triplet data.

# imgs: images to use to generate data, of shape (samples, width, height, channels)
# labels: labels for the data, of shape (samples)
# batch_size: number of triplets to generate
#
# returns: triplet of the requested batch size
#
def get_triplet_data(imgs, labels, batch_size):

image_a = [Link]((batch_size, [Link](imgs)[1], [Link](imgs)[2], [Link](imgs)[3]));

image_b = [Link]((batch_size, [Link](imgs)[1], [Link](imgs)[2], [Link](imgs)[3]));
image_c = [Link]((batch_size, [Link](imgs)[1], [Link](imgs)[2], [Link](imgs)[3]));

for i in range(batch_size):

idx1 = [Link](0, len(imgs) - 1)

idx2 = [Link](0, len(imgs) - 1)
idx3 = [Link](0, len(imgs) - 1)

while (labels[idx1] != labels[idx2]):

idx2 = [Link](0, len(imgs) - 1)

while (labels[idx1] == labels[idx3]):

idx3 = [Link](0, len(imgs) - 1)

image_a[i, :, :, :] = imgs[idx1,:,:,:]
image_b[i, :, :, :] = imgs[idx2,:,:,:]
image_c[i, :, :, :] = imgs[idx3,:,:,:]

return [image_a, image_b, image_c]

# Create augmented triplet data

def get_triplet_data_augmented(imgs, labels, batch_size):
 image_a = []
image_p = []
image_n = []

for _ in range(batch_size):
anchor_idx = [Link](0, len(imgs) - 1)

# Positive sample (same identity, different image)

pos_candidates = [Link](labels == labels[anchor_idx])[0]
pos_candidates = pos_candidates[pos_candidates != anchor_idx]
if len(pos_candidates) == 0:
continue
pos_idx = [Link](pos_candidates)

# Negative sample (different identity)

neg_candidates = [Link](labels != labels[anchor_idx])[0]
neg_idx = [Link](neg_candidates)

anchor = imgs[anchor_idx]
positive = imgs[pos_idx]
negative = imgs[neg_idx]

# Apply augmentation
anchor = augment_image(anchor)
positive = augment_image(positive)
negative = augment_image(negative)

image_a.append(anchor)
image_p.append(positive)
image_n.append(negative)

return [Link](image_a), [Link](image_p), [Link](image_n)

# Plot the first 9 triplets of a batch, good sanity check for triplet generation
# x: images in the triplets
#
def plot_triplets(x):
fig = [Link](figsize=[25, 10])
for i in range(9):
ax = fig.add_subplot(3, 9, i*3 + 1)
[Link](x[0][i,:], cmap=plt.get_cmap('Greys'))
ax.set_title('Triple ' + str(i) + ': Anchor')

ax = fig.add_subplot(3, 9, i*3 + 2)
 [Link](x[1][i,:], cmap=plt.get_cmap('Greys'))
ax.set_title('Triple ' + str(i) + ': Positive')

ax = fig.add_subplot(3, 9, i*3 + 3)
[Link](x[2][i,:], cmap=plt.get_cmap('Greys'))
ax.set_title('Triple ' + str(i) + ': Negative')

# Compute a ranked histogram, which can be used to generate a CMC curve. This function will loop
# through all probe samples. For each probe sample it will:
# - Compare the sample to all gallery samples to get a distance between the probe sample and
# all gallery samples. In this case it is the L2 distance
# - Sort the gallery samples by how close they are to the probe samples
# - Find the location of the true match
# - Update a ranked histogram based on this
# The ranked histogram will show how many samples are matched at each rank. For example,
# ranked_histogram[0] will record how many samples are matched at Rank-1.
#
# This implementation assumes that there is only one sample per ID in the gallery set
#
# NOTE: L2 distance, used here, may not be appropriate for all problems. Consider the nature
# of your problem and what distance measure you should use
#
# gallery_feat: features for the gallery data, of shape (gallery_samples, num_features)
# gallery_Y: IDs of the gallery samples, of shape (gallery_samples,)
# probe_feat: features for the probe data, of shape (probe_samples, num_features)
# probe_Y: IDs of the probe samples, of shape (probe_samples,)
# verbose: bool to indicate if debug information shoudl be printed. Be careful using this with
# large feature sets, and/or lots of samples
#
# returns: ranked histogram matching the probe samples to the gallery
#
def get_ranked_histogram_l2_distance(gallery_feat, gallery_Y, probe_feat, probe_Y, verbose = False):

# storage for ranked histogram

# length equal to number of unique subjects in the gallery
ranked_histogram = [Link](len([Link](gallery_Y)))

# loop over all samples in the probe set

for i in range(len(probe_Y)):
# get the true ID of this sample
true_ID = probe_Y[i]
if verbose:
print('Searching for ID %d' % (true_ID))
 # get the distance between the current probe and the whole gallery, L2 distance here. Note that L2
# may not always be the best choice, so consider your distance metric given your problem
dist = [Link](gallery_feat - probe_feat[i,:], axis=1, ord=2)
if verbose:
print(dist)

# get the sorted order of the distances

a = [Link](dist)
# apply the order to the gallery IDs, such that the first ID in the list is the closest, the second
# ID is the second closest, and so on
ranked = gallery_Y[a]
if verbose:
print('Ranked IDs for query:')
print(a)

# find the location of the True Match in the ranked list

ranked_result = [Link](ranked == true_ID)[0][0]
if verbose:
print(ranked_result)

# store the ranking result in the histogram

ranked_histogram[ranked_result] += 1
if verbose:
print('')

if verbose:
print(ranked_histogram)

return ranked_histogram

# Convert a ranked histogram to a CMC. This simply involves computing the cumulative sum over the histogram
# and dividing it by the length of the histogram
# ranked_hist: ranked histogram to convert to a CMC
#
# returns: CMC curve
#
def ranked_hist_to_CMC(ranked_hist):

cmc = [Link](len(ranked_hist))
for i in range(len(ranked_hist)):
cmc[i] = [Link](ranked_hist[:(i + 1)])

return (cmc / len(ranked_hist))

 # plot a CMC
# cmc: cmc data to plot
#
def plot_cmc(cmc):
fig = [Link](figsize=[10, 8])
ax = fig.add_subplot(1, 1, 1)
[Link](range(1, len(cmc)+1), cmc)
ax.set_xlabel('Rank')
ax.set_ylabel('Count')
ax.set_ylim([0, 1.0])
ax.set_title('CMC Curve')

Utility Function Demonstration

The following presents a brief demonstration of the utility functions. These portions of code do not form part of the template, or solution, and
could be commented out/removed, or you could restructure this portion of the code to form part of your solution.

Data Loading
This will load the data, pulling out the training set and validation set, and gallery and probe testing sets. Note that to ready the validation set for
the target identification function, you still need to call the sample_gallery_and_probe function (see below for details).

Note the size of the datasets here, and the number of unique subjects in each:

The training set contains images for 250 unique IDs, with multiple images for each.
The validation set contains a different 50 identities, with multiple images for each.
The testing set is the gallery and probe sets combined:
The gallery set contains 1 image for another 301 unique IDs (totally separate to those in the training and validation sets)
The probe set contains another single image for the same 301 subjets as the gallery set.

In [3]: # note that you will probably need to change the filepath here
train_X, train_Y, val_X, val_Y, gallery_X, gallery_Y, probe_X, probe_Y = load_data('Q1/')
print(train_X.shape)
print(train_Y.shape)
print(val_X.shape)
print(val_Y.shape)
print(gallery_X.shape)
print(gallery_Y.shape)
print(probe_X.shape)
print(probe_Y.shape)

# plot some images

plot_images(gallery_X, gallery_Y)

(4711, 128, 64, 3)

(4711,)
(1222, 128, 64, 3)
(1222,)
(301, 128, 64, 3)
(301,)
(301, 128, 64, 3)
(301,)
 Validation Gallery and Probe Sets
If you wish to use the validation set to simulate the test set and compute rank accuracies (or similar) to help select hyper-parameters, you will
need to extract gallery and probe samples from the set. We can do this using the following.

Note that this will give us 50 samples in both sets, as we have 50 unique IDs and we're getting one sample per ID.

In [4]: val_gallery_X, val_gallery_Y, val_probe_X, val_probe_Y = sample_gallery_and_probe(val_X, val_Y)

print(val_gallery_X.shape)
print(val_gallery_Y.shape)
print(val_probe_X.shape)
print(val_probe_Y.shape)

(50, 128, 64, 3)

(50,)
(50, 128, 64, 3)
(50,)
 Pair and Triplet Data
Pair and triplet functions are provided to pull out paired and triplet data for use with Siamese networks. If you want pairs or triplets for the
validation set, you would use the val_X and val_Y variables loaded by load_data

In [5]: # create a pair generator and display a pair

x, y = get_siamese_data(train_X, train_Y, 10)
plot_pairs(x, y)

# create a triplet and display a triplet

x = get_triplet_data(train_X, train_Y, 9)
plot_triplets(x)
 Conversion and Resizing
Functions to convert images to greyscale and resize them are also provided.

In [6]: # resize data and convert to grayscale

gallery_X_small_gray = convert_to_grayscale(resize(gallery_X, (64, 32)))
print(gallery_X_small_gray.shape)
probe_X_small_gray = convert_to_grayscale(resize(probe_X, (64, 32)))
print(probe_X_small_gray.shape)

# plot some resized and grayscale images

plot_images(gallery_X_small_gray, gallery_Y)

(301, 64, 32, 1)

(301, 64, 32, 1)
 Vectorising Data
To use the images with non-DL methods, they need to be vectorised. A function is provided for this.

In [7]: # vectorise some data

gallery_feat_vec = vectorise(gallery_X_small_gray)
probe_feat_vec = vectorise(probe_X_small_gray)
print(gallery_feat_vec.shape)
print(probe_feat_vec.shape)

#
# at this point, you have data in the shape (gallery_samples, number_of_features) and (probe_samples, number_of_features)
#

(301, 2048)
(301, 2048)

CMC Curves
 Functions to create a CMC curve are provided. These assume that you have data in the following form:

An array of gallery features, of size (gallery_samples, num_features)

An array of gallery IDs, of size (gallery_samples)
An array of probe features, of size (probe_samples, num_features)
An array of probe IDs, of size (probe_samples)

Note that the provided implementation is limited in that it assumes that you have only one sample from each gallery ID in the set of gallery
samples.

The process of generating a CMC is split across three functions:

get_ranked_histogram_l2_distance , which will compare each probe sample to all gallery samples and determine the location of the
correct match (the rank), storing this in a histogram
ranked_hist_to_CMC , which will convert the ranked histogram to a CMC
plot_cmc , which will actually plot the CMC

Two simple examples demonstrating this are given below.

In [8]: # we'll start off with just 8 IDs

num_ids = 8

# we'll create a list of IDs, these are the gallery and probe IDs (gallery_y and probe_y)
ID_1 = [Link](0, num_ids, 1)
ID_2 = [Link](0, num_ids, 1)
# we'll then generate some features
# to keep this simple, we'll use a 2D feature, i.e. each sample will have a 2D feature associated with it
# this means that we have our gallery features of size (8, 2), as we have 8 samples and 2D feaures
Feat_1 = [Link]([Link]([ID_1, ID_1]) + ([Link](2, num_ids) - 0.5)*4.0, (1,0))
Feat_2 = [Link]([Link]([ID_2, ID_2]) + ([Link](2, num_ids) - 0.5)*8.0, (1,0))

# print features and their shape to confirm what we have

print(ID_1)
print(Feat_1)
print(Feat_1.shape)
print(ID_2)
print(Feat_2)
print(Feat_2.shape)
# we can now compute the ranked histogram
# this will print out of a heap of debug data to show what's going on. In particular we see the distance between
# each probe sample and the whole gallery, and the rank of the true match, which is what is used to build the
# ranked histogram
ranked_hist = get_ranked_histogram_l2_distance(Feat_1, ID_1, Feat_2, ID_2, True)
print(ranked_hist)
cmc = ranked_hist_to_CMC(ranked_hist)
print(cmc)
plot_cmc(cmc)
[0 1 2 3 4 5 6 7]
[[ 0.52729756 0.31969625]
[ 1.65858582 -0.25461592]
[ 1.06408898 2.86131705]
[ 4.30781529 1.20168975]
[ 3.34647126 2.30267241]
[ 6.58334863 6.5276195 ]
[ 6.53437403 5.3384909 ]
[ 6.27376137 6.04152969]]
(8, 2)
[0 1 2 3 4 5 6 7]
[[ 3.35876286 0.36475717]
[ 3.73535366 3.60599889]
[-1.42948756 5.77779168]
[ 6.57633576 4.27722653]
[ 6.85310595 7.91263316]
[ 3.6240927 5.28489258]
[ 6.3435796 2.03847059]
[ 7.34902475 6.38019298]]
(8, 2)
Searching for ID 0
[2.83182383 1.80948197 3.39092015 1.26536819 1.93795422 6.95548887
5.90106206 6.38145458]
Ranked IDs for query:
[3 1 4 0 2 6 7 5]
3

Searching for ID 1
[4.5925384 4.3837554 2.77312208 2.47152074 1.36010641 4.08006646
3.29181463 3.51785787]
Ranked IDs for query:
[4 3 2 6 7 5 1 0]
6

Searching for ID 2
[5.79825955 6.77688267 3.83715366 7.33875691 5.9064572 8.04784354
7.9759687 7.70776245]
Ranked IDs for query:
[2 0 4 1 3 7 6 5]
0

Searching for ID 3
[7.22861737 6.6874405 5.69119181 3.82166346 3.78561603 2.25040389
1.06209362 1.79006058]
Ranked IDs for query:
[6 7 5 4 3 2 1 0]
4

Searching for ID 4
[9.88273963 9.67920435 7.68300149 7.1774136 6.61574988 1.41103928
2.59379999 1.95874152]
Ranked IDs for query:
[5 7 6 4 3 2 1 0]
3

Searching for ID 5
[5.85177877 5.87787136 3.52524289 4.14005096 2.99511449 3.20960525
2.91077484 2.75558411]
Ranked IDs for query:
[7 6 4 5 2 3 0 1]
3

Searching for ID 6
[6.06492555 5.21607251 5.34322913 2.20103123 3.0087308 4.49554748
3.30553121 4.00366791]
Ranked IDs for query:
[3 4 6 7 5 1 2 0]
2

Searching for ID 7
[9.12499767 8.74081143 7.20297893 6.00548505 5.71372107 0.77974002
1.32242165 1.12733498]
Ranked IDs for query:
[5 7 6 4 3 2 1 0]
1

[1. 1. 1. 3. 1. 0. 1. 0.]
[1. 1. 1. 3. 1. 0. 1. 0.]
[0.125 0.25 0.375 0.75 0.875 0.875 1. 1. ]
 The above example results in a pretty simple CMC. Let's throw more IDs at it to get something that looks a bit better. The setup here is the same
as what's above, we just have 100 gallery and probe samples now instead.

In [9]: num_ids = 100

ID_1 = [Link](0, num_ids, 1)

ID_2 = [Link](0, num_ids, 1)
Feat_1 = [Link]([Link]([ID_1, ID_1]) + ([Link](2, num_ids) - 0.5)*20.0, (1,0))
Feat_2 = [Link]([Link]([ID_2, ID_2]) + ([Link](2, num_ids) - 0.5)*80.0, (1,0))
print(Feat_1.shape)
print(Feat_2.shape)

ranked_hist = get_ranked_histogram_l2_distance(Feat_1, ID_1, Feat_2, ID_2)

cmc = ranked_hist_to_CMC(ranked_hist)
plot_cmc(cmc)

(100, 2)
(100, 2)
 With a CMC, the ideal curve is one that is always at 100%. The steeper the curve and faster it get's to 100%, the better.

Question 1 Template
The following provides a starting point for your solution, and some suggestions on how to proceed.

In [1]: #
# Task Overview
#
# This question concerns person re-identification. We can think of this as a retrieval problem, where the input is an image
# and the output is a list of people, ranked based on how similar they are to the image. To get this similarity-based
# ranking, we need a way to compare two images and measure their similarity. To do this you will use a learned representation.
# The question asks you to train two models (one non-DL, one DL) to extract such a representation of an image. Using this
# learned representation, you can then transform a pair of images into vectors, and the distance between the vectors is a
# measure of their similarity.
#
# The data you have been given is split into as follows:
# * training: Within the training set, you have roughly 2-20 images of 250 different people. You will use this data to train
# your model.
# * validation: With the validation set, you have roughly 2-20 images of 50 different people. The 50 people here are totally
# different from those in the training set. You will use this data is you want a validation set to train a
# deep net, or if you want a validation probe and gallery set to tune model hyper-parameters such as embedding
# size or number of components to keep (note that what hyper-parameters you care about will depend on what
# models you are using.
# * testing: This itself is split into gallery and probe. The gallery and probe each contain one image for 301 individuals.
# The gallery and probe both contain the same 301 individuals. This is your test set. Note that the 301 individuals
# here are again totally different from those in the training and validation sets. When thinking about this set,
# consider the re-ID problem. The gallery set contains images of 301 people that have been seen on some CCTV network.
# The probe set represents images that you are trying to match to those in the gallery set.
#

In [10]: # Load the data using the provided utility function (adjust path as necessary)
train_X, train_Y, val_X, val_Y, gallery_X, gallery_Y, probe_X, probe_Y = load_data("Q1")

# Resize all datasets to 64x32 while keeping RGB

target_size = (64, 32)
train_X_resized = resize(train_X, target_size)
val_X_resized = resize(val_X, target_size)
gallery_X_resized = resize(gallery_X, target_size)
probe_X_resized = resize(probe_X, target_size)
# Print shapes to confirm
print("Train X:", train_X_resized.shape)
print("Val X:", val_X_resized.shape)
print("Gallery X:", gallery_X_resized.shape)
print("Probe X:", probe_X_resized.shape)

Train X: (4711, 64, 32, 3)

Val X: (1222, 64, 32, 3)
Gallery X: (301, 64, 32, 3)
Probe X: (301, 64, 32, 3)
In [12]: #Visualizing Augmented Triplets
import [Link] as plt

def plot_augmented_triplets(anchor, positive, negative, count=5):

"""
Plots a few augmented triplets: anchor, positive, negative.
Args:
anchor, positive, negative: Tensors or arrays of shape (batch_size, 64, 32, 3)
count: Number of triplets to display
"""
[Link](figsize=(12, 4 * count))
for i in range(count):
# Anchor
[Link](count, 3, i * 3 + 1)
[Link](anchor[i].numpy())
[Link](f"Triplet {i} - Anchor")
[Link]("off")

# Positive
[Link](count, 3, i * 3 + 2)
[Link](positive[i].numpy())
[Link]("Positive")
[Link]("off")

# Negative
[Link](count, 3, i * 3 + 3)
[Link](negative[i].numpy())
[Link]("Negative")
[Link]("off")

plt.tight_layout()
[Link]()

In [13]: # Generate a batch of augmented triplets

a, p, n = get_triplet_data_augmented(train_X_resized, train_Y, batch_size=5)

# Plot the triplets

plot_augmented_triplets(a, p, n, count=5)
Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
In [14]: from [Link] import PCA
from [Link] import StandardScaler

# Vectorize resized images

train_X_vec = vectorise(train_X_resized)
val_X_vec = vectorise(val_X_resized)
gallery_X_vec = vectorise(gallery_X_resized)
probe_X_vec = vectorise(probe_X_resized)
# Standardize features before PCA
scaler = StandardScaler()
train_X_std = scaler.fit_transform(train_X_vec)
gallery_X_std = [Link](gallery_X_vec)
probe_X_std = [Link](probe_X_vec)

# Apply PCA
pca = PCA(n_components=100) # You can tune this using val set
train_X_pca = pca.fit_transform(train_X_std)
gallery_X_pca = [Link](gallery_X_std)
probe_X_pca = [Link](probe_X_std)

# Evaluate using CMC

rank_hist = get_ranked_histogram_l2_distance(gallery_X_pca, gallery_Y, probe_X_pca, probe_Y)
cmc = ranked_hist_to_CMC(rank_hist)

# Report rank-1, 5, and 10 accuracies

print(f"PCA Non-DL Rank-1 Accuracy: {cmc[0]:.2f}")
print(f"PCA Non-DL Rank-5 Accuracy: {cmc[4]:.2f}")
print(f"PCA Non-DL Rank-10 Accuracy: {cmc[9]:.2f}")

# Plot CMC curve

plot_cmc(cmc)

PCA Non-DL Rank-1 Accuracy: 0.08

PCA Non-DL Rank-5 Accuracy: 0.16
PCA Non-DL Rank-10 Accuracy: 0.23
In [15]: #
# DL Method
#
# Using the image data, develop your DL method. A suggested approach would be:
# - Pick a backbone for your network. This could be a network from the lecture or prac examples, or something from
# [Link]. As per Assignment 1A, the network does not need to be overly complex, but you should briefly explain
# your selection of this model. You should avoid networks so simple that they perform very badly.
# - Pick your training approach. You have been given pair and triplet creation functions above, so you can easily use either con
# or a triplet loss. You could also look at the bonus example on hard negative mining and consider this approach. Once you've
# picked your approach, set the network up with the correct number of inputs and an appropriate loss.
# - Create some pairs and triplets, and train the network using the training set
#
# If you wish to use the validation set:
# - Use the val_X and val_Y set with your pair/triplet/whatever function to determine when your model has converged
# - Use the val_gallery and val_probe sets to determine other network parameters such as embedding sizes. You would compute
# CMC curves / rank-1 accuracy for different options and pick the one that works best on the validation set (see below for
# info on computing CMC curves, etc).
#
import tensorflow as tf
from [Link] import layers, models
import numpy as np

# Define a simple CNN backbone to extract embeddings

def build_embedding_model(input_shape=(64, 32, 3), embedding_dim=64):
model = [Link]([
[Link](shape=input_shape),
layers.Conv2D(32, (3, 3), activation='relu'),
layers.MaxPooling2D((2, 2)),
layers.Conv2D(64, (3, 3), activation='relu'),
layers.MaxPooling2D((2, 2)),
[Link](),
[Link](embedding_dim)
])
return model

# Triplet loss function

def triplet_loss(anchor, positive, negative, margin=1.0):
pos_dist = tf.reduce_sum([Link](anchor - positive), axis=1)
neg_dist = tf.reduce_sum([Link](anchor - negative), axis=1)
return tf.reduce_mean([Link](pos_dist - neg_dist + margin, 0.0))

# Training loop for triplet network

def train_triplet_network(model, train_X, train_Y, batch_size=32, epochs=5):
 optimizer = [Link]()
for epoch in range(epochs):
anchor, positive, negative = get_triplet_data_augmented(train_X, train_Y, batch_size)
with [Link]() as tape:
a_embed = model(anchor, training=True)
p_embed = model(positive, training=True)
n_embed = model(negative, training=True)
loss = triplet_loss(a_embed, p_embed, n_embed)
grads = [Link](loss, model.trainable_variables)
optimizer.apply_gradients(zip(grads, model.trainable_variables))
print(f"Epoch {epoch+1}/{epochs}, Loss: {[Link]():.4f}")
return model

# Build and train the model

embedding_dim = 64
model = build_embedding_model(input_shape=(64, 32, 3), embedding_dim=embedding_dim)
model = train_triplet_network(model, train_X_resized, train_Y, batch_size=32, epochs=5)

# Generate embeddings for gallery and probe

gallery_embed = [Link](gallery_X_resized)
probe_embed = [Link](probe_X_resized)

# Evaluate using CMC

hist_dl = get_ranked_histogram_l2_distance(gallery_embed, gallery_Y, probe_embed, probe_Y)
cmc_dl = ranked_hist_to_CMC(hist_dl)
plot_cmc(cmc_dl)

# Print rank-1, rank-5, and rank-10 accuracy

print("DL Rank-1 Accuracy: {:.2f}%".format(cmc_dl[0]*100))
print("DL Rank-5 Accuracy: {:.2f}%".format(cmc_dl[4]*100))
print("DL Rank-10 Accuracy: {:.2f}%".format(cmc_dl[9]*100))

Epoch 1/5, Loss: 0.9348

Epoch 2/5, Loss: 0.9091
Epoch 3/5, Loss: 0.7182
Epoch 4/5, Loss: 0.6050
Epoch 5/5, Loss: 0.6670
10/10 ━━━━━━━━━━━━━━━━━━━━ 1s 28ms/step
10/10 ━━━━━━━━━━━━━━━━━━━━ 0s 16ms/step
DL Rank-1 Accuracy: 10.63%
DL Rank-5 Accuracy: 19.93%
DL Rank-10 Accuracy: 26.58%
In [8]: #
# Evaluation
#
# You will now evaluate your methods using the CMC curve and rank-1, -5 and -10 accuracies. The functions provided above will all
# you to obtain these values. To do this you will use code that is something like the following:
#
# gallery_feats = [Link](gallery_X)
# probe_feats = [Link](probe_X)
# ranked_hist = get_ranked_histogram_l2_distance(gallery_feats, gallery_Y, probe_feats, probe_Y)
# cmc = ranked_hist_to_CMC(ranked_hist)
# rank_1_acc = cmc[0]
# rank_5_acc = cmc[4]
# rank_10_acc = cmc[9]
# plot_cmc(cmc)
#
# In the above, the [Link] operation will be replaced with whatever is appropriate for what you have done. For the non-D
# method, this is likely to be fairly similar. For the DL method, you will likely change this to call the predict function on you
# backbone network. Your input data (gallery_X and probe_X) will also change to any resized, colour converted and/or vectorised
# version you are using.
#
# The cmc variable that results from the call to ranked_hist_to_CMC will give you the rank-1, -5 and -10 accuracies directly, and
# is what you will plot to visualise performance. Note that you may wish to plot the CMCs for both models on the same axis to mak
# it easier to compare performance.
#
import [Link] as plt

# Re-plot both CMC curves on the same axes for comparison

[Link](figsize=(8, 5))
[Link](cmc, label="PCA (Non-DL)", linewidth=2)
[Link](cmc_dl, label="CNN Triplet (DL)", linewidth=2)
[Link]("Rank")
[Link]("Recognition Rate (%)")
[Link]("CMC Curve Comparison")
[Link](True)
[Link]()
[Link]()

# Print rank accuracies for both methods

print("=== PCA (Non-DL) ===")
print("Rank-1 Accuracy: {:.2f}%".format(cmc[0]*100))
print("Rank-5 Accuracy: {:.2f}%".format(cmc[4]*100))
print("Rank-10 Accuracy: {:.2f}%".format(cmc[9]*100))
print("\n=== CNN Triplet (DL) ===")
print("Rank-1 Accuracy: {:.2f}%".format(cmc_dl[0]*100))
print("Rank-5 Accuracy: {:.2f}%".format(cmc_dl[4]*100))
print("Rank-10 Accuracy: {:.2f}%".format(cmc_dl[9]*100))

=== PCA (Non-DL) ===

Rank-1 Accuracy: 7.97%
Rank-5 Accuracy: 15.95%
Rank-10 Accuracy: 22.92%

=== CNN Triplet (DL) ===

Rank-1 Accuracy: 11.96%
Rank-5 Accuracy: 20.93%
Rank-10 Accuracy: 27.91%
In [ ]: # For your write-up, you should include:
# - briefly mention any pre-processing you did and why; or why you left the data as-is. Note that for fair comparison,
# use the same pre-processing for both the methods (non-deep learning and deep-learning) is recommended. If you split
# the data into training and validation sets you could dicsuss this here too
# - Outline your methods. For both the non-DL and DL methods, outline what you did and why, and any key parameters you
# set. For the DL method, outline how it was trained.
# - An evaluation, that covers the Rank-1, -5 and -10 results, and the CMC curves. Consider where performance differs
# between the models, and the pros and cons of each.
# - A brief discsusion of the ethical considerations as they relate to person re-id (see the assignment brief for
# further details and references to use as a starting point
# Your write-up should be supported by appropriate figures and tables. Figures and tables should have numbers and meaningful capt
# Note that figures and tables are not included in the page limits.
#
# SEE THE ASSIGNMENT BRIEF ON CANVAS FOR MORE DETAILS AND NOTE THAT A NOTEBOOK FILE DOES NOT CONSTITUTE A VALID SUBMISSION.
# YOU SHOULD WRITE UP YOUR RESPONSE IN A SEPARATE DOCUMENT
## Report: Person Re-Identification - Assignment 1B

In [ ]: