Deep Learning Lab

Experiment No.05

(PART A : TO BE REFFERED BY
STUDENTS)

Aim: Design the architecture and implement the autoencoder model for Image Compression
Prerequisite: Basic Mathematics and Statistical Concepts, Linear Algebra, Machine Learning
Outcome:
After successful completion of this experiment students will be able to

1.Build and train deep learning models such as Autoencoders.

Theory:
Autoencoders are similar to dimensionality reduction techniques like Principal Component Analysis
(PCA). They project the data from a higher dimension to a lower dimension using linear
transformation and try to preserve the important features of the data while removing the non-
essential parts.
However, the major difference between autoencoders and PCA lies in the transformation part: as you
already read, PCA uses linear transformation whereas autoencoders use non-linear transformations.

The above figure is a two-layer vanilla autoencoder with one hidden layer. In deep learning
terminology, you will often notice that the input layer is never taken into account while counting the
total number of layers in an architecture. The total layers in an architecture only comprises of the
number of hidden layers and the ouput layer.
As shown in the image above, the input and output layers have the same number of neurons.
Autoencoder can be broken in to three parts
Encoder: this part of the network compresses or downsamples the input into a fewer number of bits.
The space represented by these fewer number of bits is often called the latent-space or bottleneck.
The bottleneck is also called the "maximum point of compression" since at this point the input is
compressed the maximum. These compressed bits that represent the original input are together called
an “encoding” of the input.
Decoder: this part of the network tries to reconstruct the input using only the encoding of the input.
When the decoder is able to reconstruct the input exactly as it was fed to the encoder, you can say
that the encoder is able to produce the best encodings for the input with which the decoder is able to
reconstruct well!
There are variety of autoencoders, such as the convolutional autoencoder, denoising autoencoder,
variational autoencoder and sparse autoencoder. However, as you read in the introduction, you'll
only focus on the convolutional and denoising ones in this tutorial.Convolutional Autoencoders in
Python with Keras
Algorithm:
# Import necessary libraries
import tensorflow as tf
from tensorflow.keras.layers import Input, Conv2D, MaxPooling2D,
UpSampling2D from tensorflow.keras.models import Model
# Define the autoencoder model
def build_autoencoder(input_shape, encoding_dim):
# Encoder
input_img = Input(shape=input_shape)
x = Conv2D(32, (3, 3), activation='relu',
padding='same')(input_img) x = MaxPooling2D((2, 2),
padding='same')(x)
x = Conv2D(64, (3, 3), activation='relu',
padding='same')(x) encoded = MaxPooling2D((2, 2),
padding='same')(x)
# Decoder
x = Conv2D(64, (3, 3), activation='relu',
padding='same')(encoded) x = UpSampling2D((2, 2))(x)
x = Conv2D(32, (3, 3), activation='relu',
padding='same')(x) x = UpSampling2D((2, 2))(x)
decoded = Conv2D(1, (3, 3), activation='sigmoid', padding='same')(x)
# Build and compile the autoencoder model
autoencoder = Model(input_img, decoded)
autoencoder.compile(optimizer='adam', loss='binary_crossentropy')
 PART B

(PART B : TO BE COMPLETED BY
STUDENTS)

(Students must submit the soft copy as per following segments within two hours of the practical.
The soft copy must be uploaded at the end of the practical)

Roll. No: A19 Name: Niraj Patil

Class: BE(AI&DS) Batch: A1
Date of Experiment: 26/8/24 Date of Submission: 12/9/24
Grade:

Software Code written by student:

import os
import keras
from keras.datasets import mnist
from keras.layers import Dense
from keras.models import Sequential
import matplotlib.pyplot as plt

# Load MNIST dataset

(x_train, _), (x_test, _) = mnist.load_data()

# Preprocess images
x_train = x_train.reshape(-1, 784) / 255.0
x_test = x_test.reshape(-1, 784) / 255.0

# Encoder definition
encoder = Sequential([
Dense(128, activation='relu', input_shape=(784,)),
Dense(64, activation='relu')
])
# We can compile the model here, otherwise there will be a warning when it is
# used, but it is optional.

# Decoder definition
decoder = Sequential([
Dense(128, activation='relu', input_shape=(64,)),
Dense(784, activation='sigmoid')
])
# We can compile the model here, otherwise there will be a warning when it is
# used, but it is optional.
# Autoencoder definition
autoencoder = Sequential([
encoder,
decoder
])

# Compile and train the autoencoder

autoencoder.compile(optimizer='adam', loss='binary_crossentropy')
autoencoder.fit(x_train, x_train, epochs=10, batch_size=256)

# Evaluation
loss = autoencoder.evaluate(x_test, x_test, verbose=0)
print("Test Loss:", loss)

# Save models
encoder_model_path = 'encoder_model.h5'
decoder_model_path = 'decoder_model.h5'

encoder.save(encoder_model_path)
decoder.save(decoder_model_path)

def get_file_size(file_path):
size = os.path.getsize(file_path)
for unit in ['bytes', 'KB', 'MB', 'GB', 'TB']:
if size < 1024:
return f"{size:.2f} {unit}"
size /= 1024

encoder_model_size = get_file_size(encoder_model_path)
decoder_model_size = get_file_size(decoder_model_path)
print("Encoder model size:", encoder_model_size)
print("Decoder model size:", decoder_model_size)

# Load the models

loaded_encoder = keras.models.load_model('encoder_model.h5')
loaded_decoder = keras.models.load_model('decoder_model.h5')

# Use loaded models for prediction

#
# In a real deployment, the encoder will be on the server side, the decoder will
# be on the client side, and the encoded images are what's transferred from the
# server to the client.
encoded_imgs = loaded_encoder.predict(x_test)
decoded_imgs = loaded_decoder.predict(encoded_imgs)

# Visualization
n = 10
plt.figure(figsize=(20, 4))
for i in range(n):
# Original image
ax = plt.subplot(2, n, i + 1)
 plt.imshow(x_test[i].reshape(28, 28))
plt.gray()
ax.get_xaxis().set_visible(False)
ax.get_yaxis().set_visible(False)

# Reconstructed image
ax = plt.subplot(2, n, i + 1 + n)
plt.imshow(decoded_imgs[i].reshape(28, 28))
plt.gray()
ax.get_xaxis().set_visible(False)
ax.get_yaxis().set_visible(False)
plt.show()

Input and Output:

Epoch 1/10

235/235 [==============================] - 6s 20ms/step - loss: 0.2219

Epoch 2/10

235/235 [==============================] - 5s 22ms/step - loss: 0.1244

Epoch 3/10

235/235 [==============================] - 4s 19ms/step - loss: 0.1071

Epoch 4/10

235/235 [==============================] - 4s 18ms/step - loss: 0.0990

Epoch 5/10

235/235 [==============================] - 5s 22ms/step - loss: 0.0944

Epoch 6/10

235/235 [==============================] - 4s 18ms/step - loss: 0.0913

Epoch 7/10

235/235 [==============================] - 4s 18ms/step - loss: 0.0890

Epoch 8/10

235/235 [==============================] - 5s 22ms/step - loss: 0.0872

Epoch 9/10

235/235 [==============================] - 4s 17ms/step - loss: 0.0857

Epoch 10/10

235/235 [==============================] - 4s 17ms/step - loss: 0.0844

Test Loss: 0.08285020291805267

Encoder model size: 439.05 KB

Decoder model size: 442.12 KB

313/313 [==============================] - 1s 3ms/step

313/313 [==============================] - 1s 2ms/step

Observations and learning:

Designing an autoencoder for image compression involves understanding the trade-off between
compression ratio and image quality. The encoder must efficiently compress high-dimensional image
data into a lower-dimensional latent space, while the decoder should accurately reconstruct the image
from this compressed representation.

Conclusion:
Implementing an autoencoder for image compression highlights the balance between reducing data
size and maintaining image quality. Careful tuning of model architecture and training parameters is
crucial for achieving effective compression without significant loss of visual fidelity.
Question of Curiosity
1. How do autoencoders reduce image size?
Ans: Autoencoders reduce image size by encoding the input image into a lower-dimensional latent
space using an encoder network. This compressed representation captures the essential features of
the image. The decoder network then reconstructs the image from this compressed representation,
ideally preserving important details while discarding less critical information.

2. What are the main challenges in using autoencoders for image compression?
Ans: The main challenges include maintaining high image quality while achieving a significant
compression ratio. Over-compression can lead to loss of important details and artifacts, while under-
compression may not sufficiently reduce file size. Additionally, selecting the right network
architecture and hyperparameters requires careful tuning to balance compression and reconstruction
performance.

3. How does the choice of architecture affect the performance of an autoencoder?

Ans: The choice of architecture impacts performance by influencing how well the autoencoder can
capture and reconstruct image features. For example, deeper networks or convolutional layers can
better capture spatial hierarchies and details in images, while simpler architectures may struggle with
complex images. Choosing an appropriate architecture involves balancing model complexity with
computational efficiency and reconstruction accuracy.