Skip to content

About

Image classification on CIFAR-10 dataset using self-trained AI models

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

38 Commits

Folders and files

Repository files navigation

Image-Classification

Description

Project to perform image classification on the CIFAR-10 dataset using four AI models: Naive Bayes, a decision tree, a multi-layer perception, and a convolutional neural network (CNN) using Python and Pytorch to detect the 10 object classes in the CIFAR-10 dataset. While applying basic evaluation metrics: accuracy, confusion matrix, precision, and recall.

Overview

models/: A folder containing all saved trained models.

graphs/: A folder containing all graphs, confusion matrices.

main.py: Where we will be running our program. This file contains code on transforming and loading our data from CIFAR-10.

utils.py: Contains utility functions: select_n_img() and extract_feature_vectors(). select_n_img() will select 500 training images per class and 100 testing images per class. extract_feature_vectors() takes in the data along with a model and will generate the PCA-reduced feature vectors.

mlp.py: This is the file where we have defined our multilayer perceptron model and its variants. Training, testing and evaluation are done in this same file and metrics printed to the console.

cnn.py: This is the file where we have created our Convolutional Neural Network. Training, testing and evaluation code included.

naive_bayes.py: Contains the Naive Bayes algorithm implementation.

decision_tree.py: Contains the decision tree algorithm implementation and a help function to plot the calculated metrics (training accuracy, testing accuracy, precision, recall and f1-score).

graphs/Decision_Tree.png: A graph plotting the metrics (training accuracy, testing accuracy, precision, recall and f1-score) against the max_depths of the local implemented decision tree.

graphs/Scikit_Decision_Tree.png: A graph plotting the metrics (training accuracy, testing accuracy, precision, recall and f1-score) against the max_depths of the Scikit's decision tree.

graphs/DT_10.png: Confusion matrices for locally implemented Decision Tree algorithm with maximum depth of 10. There are multiple but the file name follows this naming convention. DT_{depth}, where depth is the decision tree's maximum depth.

graphs/Scikit_DT_10.png: Confusion matrices for Scikit's Decision Tree algorithm with maximum depth of 10. There are multiple but the file name follows this naming convention. Scikit_DT_{depth}, where depth is the decision tree's maximum depth.

.gitignore: Ignores folders cifar-10-batches-py/ and __pycache__/ since we don't want to commit these.

MLP: This is a folder that contains our pre-trained MLP models. The confusion matrix for the main model and its variants are also stored inside. .pth are model saves and .png are the confusion matrices (file is respective filename).

Getting Started

To run the program, simply run the script main.py in Interactive Window. This will ensure that the graphs are generated.

image

Image preprocessing

Running the main.py script will automatically preprocess the images. Below is a detailed explanation:

image

  1. image-datasets is how we are getting our datasets, we are saving the image batches in our root directory since data_dir = "./. We make sure to check whether we have already downloaded the dataset before.
  2. data_transforms contains instructions on how we are transforming our images. In this case for both train and test we are resizing these images to 224x224 and then normalizing them.

image

  1. We then use the utility function select_n_img to select the first 500 images of each class for the training datasets, and 100 images of each class for the testing datasets.
  2. We then use dataloaders to load data into our model, with training batch_size = 500 and testing batch_size=100.

image

  1. Using the pretrained RestNet-18 model, we removed the last layer (restnet18.fc) to use it as a fixed feature extractor.
  2. We then pass in the modified model, the dataloaders and our device (cuda:0 or cpu) to the helper function extract_feature_vectors to get our PCA-reduced training feature vectors, testing feature vectors, training labels, testing labels.

Training, evaluating and applying the models

Naive Bayes

image

  1. Train

    To train our locally implemented model. Uncomment line 70-72. This will train the model and save it in ./models/gaussian_naive_bayes.pth To train Scikit's model. Uncomment line 82-84. This will train the model and save it in ./models/scikit_gaussian_naive_bayes.pth

  2. Evaluate

    To evaluate the models and get our metrics, we are using the helper function get_naive_bayes_metrics(). This function simply calculates the metrics and print them to the console. We can choose which model we want to use via the param get_naive_bayes_metrics(..., useScikit = True/False), if True we will use Scikit's model, False we will use our locally implemented model.

    image

  3. Apply

    To apply the model, we simply load a trained model and pass in our test feature vectors test_features_pca. We can then calculate our metrics using the predictions the model returns nb_predictions. To get the model prediction we are using .predict()

Decision Tree

image

  1. Train

    To train our locally implemented model. Uncomment line 95-100. Since we are training this model at different max_depth we will save it in ./models/decision_tree_model_{depth}.pth To train Scikit's model. Uncomment line 112-117. Since we are training this model at different max_depth we will save it in ./models/scikit_decision_tree_model_{depth}.pth

  2. Evaluate

    To evaluate the models and get our metrics, we are using the helper function plot_decision_tree_metrics(). This function calculates the metrics and plot them (metrics vs tree depth). We can choose which model we want to use via the param plot_decision_tree_metrics(..., useScikit = True/False), if True we will use Scikit's model, False we will use our locally implemented model.

    image

  3. Apply

    To apply the model, we simply load our trained models into a list. We can then iterate over this list to calculate our metrics for each model. Each model metrics are then store inside the following lists: train_accuracies, test_accuracies, precisions, recalls, f1_scores. For example, train_accuracies[0] will store the training accuracy for decision tree model with max_depth = 10. train_accuracies[len(loaded_dtcs)] will store the training accuracy for decision tree model with max_depth = 50. To get the model prediction we are using .predict()

Multilayer Perceptron

  1. Open the mlp.py file.

  2. Click the run button and wait for the program to finish executing.

  3. Train

    line 69, 223, 387, 563, 713 are where the model definition starts, first one being main and rest the variants. The models will be trained using the following training loop.

    image

  4. Test

The models are tested the using the following loop. The testing loop is same for all the models.

image

  1. Evaluation

    For evaluating the results the following code snippet is used. Precision, recall and f1 measure are calculated using the sickit-learn functions. For generating the confusion matrix we use the seaborn, numpy, pandas and matplotlib libraries.

    image

Concolutional Neural Network

  1. Open the cnn.py file

  2. Click run and wait for program to execute

  3. The Evaluation metrics for the model and printed

About

Image classification on CIFAR-10 dataset using self-trained AI models

Resources

Stars

0 stars

Watchers

0 watching

Forks

Contributors

Languages