INTERNSHIP FINAL DOCUMENTATION

FOOD CLASSIFICATION USING KERAS

Group Members

Routhu Shanmukh : [email protected]

Prasanth Botta : [email protected]

Reddy Upendra : [email protected]

Nakka Srinivas : [email protected]

Abstract:

Food classification is an important task in various applications, such as food

recognition, dietary monitoring, and food safety. In this project, we propose a deep learning-
based approach for food classification using Keras and TensorFlow frameworks, along with
Streamlit for model deployment and LabelImg for image annotation.The project involves
collecting a large dataset of food images from various sources and preprocessing the data to
prepare it for model training. We utilize deep learning models, such as EfficientNet and
ResNet, for feature extraction and classification. The models are trained using a supervised
learning approach with labeled food images and evaluated using various performance
metrics.To provide a user-friendly interface for model deployment, we utilize Streamlit, a
Python web application framework, to build an interactive web-based application. Users can
upload food images to the web application, and the trained model classifies the food type in
real-time, providing accurate predictions.To generate labeled images for model training, we
use LabelImg, a graphical image annotation tool, to annotate food images with relevant food
categories. The annotated images are used to train and fine-tune the deep learning models,
improving their accuracy and performance.The project aims to develop an efficient and
accurate food classification system that can be used in various food-related applications. The
use of deep learning models, along with Keras and TensorFlow, provides a robust and
scalable solution for food classification. The deployment of the model using Streamlit
provides a user-friendly interface for end-users to interact with the system. The use of
LabelImg for image annotation streamlines the data preparation process for model training.
The results of the project are expected to contribute to the field of food classification.
 CONTENTS

1. Introduction
1.1. Project Overview 1
1.2. Project Goals and Objectives 1
1.3. Project Background and Motivation 2
2. Literature Review 2
2.1 Review of Relevant Literature and Related Work 2
2.2 Summary of Existing Approaches and Techniques 3
3. Project Lifecycle 3
3.1. Project Lifecycle 3
4. Data Collection and Preprocessing 4
4.1. Data Sources and Acquisition 4
4.2. Data Collection Methods and Techniques 4
4.3. Data Quality Assessment 5
4.4. Data Preprocessing and Cleaning Techniques 5
4.5. Exploratory Data Analysis (EDA) and Visualization 6
5. Feature Engineering 7
5.1. Feature Selection and Extraction 7
5.2. Feature Transformation and Scaling 7
5.3. Handling Missing Values 7
5.4. Handling Categorical and Textual Data 7
5.5. Feature Engineering Techniques (e.g., feature encoding, feature creation) 8
6. Model Development 8
6.1. Model Training and Validation 9
6.2. Model Evaluation Metrics and Results 9-11
7. Model Deployment 12
7.1. Deployment Environment and Infrastructure 12
7.2. Deployment Techniques (e.g., cloud, on-premises, edge devices) 13
7.3. I/O Workflow (data input/output handling) 14
8. Model Architecture 15
8.1. Detailed Model Architecture and Design 15-17
9. Unit Testing and Model Evaluation 18
9.1. Unit Test case table 18
9.2. Model Performance Evaluation 19
10. Results and Analysis 19
10.1. Comparison with Baseline Models or Existing Approaches 20
11. Conclusion 20
11.1. Summary of Project and Achievements 21
11.2. Recommendations for Future Work 21
1. Introduction :

The main aim of this project is to build an image classification model that can
accurately classify images of different types of foods.It is a crucial task in the field of
computer vision, with a wide range of applications, including nutritional analysis, food
recommendation systems, and health monitoring. Using deep learning frameworks like Keras
and TensorFlow together with image labelling software like LabelImg to annotate the dataset
is one of a method for classifying foods. Deep learning packages like Keras and TensorFlow
offer effective resources for designing and refining neural networks, with just a small amount
of code, developers can quickly create and train deep learning models using Keras,
TensorFlow also offers a wide range of functions.We use LabelImg software to draw
bounding box around objects of interest in images and label them with the class
names.Similarly a dataset of few food classes is prepared and used for training the food
classification model.

1.1 Project Overview:

The model is trained on a dataset of some food images and it is able to accurately
classify different types of food into their respective categories.This project involves several
steps like data collection , data preprocessing , model architecture design , model training,
model deployment.
1. Data collection: collecting the dataset from various sources like kaggle food
datasets, google images , image databases, etc.
2. Data preprocessing : Preprocess the collected data by resizing the images to
a standard size, normalizing the pixel values of images and splitting the dataset into
training, validation and testing sets and also performing data augmentation techniques
such as flipping , rotation , zooming to increase the diversity of the dataset.
3. Model Architecture: Selecting an appropriate deep learning architecture for
food classification such as convolutional neural networks (CNNs) or pre-trained
models which are commonly used for image classification tasks and try different
architectures , parameters for increasing the models performance.
4. Model Training: Training the model with the training dataset and evaluating
the trained model on validation and testing sets to measure its performance using
metrics like accuracy , precision , recall and F1 score.
5. Model Deployment: Deploying the trained food classification model in real-
time to classify food images.In this project streamlit is used to create a web
application which takes the image from the user and classifies the image into a
respective food category.

1.2 Project Goals and Objectives:

● The basic goal of food classification using Keras is to create a model that can
precisely categorize food products into respective groups.
● By using a collection of food photos and annotating them using LabelImg software to
train a deep learning network, this may be accomplished.
● Deploying the model in streamlit and developing a user-friendly web application.
1.3 Project Background and Motivation:

● With the increase in food items and types of food all over the world, the need
for food classification models has been increased .A model that can accurately
identify and classify food products is what we want to create since it can be used for
food identification, that data can be used in diet planning, reducing food wastage, to
bring awareness in areas like schools about different types of food and health
monitoring by classifying foods that are beneficial or harmful for a health condition.

2. Literature Review:

Food classification using deep learning techniques has gained significant attention in
recent years due to the increasing availability of food images and the growing interest in
following dietary habits and their impact on health. In this section, we review some relevant
literature and related work on food classification using deep learning methods and Keras
library.

2.1 Review of Relevant Literature and Related Work:

A study on "Food Recognition using Deep Convolutional Neural Networks for

Diabetes Dietary Management" by Yanai et al. (2015): This study proposed a food
recognition system for diabetes dietary management using deep convolutional neural
networks (CNNs). The authors collected a large-scale food image dataset and trained a CNN
model using Keras to classify 100 different food categories. The proposed model achieved an
accuracy of over 80% and demonstrated the potential of deep learning for food recognition in
the context of diabetes dietary management.

A study on "Food Image Recognition using Deep Convolutional Networks" by Kawano

et al. (2017): This study proposed a food image recognition system using deep convolutional
networks (DCNs) and evaluated its performance on a dataset of food images. The authors
compared different DCN architectures, including VGGNet, GoogLeNet, and ResNet,
implemented using Keras, and found that ResNet achieved the highest accuracy in food
classification tasks.

"Food-101 - Mining Discriminative Components with Random Forests" by Bossard et

al. (2014): This study introduced the Food-101 dataset, a large-scale dataset with 101 food
categories, and proposed a method for food classification using random forests. The authors
compared the performance of random forests with different feature extraction techniques,
including deep convolutional neural networks (CNNs) implemented using Keras, and found
that CNNs achieved state-of-the-art accuracy in food classification tasks.

"Food Image Recognition Using Convolutional Neural Networks with Preprocessing

Methods(2018)" This work was presented by Rahman et al. A CNN-based system for food
image recognition with a variety of preprocessing techniques, such as data augmentation and
feature extraction. The accuracy of food categorization tasks was shown to be greatly
enhanced by data augmentation and feature extraction approaches when the authors assessed
the performance of various CNN architectures, including VGGNet and InceptionV3
implemented using Keras.
2.2 Summary of Existing Approaches and Techniques:

In most of the studies, large-scale food image datasets have been collected and they
include more categories and variation in food appearance, lighting conditions, angles and
followed the data preprocessing techniques such as resizing, cropping, and normalization are
commonly applied to standardize the images and reduce noise or variability. Differenet CNN
architectures, such as VGG, ResNet, InceptionV3 have been used for food classification
tasks. These architectures typically consist of multiple convolutional and pooling layers
followed by fully connected layers for feature extraction and classification. Mostly Transfer
learning is used, where pre-trained models are used as a starting point and also to leverage
the knowledge learned from large datasets in other domains.Accuracy, precision, recall and
F1 score are commonly used as evaluation metrics to assess the performance of food
classification models. Cross-validation, where the dataset is divided into multiple folds for
training and testing, is often used to obtain more reliable performance. Once the CNN model
is obtained some post-processing techniques like thresholding and softmax activation are
applied to the model and then deployed to the real-world applications, such as web or mobile
applications for classification tasks.These approaches can be adapted and customized based
on the specific requirements and constraints of the project to develop an effective food
classification system.

3. Project Lifecycle:

3.1 Project Lifecycle:

The lifecycle of a machine learning or deep learning project typically follows

several stages, which can vary depending on the project's complexity and requirements. The
general overview of the project lifecycle for a machine learning or deep learning project
include these major steps:

Model
Project
Model Model optimization Model
Planning and
selection training and Deployment
setup
Evaluation
4. Data Collection and Preprocessing :

The first step is to collect a dataset of food images that will be used for training,
validation, and testing the model. The dataset should include images of different food items,
captured under different lighting conditions, angles, and resolutions, to ensure the model's
robustness and generalization ability.The collected data may contain noise, inconsistencies,
or irrelevant information that needs to be cleaned before using it for training the model. This
could involve removing duplicate images, correcting image orientation, resizing images to a
consistent size, and removing irrelevant or misleading images.The dataset must then be
annotated with the proper labels or class annotations to identify the type of food item in each
photograph after it has been gathered. We can use automatic tools or annotate manually.After
data cleaning, the dataset is typically split into training, validation, and testing sets. The
training set is used to train the model, the validation set is used to tune hyperparameters and
validate the model during training, and the testing set is used to evaluate the final model's
performance.Before feeding the data into the model, it needs to be preprocessed to ensure
consistency and compatibility with the model's input requirements. This can involve resizing
images to a uniform size, normalizing pixel values, and converting images to the appropriate
format.

4.1 Data Sources and Acquisition:

Data sources and acquisition are important considerations in any machine learning or
deep learning project, including food classification.Some common data sources and
acquisition methods for collecting food image are kaggle food datasets, various other
websites and github , google images and we can also collect images on our own by capturing
using smartphones and then annotate them.

4.2 Data Collection Methods and Techniques:

Publicly Accessible Datasets: A number of publicly accessible datasets, including

Food-101, UEC-FOOD256, and Food-101N, feature food photographs. The food
classification model may be trained and tested using these datasets, which can be collected
from internet repositories. These datasets frequently have pre-annotated labels, which
facilitates the data collection process.

Web scraping: Food-related websites, blogs about food preparation, recipe websites,
and social networking sites that post images of food are all accessible to gather images of
food. web scraping involves retrieving images and related data from web pages. However,
it's crucial to make sure that the information gathered through online scraping meets with
rights of use and copyright rules.

Data Gathering Locally: Another option is to collect food images at home by taking pictures
of various foods using a digital camera or smartphone. Using this method, it is possible to
have more control over the data collection procedure, including the image quality, lighting,
and diversity. For collecting information and annotation, it could require extra time and
resources.We can use automatic labelling tools or manually label them using image
annotating tools.
4.3 Data Quality Assessment:

Data quality assessment is an ongoing process that should be performed at various

stages of the project, including during data collection, preprocessing, model training, and
evaluation. Ensuring high data quality is critical for building a reliable, accurate, and robust
food classification model using Keras and TensorFlow.
some typical steps and techniques for evaluating the quality of data:

Data Uniformity and Comprehensiveness: Data uniformity and thoroughness refer to the
uniformity and thoroughness of the data obtained. Making sure the data is consistently
labelled, categorised, and annotated as well as that there are no missing values or gaps in the
data's content is part of this process. For the purpose of developing a trustworthy and
accurate food classification model, consistent and comprehensive data is crucial.

Data Sampling and Splitting: To guarantee that the data used for training, validation, and
testing is fair and unbiased, proper data sampling and splitting techniques should be
implemented. Random samples of the data should be taken, and suitable portions should be
distributed across the training, validation, and test sets. This ensures that the model is tested
and assessed on a variety of data, and that the performance metrics are precise as well as
indicative of the applicability of the model.

4.4. Data Preprocessing and Cleaning Techniques :

The data preparation is a stage of life cycle of Machine Learning or Deep Learning where
the data is collected from different sources and later the data is processed by cleaning and
removing unwanted or irrelevant features , transforming in order to make the data useful for
training a model to gain useful insights or to make accurate predictions. Data Preparation
involves few steps such as:

1) Data collection
The data collection is the first step in the data preparation and the performance of
machine learning models is highly dependent on the quality and quantity of the data used to
train them

2) Data cleaning:
Data cleaning involves identifying and correcting errors, inconsistencies, or missing
values in the collected data. This can include removing duplicate images, correcting
mislabeled images, filling in missing values, and resolving inconsistencies in the data. Data
cleaning helps ensure that the data used for training and evaluation is accurate and reliable.

3) Data labeling
The data labeling is the process of assigning labels or names to the data so that it can
be used to train a model.In deep learning, the labels are assigned to images , videos or other
forms of data that are being used to train the neural network

4) Data augmentation
By using the data augmentation, a deep learning model can be trained on a larger and
more different dataset can be used for training,which can improve the model ability to adapt
to new, unseen data.We used data augmentation techniques like rotation, flipping to create
additional variations of original images,these augmented images are used in combination
with labeled dataset for training the model.

5) Data normalization
Data normalization is an important step in deep learning, as it can improve
performance and stability of the model during training.To improve the performance of the
machine learning model , we normalized the pixel values of the images by scaling the pixel
values to a range of [0,1] and subtracting the mean pixel values of the dataset.This makes the
model to learn more efficiently from the images.

6) Data splitting
Data splitting in deep learning helps to create distinct sets of data for training,
validation and testing the model.By splitting the data into separate sets we can evaluate the
performance of the model on new, unseen data and ensure the data is not overfitting to
training data.We split the dataset into training , validation and test sets.The training set is
used to train the machine learning model, the validation set is used to make adjustments to
model’s architecture and hyperparameters and the test set is used to evaluate performance of
the model.

7) Data preprocessing
Before training the deep learning model, we performed additional preprocessing steps
such as manually removing some irrelevant data,converting the images to arrays, resizing
images to uniform size.The preprocessed data is then used to train the machine learning
model.

4.5. Exploratory Data Analysis (EDA) and Visualization:

In food classification project using Keras and TensorFlow, EDA and data
visualisation are essential components for understanding the data, identifying trends, and
obtaining at accurate decisions. They direct the future processes of data preprocessing,
feature engineering, and model construction and offer insights into the data quality,
distribution, relationships, and potential issues.EDA additionally provides insights into any
possible data preprocessing steps that may be necessary. For instance, it might be essential to
scale the photographs during data preprocessing if they have different sizes or resolutions.
Similarly, EDA can assist in understanding the categories and how they are distributed if
their categorical characteristics need to be encoded.

Data visualization: Use various data visualization techniques to gain insights into the data.
This can include plotting histograms, box plots, scatter plots, bar charts, etc., to visualize the
distribution, relationships, and patterns in the data. Visualization techniques can be
implemented using Python libraries such as Matplotlib, Seaborn, and Plotly.
5. Feature Engineering:

5.1 Feature Selection and Extraction

Feature Selection : After characteristics have been retrieved, it is crucial to choose

the ones that are most applicable for categorization. The degree of dimensionality of the data
may be decreased by feature selection, which can also increase model effectiveness and
prevent overfitting. Principal component analysis (PCA), recursive feature elimination
(RFE), and correlation-based feature selection are a few of the methods for feature selection.

Feature Extraction : It is possible to extract features from pictures using a variety of

approaches, such as pre-trained convolutional neural networks (CNN) or manually created
feature extraction techniques like the Scale-Invariant Feature Transform (SIFT) or Histogram
of Oriented Gradients (HOG). A pre-trained CNN model, such VGG or ResNet, can reduce
the amount of time and work required for feature extraction.

5.2 Feature Transformation and Scaling

Feature Transformation : The process of feature transformation consists of changing the

representation of the input features. Data augmentation is a typical strategy for picture
classification, which entails transforming the images—such as rotation, flipping, and
zooming—to provide more training examples. This can lessen overfitting and increase the
model's accuracy.

Feature Scaling : Feature scaling is the process of adjusting the input features to a
comparable range in order to keep some features from outweighing others. Normalisation,
which transforms the input feature values to have a mean and standard deviation of 0 and 1,
is a popular approach for feature scaling.

5.3. Handling Missing Values

When using Keras to classify images from the Food Classification 101 dataset,
handling missing values is a crucial step in the preparation of the data. There are several
approaches you may take to handling missing values, depending on the dataset and the type
of missing values. After handling the missing variables, you may train the model and assess
its effectiveness.

5.4. Handling Categorical and Textual Data

One-hot encoding and label encoding are two approaches that may be used to transform
categorical data, such as the cuisine of the dish, into numerical values. One-hot encoding
reduces each category to a binary vector of 0s and 1s, whereas label encoding gives each
category a distinct number.Using methods like word embeddings or bag-of-words, textual
data, such as the names of the foods, may be transformed into numerical vectors. When
compared to the bag-of-words representation of text documents as sparse vectors of word
frequencies, word embeddings show each word as a dense vector in a high-dimensional
space.
5.5. Feature Engineering Techniques (e.g., feature encoding, feature creation)

Encoding of Features: Object detection models need input features that record the
visual traits of the objects in the pictures. The characteristics that may be utilised as input into
the object detection model can be extracted from the pictures using feature encoding
techniques like Histogram of Oriented Gradients (HOG) and Scale-Invariant Feature
Transform (SIFT).

Feature Creation: In addition to feature encoding, new features may be made by merging old
ones or by performing extra statistical calculations on the picture data. Examples of extra
characteristics that may be employed to enhance the effectiveness of the object recognition
model include the aspect ratio, colour histogram, and texture data of the food items in the
photos.

6.Model Development:

Model development is an iterative process that involves designing, training,

evaluating, and optimizing the machine learning or deep learning models for food
classification. It requires careful consideration of the data, model architecture,
hyperparameters, and optimization techniques to achieve high accuracy and performance.

Data Preparation:
Collecting and preprocessing food images. This typically involves acquiring a labeled dataset
of food images our preparing your own dataset by using labelling tools like LabelImg for
annotation of collected images and splitting them into training, validation, and testing sets.
You may need to resize the images, normalize pixel values, and augment the data with
techniques like rotation, flip, and zoom to increase the diversity of your dataset.

Model Selection:
Following the pre-processing steps like resizing ,normalization and cleaning of the data, the
following stage is to pick the best machine learning algorithm for the purpose of food
classification. This may include evaluating the performance of several different types of
model algorithms to find which one works the best.

Model Architecture:
Design the model architecture of your neural network using Tensorflow and Keras. The
layers, their types (such as convolutional, pooling, and dense), and their parameters (such as
the quantity of filters, kernel size, activation function, and pooling technique) must be
specified. To determine which architecture is appropriate for your food classification activity,
you can experiment with a variety of them.
6.1. Model Training and Validation

Model training:
Apply the training set to your model and computing the loss, and update the model's weights
based on the optimizer. Iterate this process for multiple epochs (complete passes through the
entire dataset) to allow the model to learn from the data and then evaluate the model for using
validation set.

Model Validation:
Once you are satisfied with your model's performance on the validation set, test it on the
unseen testing set to get an estimate of its real-world performance. Compute the final
evaluation metrics and make any necessary adjustments to the model based on the testing
results.

6.2 Model Evaluation Metrics and Results:

Model evaluation metrics are used to assess the performance of the trained food classification
model. These metrics provide quantitative measures of how well the model is performing and
help in comparing different models or iterations of the same model. Some common model
evaluation metrics for the food classification project include:

Accuracy: It is the ratio of correctly predicted food samples to the total number of samples in
the evaluation dataset. It provides an overall measure of the model's correctness.

Precision: It is the ratio of true positives (correctly predicted positive samples) to the sum of
true positives and false positives (incorrectly predicted positive samples). It measures the
model's ability to correctly predict positive samples.

Recall : It is the ratio of true positives to the sum of true positives and false negatives
(incorrectly predicted negative samples). It measures the model's ability to correctly identify
all the positive samples.

F1 Score: It is the harmonic mean of precision and recall. It provides a balanced measure of
both precision and recall.

Confusion Matrix: It is a matrix that shows the count of true positive, false positive, true
negative, and false negative predictions. It provides detailed information about the model's
performance for each class.

Classification Report: It provides a comprehensive summary of the model's performance,

including precision, recall, F1 score, and support (number of samples) for each class.

ROC Curve (Receiver Operating Characteristic Curve): It is a graphical plot of the true
positive rate (sensitivity) against the false positive rate (1-specificity) at various classification
thresholds. It helps in evaluating the model's performance in terms of trade-offs between
sensitivity and specificity.

Precision-Recall Curve: It is a graphical plot of precision against recall at various

classification thresholds. It helps in evaluating the model's performance in terms of trade-offs
between precision and recall.
Confusion Matrix
Classification in machine learning is the method of classifying a given collection of data into
multiple categories. The confusion matrix is used in Machine Learning to assess the
effectiveness of categorization models.An illustration of a machine learning model's
performance on a set of test data is a confusion matrix. It is widely used to evaluate the
effectiveness of classification models that seek to predict a classification for each instance of
an input. The matrix displays the number of true positives, true negatives, false positives, and
false negatives that the model generated on the test data.

True Positive(TP): True positive is when the model predicted a positive value, and it is
correct.

True Negative(TN): True Negative is when the model predicted a negative value, and it is
actually negative.

False Postive(FP): False positive is when the model predicted a negative value, and it is
actually positive.

False Negative(FN): False Negative is when the model predicted a negative value, and it is
actually positive.
The above figure shows the confusion matrix between the actual and the predicted ones for
the model we have trained using 6 classes of data namely apple pie, baby back ribs, baklava ,
beef carpaccio , beef tartare, beet salad.The confusion matrix is a square matrix where the
row and column correspond to the model's predicted value and actual values, respectively. A
confusion matrix, in particular, shows the specific point at which a classification model loses
becomes unsteady when making predictions.The real value in statistics is the value derived
from observation or measurement of the available data. It's also referred to as the observed
value. The expected value is the variable's predicted value as determined by regression
analysis.
7.Model Deployment

7.1 Deployment Environment and Infrastructure:

Developers are able to construct interactive dashboards, data visualizations,

and other apps using Streamlit's wide range of capabilities, including buttons, sliders, drop-
down menus, and more. Streamlit is a great tool for creating machine learning applications
since it also provides a simple interface with well-known Python libraries like Pandas, Scikit-
learn, and TensorFlow. Streamlit offers a straightforward syntax for creating web apps and is
intended to be simple to use. Additional capabilities include hot reload, which enables
programmers to view changes to their code immediately. In a nutshell, Streamlit is a robust
and user-friendly Python toolkit for creating interactive web-based applications in predictive
modeling, data analysis, and various other uses. It is the perfect tool for developers of all skill
levels due to its straightforward syntax and interoperability with well-known Python
modules.

Deployment environment and infrastructure play a crucial role in making the trained
food classification model accessible and usable by end-users. Streamlit is a popular Python
library that allows for easy deployment of machine learning models as interactive web
applications. Here's an overview of the deployment environment and infrastructure using
Streamlit for a food classification project:

Deployment Platform: Streamlit can be deployed on various cloud platforms such as AWS,
Google Cloud Platform, or Heroku. The selected platform should provide the necessary
compute resources, storage, and scalability to support the expected usage of the application.

Web Application Framework: Streamlit provides a simple and user-friendly framework for
building interactive web applications. It allows for creating a customized user interface with
different components such as buttons, sliders, and plots to interact with the trained food
classification model.

Front-end Design: Streamlit offers flexibility in designing the front-end of the web
application. The front-end can be designed using Streamlit's built-in components or
customized with HTML, CSS, and JavaScript to create a visually appealing and user-friendly
interface.

Backend Integration: Streamlit allows for easy integration with the backend Python code that
includes the trained food classification model. The backend code can be written in Streamlit's
Python script, which can take care of model prediction and data processing tasks.

Model Deployment: The trained food classification model can be integrated into the
Streamlit application by loading the saved model weights or using the trained model object
directly in the Streamlit script. This allows for seamless model inference during user
interactions with the web application.

Data Handling: Streamlit provides functionalities for handling user input data, such as file
uploads or form submissions, and passing the data to the backend for model inference. It also
offers options for securely storing user data, if required.
Deployment Process: The deployment process involves creating a Streamlit app, configuring
the deployment environment, and deploying the Streamlit app to the selected platform. This
may involve setting up virtual machines, containers, or serverless computing environments,
depending on the chosen deployment platform.

Security Considerations: Ensuring the security of the deployed application and user data is
critical. Proper authentication, authorization, and encryption measures should be
implemented to protect against unauthorized access, data breaches, or other security threats.

Monitoring and Maintenance: Regular monitoring of the deployed application's performance

and health, as well as maintaining updates and backups, is necessary to ensure smooth and
uninterrupted operation of the application.

7.2 Deployment Techniques:

Local Deployment: Streamlit allows for running the web application locally on the
developer's machine for testing and development purposes. This can be achieved by
installing Streamlit library and running the Streamlit script on a local Python environment.
The web application can then be accessed via a web browser on the same machine.

Cloud Deployment: Streamlit can be deployed on cloud platforms such as AWS, Google
Cloud Platform, or Heroku, which offer scalable and reliable infrastructure for hosting web
applications. This involves setting up a virtual machine or container, installing the necessary
dependencies, and running the Streamlit script. The web application can then be accessed via
a public URL, making it accessible to users from anywhere.

Monitoring and Logging: Streamlit offers built-in logging functionalities that allow
for monitoring and logging application events and errors. Additionally, external monitoring
and logging tools can be integrated with the Streamlit application to keep track of the
application's performance, health, and usage.

Once the deployment environment and infrastructure are set up, the trained food
classification model can be accessed and used by end-users via the Streamlit web application.
Users can interact with the web application, upload food images, and receive predictions
from the model in real-time. The deployment environment and infrastructure should be
regularly monitored and maintained to ensure the continued smooth operation of the
application.
7.3 I/O Workflow (data input/output handling):

The user passes an image as input in the webpage , then the images will be stored in
temporary database and some preprocessing steps like resizing , changing threshold value of
the image.Then preprocessing is done, the input image is then fed into the TensorFlow
interpreter as input data. The interpreter takes this input data and performs the computations
defined by the model's computational graph.Once the computations are completed, the
interpreter produces the output predictions based on the model's defined computations. This
will be the predicted class label for the classified image.If the image passed by the user is a
food image then the prediction will be generated or else the model will not classify the
image.To get a successful classification the user needs to pass a food image.
8.Model Architecture

8.1 Detailed Model Architecture and Design:

Food classification is an important task in computer vision, with applications

in nutrition analysis, food recommendation systems, and dietary monitoring.We intent to
follow the waterfall model for this project.Because,The sequential nature of the model makes
it easy to follow and manage,especially for straightforward projects.This model is also
helpful in identifying the requirements and scope of the project at an early stage.Another
advantage of the waterfall model is that it provides a clear picture of the project timeline.

Waterfall Model:

The waterfall model is a sequential design process used in software development. It is a

traditional approach to software development, where the process is divided into sequential
phases, and each phase must be completed before the next one begins.
The phases of the waterfall model for this project are:
1. Requirements
2. Planning
3. Implementation
4. Testing
5. Deployment
6. Review
7. Maintenance

The model assumes that all requirements are known at the beginning of the project and that
the design is fixed before implementation begins.
One of the main advantages of the waterfall model is its simplicity and clarity. It is easy to
understand and follow, making it ideal for projects where the requirements are well
understood and the technology is stable.

In this report, we present the detailed model architecture and design for food classification
using two popular deep neural network architectures, EfficientNet and ResNet.

EfficientNet Architecture:

EfficientNet is a scalable neural network architecture that achieves state-of-the-art

performance in various computer vision tasks. It is based on the compound scaling of model
depth, width, and resolution, which allows for optimal performance. The EfficientNet
architecture consists of multiple convolutional blocks, pooling layers, fully connected layers,
and dropout regularization.

The convolutional blocks contain multiple convolutional layers with different kernel sizes
and feature map sizes, followed by batch normalization and ReLU activation. The pooling
layers are used for spatial down-sampling, and the fully connected layers produce the final
classification output. Dropout regularization is applied to prevent overfitting during training.
The softmax activation function is used at the end of the network, and the categorical cross-
entropy loss is used as the training loss.

ResNet Architecture:

ResNet is a deep neural network architecture that introduces skip connections to overcome
the limitations of deep networks. It consists of convolutional blocks, skip connections,
pooling layers, fully connected layers, dropout regularization, softmax activation, and
categorical cross-entropy loss.The convolutional blocks contain multiple convolutional
layers, followed by batch normalization and ReLU activation. The skip connections directly
connect the input and output of each convolutional block, allowing the gradient to flow
directly through the skip connections and facilitating the training of deeper networks. Pooling
layers are used for spatial down-sampling, and fully connected layers produce the final
classification output. Dropout regularization is applied to prevent overfitting. Softmax
activation is used at the end of the network, and categorical cross-entropy loss is used as the
training loss.

Model Training and Hyperparameters:

Both EfficientNet and ResNet models are trained using a large dataset of food images,
comprising various food categories. The dataset is divided into training, validation sets. The
models are trained with a learning rate of 0.001, momentum of 0.9, and batch size of 32. The
models are trained for 20 and 10 epochs with early stopping based on validation loss. Data
augmentation techniques such as random rotation, flip, and zoom are applied during training
to improve model generalization.

Model Evaluation Metrics:

The trained models are evaluated using various evaluation metrics, including accuracy,
precision, recall, F1-score, and confusion matrix. The accuracy measures the overall
classification performance of the models, while precision, recall, and F1-score provide
insights into the model's performance for each class. The confusion matrix provides a visual
representation of the model's predicted and actual classifications.
9. Unit Testing and Model Evaluation:

9.1Unit Test case table

Test Case Description Pre-Requisite Expected Result

Test case 1: Image Prediction Load an image of a Kulfi Predicted class: "Kulfi" with
a high confidence score
Test case 2: Image Prediction Load an image of a Dal Predicted class: "Dal
makhani makhani" with a high
confidence score
Test case 3: Image Prediction Load an image of a Dhokla Predicted class: "Dhokla"
with a high confidence score
Test case 4: Image Prediction Load an image of a fried rice Predicted class: "fried rice"
with a high confidence score
Test case 5: Image Prediction Load an image of an idli Predicted class: "Idli" with a
high confidence score
Test case 6: Image Prediction Load an image of a jalebi Predicted class: “jalebi" with
a high confidence score
Test case 7: Image Prediction Load an image of a non-food Predicted class: No
item (e.g., car) classification is done
Test case 8: Image Prediction Load an image with low Predicted class: "Unknown"
image quality or noise (class for non-food items)
with a low confidence score
Test case 9: Input Validation Provide an empty input Raise an error: "Input image
image cannot be empty"
Test case 10: Input Provide an image in an Raise an error: "Unsupported
Validation unsupported format (e.g., image format. Please provide
PDF) an image in a valid format
(e.g., JPEG, PNG)"
Test case 11: Input Provide a grayscale image Raise an error: "Input image
Validation must be in color (RGB)
format"
Test case 12: Input Provide an image with Raise an error: "Input image
Validation incorrect dimensions dimensions do not match
expected dimensions"
Test case 3: Image Prediction Load an image of a kaathi Predicted class: "kaathi rolls"
rolls with a high confidence score
Test case 3: Image Prediction Load an image of a kadai Predicted class: "kadai
paneer paneer" with a high
confidence score
Test case 3: Image Prediction Load an image of an pizza Predicted class: "fried rice"
with a medium confidence
score(due to insufficient
training images )
Test case 3: Image Prediction Load an image of an samosa Predicted class: "kaathi rolls"
with a medium confidence
score(due to insufficient
training images )
9.2 Model Performance Evaluation:

The trained models are evaluated using various evaluation metrics, including
accuracy, precision, recall, F1-score, and confusion matrix. The accuracy measures the
overall classification performance of the models, while precision, recall, and F1-score
provide insights into the model's performance for each class. The confusion matrix provides a
visual representation of the model's predicted and actual classifications.The model trained
using efficient-net pretrained has gained 92 percent accuracy and only few wrong
classifications were made due to insufficient training data for those categories.

10.Results and Analysis

10.1. Comparison with Baseline Models or Existing Approaches:

The datasets like “food-101”, “kaggle indian foods” has been widely used as
a benchmark dataset for food recognition tasks, and several baseline models and existing
approaches have been proposed and evaluated on this dataset.

The "food classification 101" dataset's most popular models for classifying foods are
convolutional neural networks (CNNs). On this dataset, CNN-based methods have shown
excellent accuracy rates, with some research reporting accuracies of up to 90%. To increase
the overall accuracy of food identification models, ensemble methods combine the results of
many models. According to a number of research, ensemble approaches can boost models'
accuracy on the "food classification 101" dataset.

Transfer Learning: A well-liked method for training on the "food classification 101"
dataset, transfer learning starts with pre-trained models. It has been demonstrated that
transfer learning may save training time while improving the accuracy of food identification
models.

11.Conclusion :

The achievements of this project include the successful development and deployment
of an efficientnet and resnet-based food classification system, achieving high accuracy and
performance metrics. The project has the potential to be utilized in various real-world
applications, such as food recognition for dietary monitoring, food recommendation systems,
and nutrition analysis.
11.1. Summary of Project and Achievements:

In summary, this project has demonstrated the successful implementation of

efficientnet and resnet models for food classification, achieving high accuracy and
performance. The deployed web application provides a user-friendly interface for users to
upload food images and obtain real-time predictions, making it a valuable tool for food
recognition and analysis applications.Model evaluation was conducted using various
performance metrics such as accuracy, precision, recall, and F1-score. The models achieved
an accuracy of 92% and a F1-score of 0.94, indicating high performance in classifying food
images into different categories.The project was successfully deployed using Streamlit, a web
application framework, for interactive user interface.

11.2. Recommendations for Future Work :

Increase the accuracy of current models, will focus on a number of topics, one
of which being enhancing the precision of current models. Experimenting with various model
architectures, hyperparameters, and preprocessing methods may be done to achieve this.The
dataset's size should be increased The " Indian food" dataset is a nice place to start, however
in comparison to some of the larger picture datasets, it is rather tiny. Increasing the dataset
size can help models run more accurately and give examples of meals that are more
varied.Include a temporal component.