A Mini Project Report on

Food Calorie Estimation

Submitted in partial fulfilment for the

degree of Bachelor of Technology in
Computer Science and Technology

Submitted by
Tanvi Gharte (12)
Kajal Hake (13)
Himani Patil (31)

Under the guidance of

Prof. Sumedh Pundkar

Usha Mittal Institute of Technology

S. N. D. T Women’s University
Mumbai- 400049
2020- 2021

i
 CERTIFICATE

This is to certify that Miss. Tanvi Gharte, Himani Ravindra Patil and Kajal Hake
has successfully completed seminar work on the topic Food Calorie Estimation
satisfactorily in partial fulfillment for the Bachelor’s Degree in Computer Science and
Technology under the guidance of Prof. Sumedh Pundkar during the year 2020- 2021
as prescribed by the SNDT Women’s University.

Guide Head Of Department

Prof. Sumedh Pundkar Dr. Sanjay Pawar

Principal
Dr. Sanjay Pawar

Examiner 1 Examiner 2

ii
 Abstract

An important attribute for a healthy body depends on the number of calories consumed,
hence monitoring calorie intake is necessary to maintain good health. As the like for trend
and variety of fast food items is increasing, people are also becoming more aware and con-
scious about calorie intake as the higher number of calorie intake creates a lot of problems
like obesity, hypertension, high cholesterol, diabetes, heart attack etc that invokes the need
of medical consultancy. People suffering from such problems require an easy way to control
their calorie content in their daily diet.
In order to keep track of the amount of calorie intake of a person, we made a food
recognition and calorie estimation system that identifies the food and estimates the calorie
of the food image which is given as input .We trained more than 1500 images in each category
for our model. We have used CNN (Convolutional Neural Network) as a classifier for food
recognition and obtained 97.11% accuracy. Based on the food weight in grams, volume is
estimated and the calorie of the food item is determined.
 Acknowledgement

It gives us great pleasure in presenting the mini project report on ‘Food Calorie Estima-
tion’. We would like to take this opportunity to thank our internal guide Prof. Sumedh Pund-
kar sir for his continuous support, patience, motivation, enthusiasm and immense knowledge.
He helped us in doing the research work and implementation during which we got to learn
ample of new things. However, we would like to express our deep appreciation and indebt-
edness to our professors for their endless support, kindness and understanding during the
project duration.
Our thanks and appreciation goes to our team members in developing the project. All
in all, we would like to thank everyone involved in this project and helped us with their
suggestions to make the project better.

Date:

i
Contents

Abstract i

List of Tables iii

List of Figures iv

1 Introduction 1
1.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

2 Literature Survey 3

3 Existing System
and Proposed System 7
3.1 Existing System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
3.2 Proposed System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
3.2.1 Image Preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
3.2.2 Image Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
3.2.3 Image Segmentation and Feature Extraction . . . . . . . . . . . . . . 10
3.2.4 Volume Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3.2.5 Weight and Calorie Estimation . . . . . . . . . . . . . . . . . . . . . 12

4 Architectural Overview 13

5 Technology Used 15
5.1 Hardware and Software
Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

6 Implementation Details 19

7 Conclusion 30
7.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
7.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
7.3 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

References 32

ii
List of Tables

2.1 Review of Literature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

6.1 Different Camera Brands and Light Environment . . . . . . . . . . . . . . . 20

6.2 Food Nutritional Information . . . . . . . . . . . . . . . . . . . . . . . . . . 21

iii
List of Figures

3.1 CNN Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

4.1 System Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

4.2 System Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

6.1 Dataset Used . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

6.2 Training Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
6.3 Output of Demo1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
6.4 Output of Demo2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

iv
Chapter 1

Introduction

1.1 Problem Statement

Obesity in adults and youngsters is taken under consideration as a world epidemic. The
main reason for obesity is excessive food consumption and lack of physical activities. Also,
as people are becoming used to a sedentary lifestyle, they are involuntarily carried away from
being aware of their food energy intake. Therefore, it is necessary to measure diet accurately.
Nowadays it is very difficult for a person to track the calories consumed by them. The intake
of calories plays a very vital role in one’s healthy lifestyle. Due to improvement in people’s
standard of living, obesity rates are increasing at an alarming speed and this is reflective to
the risks in people’s health.
The main objective of project study is: (1) to estimate the calorie content of food items
which includes fruits, vegetables, non-vegetarian products (chicken, fish) using deep learning
and image processing techniques. (2) to develop a system that allows user to upload picture
of the food item and measure its volume and calories; (3) to develop a web application which
will alert people about their daily diet and calorie consumption. (4) to test and develop the
functionality of the system to improve the accuracy in measuring the calorie content of the
food.
This study can be used as a guide for those who are undergoing a healthy diet lifestyle
on their own to help them determine which type of food they can eat if they are following
a limited calorie intake. This study can also help future researchers if they need additional
methods to adapt for determining the calorie content for fruits, vegetables, chicken and fish,
in which they can also apply to other types of foods such as meat, liquid items etc.

1.2 Introduction
Obesity and overweight have traditionally been linked to intake of high calorie food and
lifestyle. Obesity is defined using Body Mass Index (BMI) of the individual. A person is
considered obese when the (BMI) is greater than or equal to 30 (kg/m2 ). Unfortunately, more
and more people will meet criteria for obesity. The main cause of obesity is the imbalance
between the amount of food intake and energy consumed by the individuals. It is noticed
that obesity or overweightness cause a number of chronic diseases such as breast and colon

1
1.2. INTRODUCTION CHAPTER 1. INTRODUCTION

cancer, blood pressure, heart diseases, type II diabetes. Food is the key of human’s body.
So, a diet plan always needs to take into consideration the total number of calories to be
consumed to maintain a fit and healthy life. But, in most of the cases, people face difficulties
in estimating the amount of food intake due to lack of nutritional information, which includes
the manual process of writing down this information, and other reasons. As such, it will be
useful if there is a system to keep track and maintain the calorie intake. Therefore correct
prediction of food calorie is equally important.
In recent years, many systems built for performing the same task require the manual
input of food items, their ingredients, amount and some other specific details which becomes
a tiresome process and the users of any system wants it to be user friendly and easy to access
and use. Using image as an input and providing to the system to give the result is supposed
to be a lot easier. Hence, in our proposed model we allow users to upload the picture of a
food item as input and the rest of work will be accomplished by the machine rather than
the user.
So, to reduce this difficulty of tracking the calories consumed by a person we came up with
an idea of building an automated system for prediction of calories.We are proposing an image-
based Food recognition and Calorie Estimation System which will only require the image of
the food item for the prediction of the calories present in the food item. This system will
allow users to upload food images in different views like top view, side view, etc. For image
classification we have used Convolutional Neural Network (CNN). Food(s) and calibration
objects are detected by an object detection method called CNN.We trained more than 1500
images in for our model. In this algorithm, we are providing an image to convolutional as an
input and we get a convolutional feature map as output. Using image segmentation we are
dividing an image into different regions based on contour, characteristics of pixels of food
image. We calculated calories with the help of segmented images using formulas and thumb
whose dimensions are already known to us. Using a segmented image, we first calculate the
volume of the food item and then mass of the item and finally the calorie of the item using
relation between the mass and calorie of a food item.
Experimental results of combination of different segmentation methods such as color,
texture, graph-cut segmentation and deep learning neural network showed an accuracy of
97.11%. An accurate estimation of daily nutritional intake provides a useful solution for
keeping healthy and to prevent diseases.

Food Calorie Estimation 2

Chapter 2

Literature Survey

The aim of this section is to provide the pros and cons of existing methods and the needs for
alternative approaches. This section will present a number of the most common food intake
measuring methods, plus their advantages and disadvantages. As a result, we will show the
importance of our calorie-intake measurement system which can be used for normal people
and clinical purposes in order to enhance the treatment methods for people who suffer from
obesity and overweightness. Different calorie measurement models have been developed for
research and patient health diagnosis and prescription. Most of the systems either developed
or proposed lack either in user friendliness and accessibility or performance and accuracy or
some other parameters. Several such models are enumerated below:

Table 2.1: Review of Literature

Sr. Year Title Author Findings Limitations

No.
1 2014 Intelligent Parisa Proposed a system that in- The system often
SVM Pouladzadeh, volves capturing an image fails to detect var-
Based Shervin Shir- of the food and processing ious food portions
Food mohammadi, it through predefined steps, in mixed food; it
Intake Tarik Arici which follow a pipeline ar- also fails to seg-
Mea- chitecture. These steps in- ment them prop-
surement clude food image segmen- erly. Can detect
System tation and food portion six categories of
recognition(SVM). Calorie fruits only.
measurement is done us-
ing nutritional fact ta-
bles.The area measurement
technique proposed is based
on a depth estimation tech-
nique. Accuracy Obtained:
92.2%

3
 CHAPTER 2. LITERATURE SURVEY

Sr. Year Title Author Findings Limitations

No.
2 2016 Fruit Manpreet Proposed the system for Only 5
Recog- Kaur Bas- fruit recognition and its fruits(Apple,
nition ant Singh calorie measurement based banana, lemon,
and its Sardar, Dr. on the shape, color and tex- Sapodilla,
Calorie Sayyad D. ture. Using image segmen- mango) were
Measure- Ajij tation methods, the fruit used as an im-
ment: portion area with other ge- age dataset.No
An ometrical parameters and method applied
Image texture features were ex- for calorie calcu-
Pro- tracted fron the fruit im- lation. Already
cessing age. After that, the multi saved Nutrients
Ap- class support vector ma- fact table was
proach chine(SVM) was used to used for estimat-
classify and identify the ing correct calorie
type of fruit. That allowed category.
the system to extract fea-
tures in an exclusive ap-
proach that will give the
ability to calculate the calo-
rie of the fruit with the
help of nutrient fact table.
Databse used: USDA na-
tional nutrient database.
3 2017 Predicting R. Kohila Proposed a mobile applica- FCM algorithm
calorific and R. tion which aims to measure suffers from some
value for Meenaku- the calorific value of mixed weaknesses such
mixed mari food by image processing, as initializing
food where an image will be cap- clusters center,
using tured through the mobile determining the
image phone’s camera which then optimal num-
process- undergoes various processes ber of clusters
ing such as fuzzy c means(FCM and sensitive
-grouping a set of objects in to noise.System
such a way that the objects can recognize
in the same group are simi- only the fruit
lar to each other) clustering vegetables.Fails
and Morphological to detect non-veg
and fast food
items.

Food Calorie Estimation 4

 CHAPTER 2. LITERATURE SURVEY

Sr. Year Title Author Findings Limitations

No.
4 2018 Calorie Hemraj The proposed model fo- The system
Esti- Raikwar, cuses on estimation of num- used an SVM
mation Himanshu ber of calories in the food algorithm which
from Jain, Amit item by just taking its im- is not suit-
Fast Baghel age as input. Some im- able for large
Food age processing operations datasets.Also this
Images are performed on the im- method does not
Using age of food item followed perform very well
Support by a machine learning tech- when the data set
Vector nique known as support has more noise
Machine vector machine(SVM). Ac- i.e. target classes
curacy Obtained : 90.66% are overlapping.
Dataset used: PFID (Pitts- Dataset consists
burgh Fast Food Image of only fast food
Dataset) and website of items (Pizza,
Shutterstock. Burger, Donut,
Burrito and
Samosa) .The
model proposed
can be used to
identify food in
the image but it
cannot predict
hidden ingredi-
ents and pieces of
food
5 2018 Deep Yanchao The method takes two food For each image,
Learning- Lianga, Jian- images as its inputs: a top there is only a
Based hua Lia view and a side view; each One Yuan coin
Food image includes a calibra- as a calibration
Calorie tion object which is used object which
Esti- to estimate images scale is less flexible
mation factor.Use Faster Region- than using a
Method based Convolutional Neural thumb.Matlab
in Di- Networks to detect objects was used for the
etary and GrabCut as segmenta- implementation
Assess- tion algorithms. Dataset whereas python
ment used: ECUSTFD is far better than
MATLAB in
terms of perfor-
mance.

Food Calorie Estimation 5

 CHAPTER 2. LITERATURE SURVEY

Sr. Year Title Author Findings Limitations

No.
6 2019 Food Nimish Developed a system for Less accuracy
Calorie Tufchi,Kalash fruit recognition and calo- obtained.Only
Predic- Nikose rie estimation for the same 7 fruits (Apple,
tion ,Katha Patel using CNN algorithm. banana, car-
Dataset used : FOODD rot, cucumber,
dataset. Accuracy ob- orange, onion,
tained:90% tomato) were
used as a dataset.
Sometimes fails
to detect correct
fruit because of
less accuracy.
7 2019 Jessie R. Convolutional Neural Net- Dataset images
Determina Balbin, work (CNN) was the in- consisted of
tion Leonardo tegrated algorithm to rec- only fish and
of Calo- D. Valiente, ognize food on an image. chicken. If Food
rie Kim Martin Graph Cut image segmen- other than fish
Content P. Monsale, tation was used to deter- and chicken were
in Differ- Emmanuel mine the regions of the food tested, the system
ent Type D. Olorvida, in the image. Volume es- shows “invalid”
of Foods Gerard timation based on its mea- and the measure-
using Glenn V. surement on the area of the ments show 0
Image Salazar, segmented food image and value.
Process- Lyzza Marie the height of the food mea-
ing L. Soto sured by fixed-placed ul-
trasonic sensor. Dataset
used: 2376 training images
of the different chicken and
fish. Accuracy obtained:
food detection accuracy of
91.82%, a mean accuracy of
88.18% for the calorie esti-
mation.

Food Calorie Estimation 6

Chapter 3

Existing System
and Proposed System

3.1 Existing System

In this section, we present some of the most common food calorie measuring methods that
have been developed in the last few years. The objective here is to describe the main advan-
tages and drawbacks of these methods, in order to demonstrate the novelty and contribution
of our proposed system.
There are some methods existing for dietary assessments which involve self-reporting and
manually recorded instruments. Most of these systems ask the user to input the details of the
ingredients presented in the food. But it has some issues with the evaluation of calorie con-
sumption. Calorie consumption by a participant is prone to be biased, i.e. underestimating
and under reporting of food intake to increase the accuracy and to reduce the bias current
methods are enhanced by mobile cloud computing system, which makes use of devices such
as smartphones to capture dietary and calorie information. Next to this step, dietary and
calorie information are analyzed by employing the computing capacity of the cloud for an
assessment. However, users still have to enter the information manually. For food classifi-
cation, previous work was concentrated on basic machine learning algorithms like Random
Forest and SVM with hand-tailored features. These methods are generally based on relative
or spatial relationships of features. But these methods come with computational cost at a
large scale. Random forest came up with an accuracy of 90% whereas SVM came up with
an accuracy of 92.2%. The system often fails to detect various food portions in mixed food;
it also fails to segment them properly.
Though plenty of research and development efforts have been made in the field of visual
based dietary and calorie information analysis, the efficient extraction of information from
food images remains a challenging issue. In some methods for classification, K-NN (k nearest
neighbor) is used which takes much time to train the images and classify. If data is not
assumed properly, data loss may occur.A common problem of most of the models designed
in this area is the identification of meal. Most of the models developed are found to have
some demerits. Some of them lack precision, others require a lot of user intervention.

7
 CHAPTER 3. EXISTING SYSTEM
3.2. PROPOSED SYSTEM AND PROPOSED SYSTEM

3.2 Proposed System

To build our system, we will take advantage of the existing methods and overcome all their
drawbacks.

ˆ Proposed food recognition and calorie measurement system aims to allow the user to
upload the picture of food items at any time and any place to measure calorie value
from the uploaded photo. This unique method will provide more accurate results than
other methods.

ˆ We provide a calibration reference(user’s thumb) in our images, allowing more accurate

measurement of the size of food ingredients, leading to higher accuracy in calories
measurement.

ˆ We are using a different variety of food dataset (e.g. fruits, fast food, vegetables ,
non-veg items) for training our model.

ˆ We are building our own food dataset with multiple brands of cameras to capture
the image of the same food item, providing an opportunity for the food recognition
system to become more robust with changes in camera brands. We provide multiple
shooting angles for the same food item, allowing the development of more accurate
food recognition methods.

ˆ We are using CNN algorithm to obtain more accuracy. Also we are using Python
language rather than Matlab for better performance.

ˆ Our aim is to obtain more accuracy (greater than 94 percent) than existing models.

3.2.1 Image Preprocessing

Images give better results on a white background and also if they are fed to the system
after presenting them to the system as they are expected to be and hence pre-processing
of images is required. It includes background subtraction to remove noise and unnecessary
information. In order to have accurate results for our segmentation, a simple transformation
must be performed on the image to change the image size into standard format. To do so,
the size of each image will be compared with standard size categories. If the image size is
not compatible with any size category, some cropping or padding techniques will be applied
to the image.

3.2.2 Image Classification

For image classification we have used Convolutional Neural Network. In this algorithm, we
are providing an image to convolutional as an input and we get a convolutional feature map
as output.

ˆ CNN

Food Calorie Estimation 8

 CHAPTER 3. EXISTING SYSTEM
3.2. PROPOSED SYSTEM AND PROPOSED SYSTEM

A Convolutional Neural Network (ConvNet/CNN) is a Deep Learning algorithm which

can take in an input image, assign importance (learnable weights and biases) to various
aspects/objects in the image and be able to differentiate one from the other.
The first step in our approach is to generate a pre-trained model file with the help of
CNN network. This is performed by initially capturing a set of images of one particular class
(e.g. 50 images of apple class) and then labeling them with object name-set (object being
apple). These images will be considered as the set of relevant(positive) images and are used
to train the system.

Figure 3.1: CNN Architecture

CNN is built with following major steps:

1. Convolutional Layer: This layer is the first layer that is used to extract the various
features from the input images. In this layer, the mathematical operation of convolution is
performed between the input image and a filter of a particular size MxM. By sliding the
filter over the input image, the dot product is taken between the filter and the parts of the
input image with respect to the size of the filter (MxM). The output is termed as the Feature
map which gives us information about the image such as the corners and edges. Later, this
feature map is fed to other layers.

2. Pooling: In most cases, a Convolutional Layer is followed by a Pooling Layer. The

primary aim of this layer is to decrease the size of the convolved feature map to reduce the
computational costs. The Pooling Layer usually serves as a bridge between the Convolutional
Layer and the FC Layer
Depending upon the method used, there are several types of Pooling operations.
ˆ Max Pooling, the largest element is taken from the feature map.

ˆ Average Pooling, calculates the average of the elements in a predefined size Image
section.
ˆ Sum Pooling, the total sum of the elements in the predefined section is computed in
this.

Food Calorie Estimation 9

 CHAPTER 3. EXISTING SYSTEM
3.2. PROPOSED SYSTEM AND PROPOSED SYSTEM

3. Full connection: After receiving inputs from the previous neurons, the final layer
is the fully connected one. It takes the output of the previous layers, “flattens” them and
turns them into a single vector that can be an input for the next stage.
The FC layer helps to map the representation between the input and the output.

4. Dropout: It is a technique used to prevent a model from overfitting where randomly

selected neurons are ignored during training.

5. Activation Functions: There are several commonly used activation functions such
as the ReLU, Softmax, tanH and the Sigmoid functions.Here we are using ReLU and softmax
functions.

ˆ ReLU: ReLu refers to the Rectifier Unit, the most commonly deployed activation
function for the outputs of the CNN neurons.The main advantage of using the ReLU
function over other activation functions is that it does not activate all the neurons at
the same time.The purpose of applying the rectifier function is to increase the non-
linearity in our images.

ˆ Softmax: Softmax is implemented through a neural network layer just before the
output layer. The Softmax layer must have the same number of nodes as the output
layer. The softmax function is a function that turns a vector of K real values into a
vector of K real values that sum to 1. The input values can be positive, negative, zero,
or greater than one, but the softmax transforms them into values between 0 and 1, so
that they can be interpreted as probabilities.

3.2.3 Image Segmentation and Feature Extraction

A mixture of methods including canny edge detection, watershed segmentation, morphologi-
cal operators and Otsu’s method will be used to segment the food item to obtain the contour
of the food and the contour of the thumb. We use the thumb finger for calibration purposes.
The thumb is placed next to the dish while clicking the photo and this thumb gives us the
estimate of the real-life size of the food item and helps estimate volume accurately.
We have 3 factors from image segmentation
1. Foods pixel area
2. Skin pixel area
3. Actual skin area ( skin multiplier )

Food Calorie Estimation 10

 CHAPTER 3. EXISTING SYSTEM
3.2. PROPOSED SYSTEM AND PROPOSED SYSTEM

3.2.4 Volume Estimation

Finger is used as a reference object in this method. A food image is captured, where the
user’s finger is placed next to the food object on the plate. Since the dimension of the user’s
finger is known, the system can compute the corresponding dimension of the food object,
when captured from top view and side view. These dimensions are then used in calculating
the volume of the food object, which is further mapped to the corresponding calorie value
of the food object.

We know that, our thumb size is approximately 5 * 2.3cm is a skin multiplier

5.0
pix to cm multilpier = P ixelHeight

ˆ Calculation of food area:

F oodpixelArea∗ActualSkinArea
EstimatedArea = SkinP ixelArea

We have four type of shape of foods

1. Sphere - like apple, orange, tomato, onion
2. Cylinder – like banana, cucumber, carrot , egg
3. Columnar -like bread, cheese
4. Irregular -like noodles, sauce

ˆ Volume estimation of sphere:

q
EstimatedF oodArea
EstimatedRadius = Π

4
EstimatedV olume = 3
∗ Π ∗ Radius

ˆ Volume Estimation For Cylinder:

EstimatedHeight = P ixelHieght ∗ pixeltocmmumtiplier

EstimatedF oodArea
EstimatedRadius = (2∗EstimatedHeight)

EstimatedV olume = Π ∗ EstimatedHeight ∗ Radius2

ˆ Volume Estimation For Columnar object:

EstimatedHeight = P ixelHeight ∗ pixtocmmultiplier

Food Calorie Estimation 11

 CHAPTER 3. EXISTING SYSTEM
3.2. PROPOSED SYSTEM AND PROPOSED SYSTEM
EstimatedF oodArea
EstimatedRadius = (3.0∗EstimatedHeight)

4
EstimatedV olume = 3
∗ Π ∗ Radius ∗ Height

ˆ Volume Estimation For Irregular-sized object:

EstimatedV olume = EstimatedF oodArea ∗ 0.5(Assuming width as 0.5)

3.2.5 Weight and Calorie Estimation

We are going to provide food density (g/cm3 ) and food calories (cal/g) to the system. Using
this information we can estimate weight and calories intake in given food.

Food Calorie Estimation 12

Chapter 4

Architectural Overview

Figure 4.1: System Architecture

13
 CHAPTER 4. ARCHITECTURAL OVERVIEW

Figure 4.2: System Flow

Step 1: User will upload the image of food item onto the webpage
Users can upload the image of the food product which is already clicked by them and
that image will act as input to the system. For all images we have considered a white plate
to ignore the background of the images.

Step 2: Image Recognition will occur

System will recognize the given food item.CNN is used in food recognition and provides
higher performance than the traditional methods.

Step 3: Image Segmentation

System segment the image of the food item to obtain the contour of the food and the
contour of the thumb.

Step 4: Volume will get calculated and Calorie gets Detected

Using a segmented image, the system first calculates the volume of the food item and
then mass of the item.Finally the system will calculate the calorie of the item using relation
between the mass and calorie of a food item.

Step 5: Experimental Results will be displayed to the user

Recognized food items and the predicted calories will be displayed to the user as an
output.

Food Calorie Estimation 14

Chapter 5

Technology Used

5.1 Hardware and Software

Requirements
Hardware Requirements

ˆ Processor: Intel(R) core (TM) i5-1035G1 CPU 1.19 GHz

ˆ Memory (RAM): 8 GB RAM

ˆ Hard Drive: 64 GB

ˆ Camera: Canon SD1400, iPhone 4, Samsung S4

Software Requirements

1. Anaconda Navigator:

ˆ Anaconda Navigator is a desktop graphical user interface (GUI) included in Anaconda

distribution.

ˆ Navigator allows us to launch common Python programs and easily manage conda
packages, environments, and channels without using command-line commands.

ˆ We can use it to find the packages you want, install them in an environment, run the
packages, and update them – all inside Navigator.

For our project we are using, Jupyter Notebook and PyCharm.

15
5.1. HARDWARE AND SOFTWARE
REQUIREMENTS CHAPTER 5. TECHNOLOGY USED

2. Jupyter Notebook

Jupyter is a free, open-source, interactive web tool known as a computational notebook,

which researchers can use to combine software code, computational output, explanatory text
and multimedia resources in a single document.
The Jupyter Notebook application allows us to create and edit documents that display
the input and output of a Python or R language script. Once saved, we can share these files
with others.
Features of Jupyter Notebook:

Data visualizations:

ˆ Jupyter Notebook helps us for data visualizations, but also shares and allows us to do
interactive changes to the shared code and data set.

Code sharing:

ˆ With a Jupyter Notebook, we can view code, execute it, and display the results directly
in our web browser.

Live interactions with code:

ˆ Jupyter Notebook code isn’t static; it can be edited and re-run incrementally in real
time, with feedback provided directly in the browser.

ˆ Notebooks can also embed user controls (e.g., sliders or text input fields) that can be
used as input sources for code.

Documenting code samples:

ˆ If we have a piece of code and we want to explain line-by-line how it works, with live
feedback all along the way, we could embed it in a Jupyter Notebook.

ˆ The code will remain fully functional and we can add interactivity along with the
explanation, showing and telling at the same time.

Food Calorie Estimation 16

5.1. HARDWARE AND SOFTWARE
REQUIREMENTS CHAPTER 5. TECHNOLOGY USED

3. Pycharm

PyCharm is the most popular IDE used for the Python scripting language. PyCharm
is a hybrid-platform developed by JetBrains as an IDE for Python. It is commonly used
for Python application development. Some of the unicorn organizations such as Twitter,
Facebook, Amazon, and Pinterest use PyCharm as their Python IDE!

Features of PyCharm:

Intelligent Code Editor:

ˆ It helps us write high-quality codes! It helps identify errors easily.

ˆ It consists of color schemes for keywords, classes, and functions. This helps increase
the readability and understanding of the code.

ˆ It provides the autocomplete feature and instructions for the completion of the code.

Code Navigation:

ˆ It helps developers in editing and enhancing the code with less effort and time.

ˆ With code navigation, a developer can easily navigate to a function, class, or file.

ˆ A programmer can locate an element, a symbol, or a variable in the source code within
no time.

Assistance for Many Other Web Technologies:

ˆ It helps developers create web applications in Python.

ˆ It supports popular web technologies such as HTML, CSS, and JavaScript. It also
supports AngularJS and NodeJS for developing web applications.

ˆ Developers have the choice of live editing with this IDE. At the same time, they can
preview the created/updated web page.

Support for Popular Python Web Frameworks:

ˆ PyCharm supports web frameworks such as Django. It helps in debugging the codes
of Django.

ˆ It provides the autocomplete feature and suggestions for the parameters of Django.

Food Calorie Estimation 17

5.1. HARDWARE AND SOFTWARE
REQUIREMENTS CHAPTER 5. TECHNOLOGY USED

ˆ It also assists web2py and Pyramid, the other popular web frameworks.
Assistance for Python Scientific Libraries:
ˆ PyCharm supports Python’s scientific libraries such as Matplotlib, NumPy, and Ana-
conda.
ˆ These scientific libraries help in building projects of Data Science and Machine Learn-
ing.
ˆ It consists of interactive graphs that help developers understand data.
ˆ It is capable of integrating with various tools such as IPython, Django, and Pytest.
This integration helps innovate unique solutions.
4. VS Code

Visual Studio Code is a powerful source code editor which is available for Windows, ma-
cOS and Linux. Instead of a project system, it allows users to open one or more directories,
which can then be saved in workspaces for future reuse.

Features of VS Code:

Language support:
ˆ Visual Studio Code includes basic built-in support for most common programming
languages such as C++, Java, Python, PHP, Go,etc.
ˆ It provides syntax highlighting, bracket matching, code folding, and configurable snip-
pets.
ˆ VS Code also ships with IntelliSense for JavaScript, TypeScript, JSON, CSS, and
HTML, as well as debugging support for Node.js.
Data collection:
ˆ Visual Studio Code collects usage data and sends it to Microsoft, although this can be
disabled.
ˆ It is in the open-source nature of the application, due to which the telemetry code is
accessible to the public.
Version control:
ˆ Source control is a built-in feature of Visual Studio Code.
ˆ It has a dedicated tab inside of the menu bar where you can access version control
settings and view changes made to the current project.

Food Calorie Estimation 18

Chapter 6

Implementation Details

Phase I : Dataset Collection

Phase II : Environment Setup

ˆ Setting up of model

Phase III : Training Dataset

Phase IV : Estimation Process

ˆ Image segmentation

ˆ Calorie estimation

Phase V : Testing model

Phase VI : Deployment of Model

Phase I : Dataset Collection

The produced data set comprising 13 different categories of food and fruits. These food
and fruit images are divided into training and testing sets, where around 50 of the fruit
images from each group are used to train the system and the remaining images serve as the
testing set.
We took into consideration important factors that affect the accuracy of our results.
Specifically, we used a variety of the following components: Camera, Lighting, Shooting
Angle, White Plate, Thumb, Single and Mixed Food. Each component will be described in
details in the following subsections.

Camera

We used three different cameras for our experiments, consisting of Canon SD1400 (14.1-
megapixel resolution, 2.7-inch Pure Color System LCD, 28mm wide-angle lens; 4x optical
zoom and Optical Image Stabilizer) iPhone 4(5 megapixel resolution, LED flash, VGA-
quality photos and video at up to 30 frames per second with the front camera), and Samsung
S4 cameras (13 megapixel resolution, 41, autofocus, LED flash, Dual shot).

19
 CHAPTER 6. IMPLEMENTATION DETAILS

Lighting

Lighting and illumination is one of the important parameters which affect the system
outcome because illumination directly affects the image segmentation algorithm, which in
turn affects the rest of the algorithms. To take this into account, we put the same plate in
three different locations with different illuminations (sunlight) and we took pictures.

Shooting Angle

Another effective parameter is the angle of photography; we have chosen three different
angles which are approximately 30, 90, and 150 degrees from the plate of food for all pictures.
This means that for each plate in 3 different lighting locations we have also taken 3 pictures
from different angles.

White Plate

For all images we have considered a white plate to ignore the background of the images.
By using white plate, food segmentation and food recognition will be easier to perform.

Thumb

The thumb of the user and its placement on the plate are also shown in Figure 2. There
is a one-time calibration process for the thumb, which is used as a size reference to measure
the real-life size of food portions in the picture. Compared to the calibration methods of
similar systems, using the thumb is more flexible, controllable, and reliable.

Food Item Types

The dataset contains images taken with different cameras, illuminations, and angles.
Having a wide variety of food and fruits gives a better and more reliable dataset in order
to increase the accuracy of calorie food measurement systems. In the dataset, the images
are divided into 6 categories considering the capturing device, background, and lighting
condition. For example, images in category 1 are captured with a Samsung camera, within
a light environment with a white background, and from different shooting angles.

Table 6.1: Different Camera Brands and Light Environment

Category Camera Lighting

1 Samsung-S4 Light Environment
2 Samsung-S4 Dark Environment
3 IOS-4 Light Environment
4 IOS-4 Dark Environment
5 CanonSD1400 Light Environment
6 CanonSD1400 Dark Environment

Food Calorie Estimation 20

 CHAPTER 6. IMPLEMENTATION DETAILS

Table 6.2: Food Nutritional Information

Food Item Density Calorie Shape

(g/cm3 ) (Kcal/g)
Apple 0.609 52 Sphere
Banana 0.94 89 Cylindrical
Beans 0.721 31 Irregular
Bread 0.2 67 Columnar
Carrot 0.641 41 Cylindrical
Cheese 0.44 60 Columnar
Cucumber 0.641 16 Cylindrical
Egg 1.104 54 Cylindrical
Noodles 0.52 137 Irregular
Onion 0.513 40 Sphere
Orange 0.482 47 Sphere
Sauce 1.1 37 Irregular
Tomato 0.481 18 Sphere

Figure 6.1: Dataset Used

Phase II -Environment Setup

Setting up of model

Food Recognition deals with recognition of food items when given an image.For this
problem we used Convolutional Neural Network (CNN).
ˆ Firstly we imported all the libraries such as Tensorflow and tflearn which were needed
to implement CNN .

Food Calorie Estimation 21

 CHAPTER 6. IMPLEMENTATION DETAILS

ˆ Then we have used 5 convolution layers such as convolutional layers, pooling layers
followed by fully-connected layers.Also used dropout layer and activation functions
such as ReLu and Softmax.

ˆ The CNN takes an input image and applies convolutional and then sub-sampling layers
. After two such computations, the data is fed into the fully connected neural network,
where it performs the classification task.

ˆ Also used Adam optimizer and categorical cross entropy loss function with learning
rate 0.0001 to calculate and minimize loss as well as optimize model accuracy.

Phase III - Training dataset

Fine tuning the model on our dataset took about 1 hours on a single Windows 10 In-
tel(R) core(TM) i5-1035G1 CPU with 8GB of memory.For this training we used more than
100 images of each food with 400*400 image size. All this work done in Python 3.9.0 with
Anaconda Distribution 4.8.3. In this section we will be discussing the results about by using
CNN algorithm on our image dataset. For classification purposes we used 40 epochs. After
40 epochs we got an accuracy of about 97.11% in image classification and loss about 0.39% .

Food Calorie Estimation 22

 CHAPTER 6. IMPLEMENTATION DETAILS

Food Calorie Estimation 23

 CHAPTER 6. IMPLEMENTATION DETAILS

Figure 6.2: Training Dataset

Phase IV : Estimation Process

Till now we have only been able to find out the calories of spherical food items including
apple, onion, tomato and orange.

ˆ Image Segmentation

As we are using canny edge detection, watershed segmentation, morphological operators and
Otsu’s method to segment the food item, we are successfully able to recognize contour of
food and contour of thumb.

Following results are obtained after segmenting the food image:

Food Calorie Estimation 24

 CHAPTER 6. IMPLEMENTATION DETAILS

Food Calorie Estimation 25

 CHAPTER 6. IMPLEMENTATION DETAILS

Food Calorie Estimation 26

 CHAPTER 6. IMPLEMENTATION DETAILS

Food Calorie Estimation 27

 CHAPTER 6. IMPLEMENTATION DETAILS

ˆ Calorie Estimation

After segmentation is done. We are able to find calorie correct calories of spherical food
items by volume and weight estimation.

Phase V : Testing the model

ˆ To test the dataset we have run one demo to ensure the correct food is recognized.

ˆ Considered only spherical food items like apple, orange, onion, etc. and provided the
relevant formulas according to shape.

ˆ We know food density (g/cm3 ) and food calories (kcal/g), using this information we
have estimated weight and calories.

ˆ We got the following result after running the demo model.

Food Calorie Estimation 28

 CHAPTER 6. IMPLEMENTATION DETAILS

Figure 6.3: Output of Demo1

Figure 6.4: Output of Demo2

Food Calorie Estimation 29

Chapter 7

Conclusion

7.1 Conclusion
We have verified that our proposed measurement system is useful, convenient and very user-
friendly, which could encourage users to use the system at each and every meal time.We built
our own dataset comprising more than 1500 food images with careful attention of shooting
angle and illumination variations. In the above project, we have calculated the calories of
spherical shape food items using deep learning and image processing. For that we have
implemented three modules.First, we detect the object using CNN.Our results shown that
the accuracy of our model for food recognition is 97.11%. After that image segmentation is
performed and based on volume estimation, weight and calories are calculated successfully
for spherical food items.

7.2 Future Work

ˆ As a future work, the model can be extended to include more categories and more
number of images per categories and can estimate the calorie from all types of food.

ˆ In the next phase, we are going to work on volume and calorie estimation process for
all shapes of food.

ˆ We are planning to deploy our model into the web application for user convenience.

ˆ We plan to enhance our system by notifying the user about how much the user should
intake in calories.

7.3 Applications
ˆ Helps people to deal with problems like obesity, hypertension, high cholesterol, dia-
betes, heart attack,etc which may further lead to medical consultancy.

ˆ Helps people to focus on weight loss resulting in a healthy lifestyle.

30
7.3. APPLICATIONS CHAPTER 7. CONCLUSION

ˆ Helps fitness freaks to keep a track of their daily calorie consumption.

ˆ Motivating people to adapt a healthy lifestyle by tracking calorie intake.

Food Calorie Estimation 31

References

[1] Jessie R Balbin, Leonardo D Valiente, Kim Martin P Monsale, Emmanuel D Olorvida,
Gerard Glenn V Salazar, and Lyzza Marie L Soto. Determination of calorie content
in different type of foods using image processing. In 2019 IEEE 11th International
Conference on Humanoid, Nanotechnology, Information Technology, Communication and
Control, Environment, and Management (HNICEM), pages 1–6. IEEE.
[2] Yuri Y Boykov and M-P Jolly. Interactive graph cuts for optimal boundary & region seg-
mentation of objects in nd images. In Proceedings eighth IEEE international conference
on computer vision. ICCV 2001, volume 1, pages 105–112. IEEE, 2001.
[3] Meng-Lin Chiang, Chia-An Wu, Jian-Kai Feng, Chiung-Yao Fang, and Sei-Wang Chen.
Food calorie and nutrition analysis system based on mask r-cnn. In 2019 IEEE 5th
International Conference on Computer and Communications (ICCC), pages 1721–1728.
IEEE, 2019.
[4] Manal Chokr and Shady Elbassuoni. Calories prediction from food images. In Twenty-
Ninth IAAI Conference, 2017.
[5] R Kohila and R Meenakumari. Predicting calorific value for mixed food using image
processing. In 2017 International Conference on Innovations in Information, Embedded
and Communication Systems (ICIIECS), pages 1–4. IEEE, 2017.
[6] Yanchao Liang and Jianhua Li. Deep learning-based food calorie estimation method in
dietary assessment. arXiv preprint arXiv:1706.04062, 2017.
[7] Parisa Pouladzadeh, Shervin Shirmohammadi, and Rana Al-Maghrabi. Measuring calorie
and nutrition from food image. IEEE Transactions on Instrumentation and Measurement,
63(8):1947–1956, 2014.
[8] Manpreetkour Basantsingh Sardar and Dr Sayyad D Ajij. Fruit recognition and its calorie
measurement: An image processing approach. Journal of Emerging Trends in Computing
and Information Sciences, 2016.
[9] Reza Dea Yogaswara, Eko Mulyanto Yuniarno, and Adhi Dharma Wibawa. Instance-
aware semantic segmentation for food calorie estimation using mask r-cnn. In 2019
International Seminar on Intelligent Technology and Its Applications (ISITIA), pages
416–421. IEEE, 2019.

[7] [8] [5] [6] [1] [9] [3] [2] [4]

32