Assoc Prof Nicholas Vun

Associate Chair (Academic)

School of Computer Science
And Engineering

TinyML - ‘Hello World’ of TinyML

Main Reference

Chapter 4 of
TinyML Book

https://tinymlbook.files.wordpress.com/2020/01/tflite_micro_preview.pdf 1
“Hello World”
“Hello World”
• a basic program to confirm the correct syntax and proper operation of a
program.

We will learn the basic steps involved in training a neural network

• to learn about the trigonometry sine function

but with the following dataset

2
The goal
To train a neural network model that can take a value x and predict its sine value y
• without using (and not knowing) the sin() function

That is
• the model will predict the (approximate) result based on the labelled data we
feed during the training

This is a Regression ML
• as the output is a continuous value and the model will try to find the best-fit line
3
Aside: TensorFlow and Keras
TensorFlow is an open source project (originally developed by Google) that
provides a set of tools for ML that can perform the following functions
• Building a model
• Training the model
• Evaluating the performance
• Deploying (on conventional computer)

Keras is a high-level API that provide a Python interface to use TensorFlow

• make it easy to build and train the deep learning network
• without getting into the detail of TensorFlow

4
Workflow of Training and Building a Model
The following are typical steps involved in developing a model based on
machine learning (deep learning)

1. Decide on a goal
2. Select, collect and label a dataset
3. Design a model architecture
4. Train the model using the dataset
5. Convert the model
6. Run the inference
7. Evaluate and troubleshoot

5
Building the Model – ‘Collecting’ the dataset
We will need to collect labelled data the relate the expected sine output for an
input.

How do we collect the data needed?

• we will generate the data of a sine function using the built-in library available in
Jupyter Notebook

To make the data closer to be more realistic

• we will add random noise to the input

6
Generating the dataset
Run the Jupyter Notebook, select ‘New’ and ‘Python 3’
• which will open a ‘cell’

Then enter the program instruction into the cell and execute it

7
Generating the dataset
Key in the following ‘import’ directive, which is to load the various libraries we need
to use in the program

We also give each library an

alias that is easier to refer to
in our program

Press the ‘Run’ button after

entering each line of
instruction
• which will immediately
execute the instruction in
the cell

8
Aside: Entering Instruction
You can also key in multiple lines of instructions in a cell before running them

But at this stage

• it is probably better to enter one instruction per cell, run it to check that each is
correctly executed before entering the next line of instruction

9
Generating the dataset
Next, we are going to ‘collect’ 1000 values of a dataset that corresponds (but not
exactly) to the sine function

We will do this here by using the following Python code to generate the dataset

seed() methods - to generate a random seed to be used to generate random

number later 10
Generating the dataset

The above is is to generate the x values to be used as input.

We can check the values by typing the name of the variable

11
Generating the dataset
Next we will shuffle the x values to introduce more randomness
• Deep learning’s effectiveness depends on data being fed are in random order

12
Generating the dataset
Now we generate the ‘expected output’ of the x_values
• by using the sin() function (provided in NumPy)

We can visually confirm that the dataset is as expected (using the matplotlib
library) as follows:

13
Generating the dataset
Next we will add noise to the output y_values
• to simulate how real-world data is like typically
• this is the dataset we will use to train our deep learning model.

Let’s call these y_values

the observed y_values

14
Splitting the dataset
To evaluate the performance during and after training
• we will split the data set into three parts: training, validation and test
• in the ratios: 60%, 20%,20%

15
Verifying the dataset
We can visually check that the 3 sets of data are distributed randomly across the
whole cycle.

16
Workflow of Training and Building a Model
The following are typical steps involved in developing a model based on
machine learning (deep learning)

1. Decide on a goal
2. Select, collect and label a dataset
3. Design a model architecture
4. Train the model using the dataset
5. Convert the model
6. Run the inference
7. Evaluate and troubleshoot

17
Design of the model
With the dataset available, we will now create a model based on neuron network.

The model will be fed with the x_value

• which predicts a y_value*
• then compare predicted y_value* against the Observed y_value (i.e. the one
randomized)

We will start with using a simple neural network to attempt learning the pattern
between the input and output and make prediction
• we will define the model using Keras, the high-level API for Tensorflow
18
 Creating the model

The model is based on a Sequential model (which is called model_1 here).

It will consist of two layers (excluding the input node which is not considered as neuron).

First (hidden) layer will be of Dense type, with 16 neurons, with 1 input (i.e. a scalar)

• Dense means that each input will be fed into all the 16 neurons
• known as a fully connected layer
• Each neuron will be activated based on its input, weight and bias
19
and pass to the activation function: activation_function((input*weight)+bias)
Aside: Activation Functions
Step in using Activation function:
1. Multiplied input with the Weight
2. Then add the Bias
3. Apply the result to the function

There are several commonly used activation functions for model training

ReLU SoftPlus SigMoid

20
 Creating the model (cont.)
2nd (output) layer consist of 1 neuron (only 1 output possible) which output the
value
• It will have 16 inputs (with 16 weights) connected from the 1st layer
• No activation required

• Output = sum((inputs * weights)) + bias

The model architecture

• values of the weights and biases
are to be learned during Training.

21
Creating the model
Next we call the function compile() which specifies
• the algorithm (optimizer) to be used to adjust the model during training
• need experimentations in practice to find the most suitable one
• e.g. root mean squared propagation
• the loss function to calculate the loss
• how far the predicted value is from the expected value (the data we give)
• e.g. mean squared error which is usually use for linear regression problems
• additional function to judge the performance
• e.g. mean absolute error

22
Print a summary of the Model Create

32 connections with 32 weights

17 Biases with 17 neurons
23
Workflow of Training and Building a Model
The following are typical steps involved in developing a model based on
machine learning (deep learning)

1. Decide on a goal
2. Select, collect and label a dataset
3. Design a model architecture
4. Train the model using the dataset
5. Convert the model
6. Run the inference
7. Evaluate and troubleshoot

24
Training of model
To train the model, use the fit() method (i.e. a function tied to an object in Python)

• specifies the dataset to use

• Train and Validate data generated earlier on
• the epochs (1000 time, each time training with 600 data of different orders)
• the batch size
• the number of datapoint to train before adjusting the parameters (Weights
and Biases) during each epoch
• e.g. after every 16 training data (known as mini batch size)
• big batch size – less update, faster to train, but lead to overfitting
• small batch size – more update, slower to train

25
Training of model – Training Metrics

loss = mean squared error of training data

mae = mean absolute error of training data
val_loss = loss of validation data
val_mae = mae of validation data
26
Training of model – Graphing the History of Training

27
Training of model – Graphing the History of Training

28
Training of model – Graphing the History of Training

Error = 0.3
• 15% (output range of ±1)

29
Aside: Model Performance

30
Model Prediction vs Expect values (Test data)

31
Workflow of Training and Building a Model
The following are typical steps involved in developing a model based on
machine learning (deep learning)

1. Decide on a goal
2. Select, collect and label a dataset
3. Design a model architecture
4. Train the model using the dataset
5. Convert the model
6. Run the inference
7. Evaluate and troubleshoot

32
Improving our Model
Increase model to 3 layers

33
Improving our Model

34
Improving our Model

35
Improved Model Performance Metrics

36
Improved Model Performance Metrics

37
Improved Model Performance Metrics

Error = 0.09
• 4.5% (output
range of ±1)

38
Model Prediction vs Expect values using Test data

39
Workflow of Training and Building a Model
The following are typical steps involved in developing a model based on
machine learning (deep learning)

1. Decide on a goal
2. Select, collect and label a dataset
3. Design a model architecture
4. Train the model using the dataset
5. Convert the model
6. Run the inference
7. Evaluate and troubleshoot

40
Converting Model to TensorFlow Lite
TensorFlow Lite
• a set of tools that enables on-device machine learning
• to deploy models on mobile, embedded, and IoT devices.

TF Lite converts the model to a size-efficient format for memory constrained

devices
• ‘Flatbuffer’ format

Can further apply optimizations to reduce size and increase speed

• with trade-off in accuracy
• E.g. Quantization – replace 32-bit floating point with 8-bit integer
• smaller size (4 bytes to 1 byte)
• faster execution (integers)

TF Lite models
• executed by using the TensorFlow Lite Interpreter 41
Converting Model to TensorFlow Lite
Converting the model

With quantization

42
 Converting Model to TensorFlow Lite with quantization
With quantization

43
Running the TensorFlow Lite Interpreter
Steps involved:
1. Instantiate an Interpreter
2. Call methods (i.e. function) to allocate memory for the model
3. Write the input to the input tensor
4. Invoke the model
5. Read the output from the output tensor

44
Running the TensorFlow Lite Interpreter (cont.)

45
 Running the TensorFlow Lite Interpreter (cont.)
Check the Lite model performance

46
 TensorFlow Lite Model
Compare model size with and without quantization

??

47
Lab Session – Wed 18/1/2023

48