MathDance Workshop
Introduction to AI (ML/DL) with
Teachable Machine & Scratch
for the Roeper School
Jan 16, 2025
CJ Chung, PhD
Full Professor of Computer Science
Founder of Robofest & Director of the CS AI Robotics (CAR) lab
Lawrence Technological University, Southfield, Michigan, USA
[Link]
CJ Chung License CC BY 4.0 1
CJ Chung, PhD
Full Professor of Computer Science
Founder of Robofest and Director of the CAR (CS AI Robotics) lab
Lawrence Technological University, Southfield, Michigan, USA
[Link]
CJ Chung License CC BY 4.0 2
Pre Survey
[Link]
WqfAnMh0w7kfzvgQ/viewform?usp=sharing
CJ Chung License CC BY 4.0 3
Goals of MathDance Workshops
1. Learn basic concept of ML/DL
2. Learn/Sharpening Scratch Programming
Skills
3. Learn mathematical functions
4. Integrate Math, AI, and Scratch to
develop a MathDance Game App
5. Advance Quality Education, one of the
UN’s Sustainable Development Goals
CJ Chung License CC BY 4.0 4
This presentation file in PDF is available at
[Link]
CJ Chung License CC BY 4.0 5
Workshop Assistant
Devson Butani
Robotics Research Lab Manager
CJ Chung License CC BY 4.0 6
Agenda
(1) Intro to Machine Learning (ML)
(2) Introduction to Teachable Machine (TM)
(3) Introduction to Scratch Coding
(4) Hand MathDance Game App Development
CJ Chung License CC BY 4.0 7
What is Machine Learning (ML) - A subfield of AI
AI mimics the intelligence of humans
ML?
makes machines (models*) learn
from example-data**
DL?
makes machines learn from example
data using brain-inspired algorithms.
Today, we will use only labeled sample data using “supervised
learning method”.
(*) The model learns the mapping between input and output
(**) labeled, unlabeled, or past experience.
CJ Chung License CC BY 4.0 8
…
label Cat Cat Cat Cat
CJ Chung License CC BY 4.0 9
…
label Dog Dog Dog Dog
CJ Chung License CC BY 4.0 10
What’s the label of this image?
CJ Chung License CC BY 4.0 11
Example of ML
Unseen
image
Cat …
DL
Dog …
Dog
Class / Label
CJ Chung License CC BY 4.0 12
Example of ML: What is wrong?
Unseen
image
Dog …
DL
Cat …
Cat
Miss labeled
CJ Chung License CC BY 4.0 13
Example of ML: What is wrong??
Cat …
DL
Dog …
CJ Chung License CC BY 4.0 14
Normal Chest X-rays
CJ Chung License CC BY 4.0 15
COVID 19 Chest X-Rays
CJ Chung License CC BY 4.0 16
Chest X-Rays for Pneumonia Viral Case
CJ Chung License CC BY 4.0 17
?
CJ Chung License CC BY 4.0 18
Unseen
image
Normal
DL
Covid19
…
Normal
Pneumonia
Viral
…
CJ Chung License CC BY 4.0 19
DL
Inference
ML4K mathDance1
DEMO
CJ Chung License CC BY 4.0 20
MathDance1
Day 1
Project
[Link] 21
CJ Chung License CC BY 4.0
Agenda
(1) Intro to Machine Learning (ML)
(2) Introduction to Teachable Machine (TM)
(3) Introduction to Scratch Coding
(4) Hand MathDance Game App Development
CJ Chung License CC BY 4.0 22
𝑦𝑦 = 𝑥𝑥 𝑦𝑦 = −𝑥𝑥
Roll up your sleeves!
CJ Chung License CC BY 4.0 23
𝑦𝑦 = 0 𝑦𝑦 = 10𝑥𝑥
Roll up your sleeves!
CJ Chung License CC BY 4.0 24
5 classes (one-hand gestures only)
𝑦𝑦 = 0 Scratch app
…
New
image
𝑦𝑦 = 𝑥𝑥
…
𝑦𝑦 = −𝑥𝑥 train Trained
…
DL Model
𝑦𝑦 = 10𝑥𝑥
… 𝑦𝑦 = 10𝑥𝑥
𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛 50+ samples
CJ Chung License CC BY 4.0 25
Tools to use today
Teachable Machine (TM) - using Pretrained MobileNet
[Link]
Machine Learning for Kids (ML4K)
[Link]
Please bookmark them!
CJ Chung License CC BY 4.0 26
Let’s first train a model for 4 hand poses
(1) Go to [Link]
CJ Chung License CC BY 4.0 27
(2) Collect images for 5
classes
Use “minus x”, not “-x” due to
Scratch Speech function.
(3) Train
CJ Chung License CC BY 4.0 28
Teachable Machine (TM) is using MobileNet
A simple but efficient and not very computationally intensive
convolutional neural networks (CNN)
TM leverages the pre-trained knowledge of MobileNet,
which has been trained on a large dataset like ImageNet,
allowing you to train custom classifications with relatively
few images
Specifically designed to be lightweight and run well on
mobile devices, making it a suitable choice for browser-
based applications like TM
While using the MobileNet backbone, TM allows you to add
your own custom classification last layer to recognize the
specific objects you want to identify
CJ Chung License CC BY 4.0 29
(4) Test thoroughly and Save as mathDance1
(Task1)
To save
[Link]
CJ Chung License CC BY 4.0 30
Saving the TM project
Rename as [Link] and save the file.
Remember the saved location
Release your Webcam and
Close your browser
CJ Chung License CC BY 4.0 31
Agenda
(1) Intro to Machine Learning (ML)
(2) Introduction to Teachable Machine (TM)
(3) Introduction to Scratch Coding
(4) Hand MathDance Game App Development
CJ Chung License CC BY 4.0 32
Why Scratch Coding? Great for youth for several reasons:
Easy of learning – Visual programming (VP), drag-and-
drop blocks, focus on concepts
Multimedia (visual, sound) feedback
Learn programming concepts
Improve problem-solving skills
Promote Creativity, Expression, and Imagination
Cross-Curricular benefits – integrates math, science,
art, robotics, AI, story telling, and physical activities
Q: Is VP such as Scratch for professional engineers too? YES! Simulink
CJ Chung License CC BY 4.0 33
How many “Hellos” do you
hear when you click the green
flag?
CJ Chung License CC BY 4.0 34
Let’s test the code using ML4K using your browser (Task2)
Go to: [Link]
Click on button
Click on Extensions button
Click on “Text to Speech” extension
CJ Chung License CC BY 4.0 35
Scratch Variables (Task3)
What do you hear?
CJ Chung License CC BY 4.0 36
ML4K Scratch Project Editor
Tool select tabs Project options View options
Coding Stage
Workspace
Sprite menu
Block Palette Backdrop menu
CJ Chung License CC BY 4.0 37
Six Scratch Block Shapes
Hat blocks are the C blocks loop the blocks
blocks that start every or check if a condition is
script. true.
Reporter blocks can hold
Stack blocks perform numbers and strings.
commands.
Boolean blocks are the Cap blocks are the blocks
conditions — they are that end scripts.
either true or false.
CJ Chung License CC BY 4.0 38
Scratch Block Categories
Motion blocks deal with the movement of sprites. They relate mainly to the
x and y position and direction of the sprite.
Looks blocks are related to the appearance of sprites and the stage.
Sound blocks control sound. (*)
Events blocks are related to various triggers in a project, or when one part
signals another to run. (*)
Control blocks run the basic flow of a project in the desired fashion. (*)
Sensing blocks associate with sprites and the stage detecting conditions. (*)
Operators blocks deal with many mathematical functions. (*)
Variables blocks are for storing and accessing data. (*)
My blocks are blocks that hold custom procedures for a selected sprite. (*)
Images blocks contain costume & backdrop image reporters (*)
(*) used in MathDance workshop
CJ Chung License CC BY 4.0 39
How to create a List?
Create a list named
“shoppingCart” that can have
multiple items
CJ Chung License CC BY 4.0 40
How to create “My Block” to define my “shopping”
will appear on workspace
CJ Chung License CC BY 4.0 41
To define My Block – “shopping”
CJ Chung License CC BY 4.0 42
To use My Block and List (Task4)
What will you hear?
CJ Chung License CC BY 4.0 43
repeat until <loop-Boolean-termination-condition>
(Task5)
How many hellos can
you hear?
Boolean block - either true or false
0 > 2? No (hello)
1 > 2? No (hello)
2 > 2? No (hello)
3 > 2? Yes. End the loop
CJ Chung License CC BY 4.0 44
Q: What is the color of variable block in Scratch?
CJ Chung License CC BY 4.0 45
Q: What is the color of operator block?
CJ Chung License CC BY 4.0 46
Q: What is the color of “My Blocks”?
CJ Chung License CC BY 4.0 47
3 min stretch break
CJ Chung License CC BY 4.0 48
Stretch
Break
with
MathDance
[Link]
CJ Chung License CC BY 4.0 49
Nested Repeat with
repeat until <loop-Boolean-termination-condition>
(Task6)
How many
wows can you
hear?
CJ Chung License CC BY 4.0 50
Review Q: How many Wows?
3 > 3 ? No (wow)
4 > 3 ? Yes. End of Repeat
CJ Chung License CC BY 4.0 51
“If” with logical “and” operator (Task 7)
Two
variables
CJ Chung License CC BY 4.0 52
“If” with logical “and” operator vs. Nested “if”s
CJ Chung License CC BY 4.0 53
Review Q: What will you hear?
CJ Chung License CC BY 4.0 54
Agenda
(1) Intro to Machine Learning (ML)
(2) Introduction to Teachable Machine (TM)
(3) Introduction to Scratch Coding
(4) Hand MathDance Game App Development
CJ Chung License CC BY 4.0 55
Steps to use your trained TM model in Scratch
(1) Go to [Link]
[Link]
(2) Train, Test and Collect
more images, if needed
(3) ReTrain
CJ Chung License CC BY 4.0 56
(4) Check all the images, Test thoroughly, and
re-save as [Link]
To save
CJ Chung License CC BY 4.0 57
(5) Export Model
CJ Chung License CC BY 4.0 58
(6) “Upload my model” (takes time)
(7) Copy the URL, paste it to Notepad (recommended)
(8) Close TM or Turn off “Input” on the TM (Scratch needs to access cam!)
CJ Chung License CC BY 4.0 59
( 9) Go to [Link]
(10) Click on (at the bottom)
(11) Paste the URL copied in (7) and click on “OPEN SCRATCH”
Then you will see
If the model is not updated in Scratch, try Scratch Project
removing the slash at the end of the URL…
Editor.
CJ Chung License CC BY 4.0 60
(12) When Scratch is open:
(12.1) First check
You may have
another one, if the
model has been
updated.
5 classes
(12.2) Import 2 extensions:
CJ Chung License CC BY 4.0 61
(13) Remove the “Cat” sprite
(14) Name the project file: mathDance1
(15) Create the following and test (Task8)
Add a comment block by right
clicking on the coding workspace
to specify the TM file and URL
(16) Save the code as: mathDance1.sb3
CJ Chung License CC BY 4.0 62
Summary: Steps to train a model and create a Scratch app
1. Go to: [Link] (and open saved TM project file)
2. Collect (more) samples for each class
3. Train Model
4. Test thoroughly. Good idea to re-save your TM project to your drive
5. Click on “Export Model” button
6. Click on “Upload my model” button (will take time)
7. Copy the sharable link (URL) and paste it to a NotePad with the .tm file name
8. Close TM or Turn off camera “Input” on the TM (Important!!!)
9. Launch ML4K pretrained page: [Link]
10. Click on “Open TensorFlow model” (at the bottom)
11. Paste the URL (See 7) from TM. Make sure to choose “image or pose”. Click on “OPEN SCRATCH” button
12. Check if the correct model is connected; Import extensions: “Video Sensing” and “Text to Speech”
13. Remove the “Cat” sprite (Important)
14. Name the project file
15. Create the code to check if the ML model is ready
16. SAVE the Scratch project file, for example, mathDance1.sb3
CJ Chung License CC BY 4.0 63
Test This Code (Task9)
CJ Chung License CC BY 4.0 64
Trouble Shoot – No Wecam Image on the Stage?
Teachable Machine might be holding your camera.
Method 1: Method 2:
1. Turn off the cam on Teachable 1. Turn off the cam on Teachable
Machine or close TM web browser Machine or close TM web browser.
2. Run 2. Save the current Scratch code onto a
sb3 file.
3. Close the web browser.
4. Relaunch a web browser.
5. Go to ML4K and click on get started.
6. Load the saved .sb3 file.
CJ Chung License CC BY 4.0 65
Day 1
Project
[Link] 66
CJ Chung License CC BY 4.0
DL
What do we
get from the
model when a
Inference webcam
image is sent?
ML4K
CJ Chung License CC BY 4.0 67
Day 1 Project (Task 10) Today
Variables List My Blocks
CJ Chung License CC BY 4.0 68
(Task10 a)
Next slide
CJ Chung License CC BY 4.0 69
(Task10 b)
Save your .sb3
file!
CJ Chung License CC BY 4.0 70
Ideas to improve this program?
CJ Chung License CC BY 4.0 71
Day 2
Project
[Link] CJ Chung License CC BY 4.0 72
Agenda Day 2
(1) Review: Intro to Machine Learning (ML)
(2) Review: Intro to Teachable Machine (TM)
(3) Intro to Scratch Coding II
(4) Hand MathDance Game App Development II, III
(5) More Math Functions and Animation
(6) TM Pose Models
(7) Wrap Up
CJ Chung License CC BY 4.0 73
Day 2 Video – UN’s AI for Good Program
[Link]
CJ Chung License CC BY 4.0 74
License Information
CC BY 4.0
The CC BY 4.0 license allows anyone to share, adapt, and use the licensed work for any
purpose, as long as they give appropriate credit to the original creator
[Link]
CJ Chung License CC BY 4.0 75
Post Survey
[Link]
JjmR0avpDucVc2p36vwao9sKdw/viewform?usp=sharing
CJ Chung License CC BY 4.0 76
Thank you! Post-survey link
will be sent to you
Questions?
CJ Chung License CC BY 4.0 77
