0% found this document useful (0 votes)
11 views79 pages

FPGA an 02039 1 2 Lattice SensAI Studio User Guide

The Lattice sensAI Studio User Guide provides comprehensive instructions for installing, configuring, and utilizing the sensAI Studio application. It covers prerequisites, deployment, project management, dataset handling, job scheduling, and user management, along with troubleshooting tips. The document emphasizes that all information is provided as-is and that users should independently verify product suitability.

Uploaded by

rejnop
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
11 views79 pages

FPGA an 02039 1 2 Lattice SensAI Studio User Guide

The Lattice sensAI Studio User Guide provides comprehensive instructions for installing, configuring, and utilizing the sensAI Studio application. It covers prerequisites, deployment, project management, dataset handling, job scheduling, and user management, along with troubleshooting tips. The document emphasizes that all information is provided as-is and that users should independently verify product suitability.

Uploaded by

rejnop
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 79

Lattice sensAI Studio User Guide

Application Note

FPGA-AN-02039-1.2

June 2023
Lattice sensAI Studio User Guide
Application Note

Disclaimers
Lattice makes no warranty, representation, or guarantee regarding the accuracy of information contained in this document or the suitability of its
products for any particular purpose. All information herein is provided AS IS, with all faults and associated risk the responsibility entirely of the Buyer.
Buyer shall not rely on any data and performance specifications or parameters provided herein. Products sold by Lattice have been subject to limited
testing and it is the Buyer's responsibility to independently determine the suitability of any products and to test and verify the same. No Lattice
products should be used in conjunction with mission- or safety-critical or any other application in which the failure of Lattice’s product could create a
situation where personal injury, death, severe property or environmental damage may occur. The information provided in this document is
proprietary to Lattice Semiconductor, and Lattice reserves the right to make any changes to the information in this document or to any products at
any time without notice.

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

2 FPGA-AN-02039-1.2
Lattice sensAI Studio User Guide
Application Note

Contents
Acronyms in This Document ................................................................................................................................................. 8
1. Introduction .................................................................................................................................................................. 9
1.1. Pre-requisites ...................................................................................................................................................... 9
1.1.1. Hardware Requirement .................................................................................................................................. 9
1.1.2. Software Requirements.................................................................................................................................. 9
1.2. Limitations or Known Issues.............................................................................................................................. 10
1.3. Stop and Remove Older Version of Webapp .................................................................................................... 10
2. sensAI Studio Installation Configuration .................................................................................................................... 11
2.1. Public Dataset Configuration ............................................................................................................................ 11
2.2. Server IP Configuration ..................................................................................................................................... 11
2.3. HTTPs Support Configuration ............................................................................................................................ 11
2.4. NFS Configuration ............................................................................................................................................. 13
2.5. Kubernetes Configuration ................................................................................................................................. 14
3. Deploying sensAI Studio ............................................................................................................................................. 15
3.1. Deployment ....................................................................................................................................................... 15
4. Getting Started ........................................................................................................................................................... 16
4.1. Logging in the Web Application ........................................................................................................................ 16
4.2. Logging Out of the Web Application ................................................................................................................. 18
4.3. Changing the Password ..................................................................................................................................... 18
5. Dashboard .................................................................................................................................................................. 20
6. Project ........................................................................................................................................................................ 21
6.1. Viewing Project Details ..................................................................................................................................... 21
6.2. Updating Project Details ................................................................................................................................... 22
6.3. Adding Project ................................................................................................................................................... 22
6.3.1. Model Zoo .................................................................................................................................................... 23
6.3.2. Local Model .................................................................................................................................................. 24
6.3.3. Other Repo ................................................................................................................................................... 28
6.4. Project Graph .................................................................................................................................................... 28
6.5. Download Project Zip ........................................................................................................................................ 29
7. Workspace .................................................................................................................................................................. 30
7.1. Adding Public Workspace .................................................................................................................................. 30
7.2. Adding Custom Workspace ............................................................................................................................... 31
8. Dataset........................................................................................................................................................................ 32
8.1. Viewing Dataset Details .................................................................................................................................... 32
8.2. Adding Dataset .................................................................................................................................................. 33
8.2.1. Public ............................................................................................................................................................ 33
8.2.2. URL ............................................................................................................................................................... 34
8.2.3. AWS .............................................................................................................................................................. 34
8.2.4. GCP ............................................................................................................................................................... 35
8.2.5. Uploading Dataset ........................................................................................................................................ 36
8.2.6. Stream Dataset ............................................................................................................................................. 37
8.2.6.1. Manual Capture ................................................................................................................................... 37
8.2.6.2. Continuous Capture ............................................................................................................................. 37
8.2.6.3. Video Stream........................................................................................................................................ 38
8.3. Editing Dataset .................................................................................................................................................. 38
8.4. Visualizing the Dataset ...................................................................................................................................... 40
8.4.1. Changing Existing Class................................................................................................................................. 41
8.4.2. Resizing Existing Bounding Box .................................................................................................................... 42
8.4.3. Drawing a New Bounding Box ...................................................................................................................... 43
8.4.4. Deleting a New Bounding Box ...................................................................................................................... 43
8.5. Auto-Annotation ............................................................................................................................................... 43
8.6. Converting Annotation ...................................................................................................................................... 45

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-AN-02039-1.2 3
Lattice sensAI Studio User Guide
Application Note

8.7. Downloading Dataset ........................................................................................................................................ 47


8.8. Pre-Processing the Dataset ............................................................................................................................... 47
8.8.1. Dataset Conversion Sample ..........................................................................................................................48
8.8.2. Dataset Auto-Annotation Sample.................................................................................................................48
8.9. Dataset Augmentation ...................................................................................................................................... 49
8.10. Dataset append and train/val/test split ............................................................................................................ 49
8.11. Dataset Merging ................................................................................................................................................ 50
9. Jobs ............................................................................................................................................................................. 51
9.1. Adding Training Job ........................................................................................................................................... 51
9.1.1. Logs ...............................................................................................................................................................52
9.1.2. Freeze ...........................................................................................................................................................53
9.1.3. TensorBoard .................................................................................................................................................54
9.2. Adding Neural Network Compiler Job ............................................................................................................... 55
9.3. Adding Download Job Artefacts ........................................................................................................................ 57
9.4. Download Complete Job ................................................................................................................................... 58
9.5. Modifying a Job ................................................................................................................................................. 58
9.6. Scheduling a Job ................................................................................................................................................ 59
9.7. Job Metrics ........................................................................................................................................................ 59
9.7.1. Adding Metrics .............................................................................................................................................59
9.8. Comparing Jobs ................................................................................................................................................. 60
9.9. Job Graphs ......................................................................................................................................................... 61
10. Code ............................................................................................................................................................................ 62
11. User Management ...................................................................................................................................................... 63
11.1. Creating a New User .......................................................................................................................................... 63
11.2. Removing a User ............................................................................................................................................... 64
11.3. Updating the User Details ................................................................................................................................. 65
12. Group Management ................................................................................................................................................... 66
12.1. Creating a Group ............................................................................................................................................... 66
12.2. Adding and Removing Group Members ............................................................................................................ 67
12.3. Deleting a Group ............................................................................................................................................... 68
13. Memory Management ................................................................................................................................................ 69
13.1. Configuring Memory Management ................................................................................................................... 69
13.2. Memory Management User Setting .................................................................................................................. 70
14. AWS NFS Setup Guide ................................................................................................................................................. 71
15. AWS Kubernetes (EKS) Cluster Creation ..................................................................................................................... 72
16. Troubleshooting .......................................................................................................................................................... 74
16.1. Database Connection Failed during Boot .......................................................................................................... 74
16.2. Dataset Validation Fails ..................................................................................................................................... 74
16.3. Not Able to Open Application Web Page .......................................................................................................... 75
16.4. Model Zoo Projects or Workspaces Are Not Available ..................................................................................... 75
16.5. Not Able to See Camera for Stream Dataset ..................................................................................................... 75
Technical Support Assistance ............................................................................................................................................. 77
Revision History .................................................................................................................................................................. 78

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

4 FPGA-AN-02039-1.2
Lattice sensAI Studio User Guide
Application Note

Figures
Figure 2.1. Public Dataset Configuration ............................................................................................................................ 11
Figure 2.2. Public Dataset Path Selection Reference .......................................................................................................... 11
Figure 2.3. Server IP Configuration ..................................................................................................................................... 11
Figure 2.4. Https Configuration for API PATH ..................................................................................................................... 12
Figure 2.5. Https Configuration in ltf.config ....................................................................................................................... 12
Figure 2.6. Https Configuration in docker-componse.yaml ................................................................................................ 12
Figure 2.7. Web Server Frontend Port Proxy Configuration for https ................................................................................ 12
Figure 2.8. Web Server Backend Port Proxy Configuration for https ................................................................................. 13
Figure 2.9. Web Server TensorBoard Port Proxy Configuration for https .......................................................................... 13
Figure 2.10. Web Server JupyterLab Port Proxy Configuration for https ........................................................................... 13
Figure 2.11. NFS Configurations ......................................................................................................................................... 14
Figure 2.12. Kubernetes Configurations ............................................................................................................................. 14
Figure 3.1. Deploying sensAI Studio ................................................................................................................................... 15
Figure 4.1. Login Page ......................................................................................................................................................... 16
Figure 4.2. Password Change Window ............................................................................................................................... 16
Figure 4.3. User Volume Configuration .............................................................................................................................. 17
Figure 4.4. Web Tour .......................................................................................................................................................... 17
Figure 4.5. Sign Out Window .............................................................................................................................................. 18
Figure 4.6. Change Password Option .................................................................................................................................. 18
Figure 4.7. Change Password Window ............................................................................................................................... 19
Figure 5.1. Dashboard Page ................................................................................................................................................ 20
Figure 6.1. Project Page ...................................................................................................................................................... 21
Figure 6.2. View Project ...................................................................................................................................................... 21
Figure 6.3. Update Project .................................................................................................................................................. 22
Figure 6.4. Add New Project ............................................................................................................................................... 22
Figure 6.5. Clone Project..................................................................................................................................................... 23
Figure 6.6. Pull Project ........................................................................................................................................................ 23
Figure 6.7. Create Local Model Project............................................................................................................................... 24
Figure 6.8. Local Project Training Example ......................................................................................................................... 26
Figure 6.9. Local Project TensorBoard Example ................................................................................................................. 27
Figure 6.10. Local Project Freeze Example ......................................................................................................................... 27
Figure 6.11. Download Model JSON ................................................................................................................................... 28
Figure 6.12. Project Graph .................................................................................................................................................. 29
Figure 6.13. Project Page .................................................................................................................................................... 29
Figure 7.1. Workspace Page ............................................................................................................................................... 30
Figure 7.2. Add Public Workspace ...................................................................................................................................... 30
Figure 7.3. Add Custom Workspace ................................................................................................................................... 31
Figure 8.1. Dataset Page ..................................................................................................................................................... 32
Figure 8.2. Dataset View ..................................................................................................................................................... 32
Figure 8.3. Dataset Type Selection ..................................................................................................................................... 33
Figure 8.4. Create Public Dataset ....................................................................................................................................... 33
Figure 8.5. Create Dataset from URL .................................................................................................................................. 34
Figure 8.6. AWS Credential File .......................................................................................................................................... 34
Figure 8.7. AWS Dataset Creation ...................................................................................................................................... 35
Figure 8.8. GCP Credential File ........................................................................................................................................... 35
Figure 8.9. GCP Dataset Creation ....................................................................................................................................... 36
Figure 8.10. Upload Dataset ............................................................................................................................................... 36
Figure 8.11. Manual Capture .............................................................................................................................................. 37
Figure 8.12. Continuous Capture ........................................................................................................................................ 38
Figure 8.13. Video Stream .................................................................................................................................................. 38
Figure 8.14. Edit Dataset .................................................................................................................................................... 39
Figure 8.15. Dataset Edit Options ....................................................................................................................................... 39

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-AN-02039-1.2 5
Lattice sensAI Studio User Guide
Application Note

Figure 8.16. View Dataset ...................................................................................................................................................40


Figure 8.17. Dataset Visualization Option ..........................................................................................................................40
Figure 8.18. Dataset Visualization Window ........................................................................................................................41
Figure 8.19. Dataset Visualization Top Panel ......................................................................................................................41
Figure 8.20. Dataset Visualization Bottom Panel ................................................................................................................41
Figure 8.21. Change Class Label in Dataset Visualization ...................................................................................................42
Figure 8.22. Resize Box in Dataset Visualization .................................................................................................................42
Figure 8.23. Dataset Visualization Delete Box ....................................................................................................................43
Figure 8.24. GCP Credential File .........................................................................................................................................44
Figure 8.25. GCP Dataset Creation......................................................................................................................................44
Figure 8.26. Dataset Auto-Annotation ................................................................................................................................44
Figure 8.27. Dataset Auto-Annotation Model Selection .....................................................................................................45
Figure 8.28. Start Dataset Auto-Annotation .......................................................................................................................45
Figure 8.29. Convert Dataset Annotation ...........................................................................................................................46
Figure 8.30. Dataset Annotation Conversion Options ........................................................................................................46
Figure 8.31. Dataset Annotation Start Conversion .............................................................................................................46
Figure 8.32. Dataset Download ..........................................................................................................................................47
Figure 8.33. Dataset Pre-Processing ...................................................................................................................................47
Figure 8.34. Dataset Augmentation ....................................................................................................................................49
Figure 8.35. Merge Dataset in the Dataset Page ................................................................................................................50
Figure 8.36. Merge Dataset Dialog .....................................................................................................................................50
Figure 9.1. The Job Page .....................................................................................................................................................51
Figure 9.2. Add Job .............................................................................................................................................................52
Figure 9.3. Job Logs .............................................................................................................................................................53
Figure 9.4. Freeze Button....................................................................................................................................................53
Figure 9.5. Freeze Logs .......................................................................................................................................................53
Figure 9.6. TensorBoard Launch Button .............................................................................................................................54
Figure 9.7. TensorBoard Window .......................................................................................................................................54
Figure 9.8. Neural Network Compiler Job ...........................................................................................................................55
Figure 9.9. Neural Network Compiler Options ...................................................................................................................56
Figure 9.10. Neural Network Compiler Logs .......................................................................................................................56
Figure 9.11. Download Artefacts ........................................................................................................................................57
Figure 9.12. Select Download File .......................................................................................................................................57
Figure 9.13. Encrypt Model.................................................................................................................................................58
Figure 9.14. Download Complete Job .................................................................................................................................58
Figure 9.15. Schedule a Job ................................................................................................................................................59
Figure 9.16. Compare Jobs ..................................................................................................................................................60
Figure 9.17. Job Details Comparison...................................................................................................................................60
Figure 9.18. Job Parameter Comparison.............................................................................................................................60
Figure 9.19. Job Metrics Comparison .................................................................................................................................60
Figure 9.20. Job Graphs ......................................................................................................................................................61
Figure 10.1. JupyterHub for Code .......................................................................................................................................62
Figure 11.1. User Creation Option ......................................................................................................................................63
Figure 11.2. Add User Option .............................................................................................................................................63
Figure 11.3. Create User .....................................................................................................................................................64
Figure 11.4. Remove User Option .......................................................................................................................................64
Figure 11.5. Update User Option ........................................................................................................................................65
Figure 11.6. Update User Details ........................................................................................................................................65
Figure 12.1. Group Management Option ...........................................................................................................................66
Figure 12.2. Group List ........................................................................................................................................................66
Figure 12.3. Create a Group ................................................................................................................................................67
Figure 12.4. Update Group Option .....................................................................................................................................67
Figure 12.5. Adding and Removing Group Members .........................................................................................................67
Figure 12.6. Delete Group Option.......................................................................................................................................68

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

6 FPGA-AN-02039-1.2
Lattice sensAI Studio User Guide
Application Note

Figure 13.1. Memory Management Option ........................................................................................................................ 69


Figure 13.2. Memory Management Global Configuration ................................................................................................. 69
Figure 13.3. Memory Management User Configuration .................................................................................................... 70
Figure 13.4. Memory Management Global Configuration Update ..................................................................................... 70
Figure 15.1. Kubernetes Service Account, Roles and Namespace ...................................................................................... 72
Figure 15.2. Kubernetes Service Account, Roles and Namespace Creation ....................................................................... 73
Figure 15.3. API Token ........................................................................................................................................................ 73
Figure 16.1. Boot Error during Database Setup .................................................................................................................. 74
Figure 16.2. Dataset Structure Tool Tip .............................................................................................................................. 74
Figure 16.3. Dataset Structure Example – Kitti ................................................................................................................... 75
Figure 16.4. Chrome Treat Origin as Secure ....................................................................................................................... 76

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-AN-02039-1.2 7
Lattice sensAI Studio User Guide
Application Note

Acronyms in This Document


A list of acronyms used in this document.
Acronym Definition
API Application Programming Interface
GPU Graphics Processing Unit
HTTP Hypertext Transfer Protocol
IP Internet Protocol
JSON JavaScript Object Notation
ML Machine Learning
NN Neural Network

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

8 FPGA-AN-02039-1.2
Lattice sensAI Studio User Guide
Application Note

1. Introduction
The Lattice sensAI™ Studio is a user interface-based tool for training, validating, and compiling Machine Learning (ML)
models optimized for Lattice FPGAs. The tool makes it easy to take advantage of transfer learning to deploy ML models.
It allows you to select from a range of models pre-trained to cover popular use cases, bring your own data for
additional training, validate the quality of training using TensorBoard, and compile for Lattice FPGAs.
The features of the Lattice sensAI Studio are:
• Model Zoo with variety of models based on multiple architecture
• Option to bring your own model and environment into framework
• Easy to use dataset pre-processing such as labeling, annotation conversion, augmentation
• Model training, freezing, TensorBoard
• Different types of transfer learning
• Storage management
• Informative dashboard for better visualization
• Support to compile model for various Lattice provided FPGAs
• JupyterLab interface for code editing
• User management and collaboration
• DIY installation and setup with docker container on CPU/GPU PC/Server/Cloud
• Support for Distributed Machine Learning training on cloud using Kubernetes
This document describes the installation/configuration, usage, and troubleshooting of the Lattice sensAI Studio.

1.1. Pre-requisites
The hardware, software, connection, and general requirements for this demonstration are provided in the following
sections.

1.1.1. Hardware Requirement


The software requires the following hardware component:
• PC with Ubuntu 16.04/18.04/20.04 x64 distribution

1.1.2. Software Requirements


This software product requires the following software components to be installed:
• NVIDIA Container Toolkit- to use NVIDIA GPUs for model training
• Docker-ce (version 20.10.2 or above)
• Docker-compose (version 1.27.4 or above)
• Web browser: Google Chrome (preferred) or Firefox
• Pull preferred docker images (to get process faster after webapp is started)
• Training docker image
• docker pull ghcr.io/latticesemi/ml-training:tf-1.14.0
• Custom Ubuntu image
• docker pull ghcr.io/latticesemi/ubuntu
• Jupyter image
• docker pull ghcr.io/latticesemi/jupyter
• NN compiler (sensAI) (pull versions which are to be used)
• docker pull ghcr.io/latticesemi/sensai:v4.1
• docker pull ghcr.io/latticesemi/sensai:v5.0
• Create logs volume (run bellow command to create volume)
• docker volume create logs
• Pull latest sensAI Studio docker images

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-AN-02039-1.2 9
Lattice sensAI Studio User Guide
Application Note

• docker pull ghcr.io/latticesemi/web:<version>


• docker pull ghcr.io/latticesemi/app:<version>
For example:
• docker pull ghcr.io/latticesemi/web:v1.4
• docker pull ghcr.io/latticesemi/app:v1.4

1.2. Limitations or Known Issues


• None

1.3. Stop and Remove Older Version of Webapp


• Run the command below from the directory where the webapp is started previously.
• docker-compose --env-file ltf.config down
• Run the command below to remove extra containers and volumes (optional: Required if you want to do clean
setup again).
• docker container prune
• docker volume prune

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

10 FPGA-AN-02039-1.2
Lattice sensAI Studio User Guide
Application Note

2. sensAI Studio Installation Configuration

2.1. Public Dataset Configuration


• Configure the local dataset path in ltf.config file to make it accessible as a public dataset.

Figure 2.1. Public Dataset Configuration

• Update the absolute path of local dataset parent at both locations as mentioned in the image above.
Note: This is only required if you want to make any dataset public.
• Check the reference image below for path selection. As Figure 2.2 shows, there are multiple datasets available in
dataset folder. You can add parent path where all datasets are present.

Figure 2.2. Public Dataset Path Selection Reference

2.2. Server IP Configuration


• Server IP configuration is required to access the webapp. It enables webapp access through public IP or internal IP.
• Update the deployment machine IP variable named “BACKEND_SERVER_ADDRESS” in ltf.config.
Note: Please note that localhost or 127.0.0.1 does not work as the server IP.

Figure 2.3. Server IP Configuration

2.3. HTTPs Support Configuration


This configuration is only required if the user wants to deploy sensAI Studio with https URL.
To run sensAI Studio with https, below changes are required apart from the configurations mentioned above.

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-AN-02039-1.2 11
Lattice sensAI Studio User Guide
Application Note

• Change the API path as below in ltf.config:

Figure 2.4. Https Configuration for API PATH

• Enable HTTPS in ltf.config as below:

Figure 2.5. Https Configuration in ltf.config

• Change API_PATH in docker-compose.yaml as below in the services/web section:

Figure 2.6. Https Configuration in docker-componse.yaml

Here “/api” is where a request is passed from frontend to backend. Same needs to be configured in the web server
configuration (for example, in nginx conf).
Apart from the configurations above, a web server also needs to be configured to redirect request from API path to API
port and vice versa. Below are some example configurations for setting up a proxy.
• Proxy ‘/’ to frontend server port (SERVER_PORT). See the example below:

Figure 2.7. Web Server Frontend Port Proxy Configuration for https

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

12 FPGA-AN-02039-1.2
Lattice sensAI Studio User Guide
Application Note

• Proxy API path (for example, /api) to backend server port (BACKEND_SERVER_PORT)

Figure 2.8. Web Server Backend Port Proxy Configuration for https

• Proxy TensorBoard port to /tensorboard

Figure 2.9. Web Server TensorBoard Port Proxy Configuration for https

• Proxy JupyterLab code port to /code

Figure 2.10. Web Server JupyterLab Port Proxy Configuration for https

2.4. NFS Configuration


sensAI Studio supports NFS as an option for storage. If this option is enabled, all user data is stored in the NFS drive
instead of the local system. This requires changing the following variables in ltf.config.
• Set NFS_VOLUME=true.
• Add NFS_SERVER_IP.
• Add NFS SERVER_PATH_USERDATA.
Note: NFS must be accessible from the deployment server.

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-AN-02039-1.2 13
Lattice sensAI Studio User Guide
Application Note

Figure 2.11. NFS Configurations

Refer to Create an NFS file share on AWS.

2.5. Kubernetes Configuration


sensAI Studio supports running Machine Learning training jobs on Kubernetes cluster. To use Kubernetes, NFS is
mandatory as the data needs to be stored in shared path which is accessible from the deployment server and
Kubernetes cluster.
• Change the following variables in ltf.config:
• Set ENABLE_KUBERNETES=true.
• Provide the Kubernetes cluster name.
• Provide Kubernetes API server and port details.
• Provide the namespace which is to be used for creating training pods.
• Note-Application should have the required permissions in the given namespace.
• Provide the username and API token to access Kubernetes cluster.
Note: The Kubernetes master node must be accessible from the deployment server.

Figure 2.12. Kubernetes Configurations

Refer to AWS Kubernetes (EKS) Cluster Creation for example of creating cluster on AWS cloud.

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

14 FPGA-AN-02039-1.2
Lattice sensAI Studio User Guide
Application Note

3. Deploying sensAI Studio


This chapter describes how to deploy sensAI Studio.

3.1. Deployment
• Deploy the webapp with the command below:
# docker-compose --env-file ltf.config up -d

Figure 3.1. Deploying sensAI Studio

Note: ‘-d’ is for running app as daemon. To debug and get logs, run command without ‘-d’.
• Go to http://<SERVER_IP>:< SERVER_PORT> in the browser to launch sensAI Studio. For HTTPs cases, use the web
address specified as BACKEND_SERVER_ADDRESS to access application.

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-AN-02039-1.2 15
Lattice sensAI Studio User Guide
Application Note

4. Getting Started
This section provides information on how to use the Lattice Machine Learning Training Framework.

4.1. Logging in the Web Application


To log into the web application:
1. Enter http://<SERVER_IP>:6001 in the web browser and the login page is displayed. The default credentials are
listed as follows:
• Email – [email protected]
• Password – 12345678

Figure 4.1. Login Page

Note: For the fresh install, the initial database setup takes some time. If you notice a login error at first login, wait
for some time and try again.
2. Type in the new password and select the “I agree terms and conditions” box when you are asked to change the
password during the first login.
3. Click on Change.

Figure 4.2. Password Change Window

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

16 FPGA-AN-02039-1.2
Lattice sensAI Studio User Guide
Application Note

4. If the login is successful, the user is directed to the volume configuration page. Users can provide their own NFS
details if they want to store data on NFS. If the user selects default, the user data is stored in volume configured by
the system admin in docker-compose.yaml.

Figure 4.3. User Volume Configuration

5. After the volume configuration, the user is redirected to the Dashboard page. For first time users, you can view the
web tour before moving to any other window. This helps you go through all the major parts of the web application.

Figure 4.4. Web Tour

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-AN-02039-1.2 17
Lattice sensAI Studio User Guide
Application Note

4.2. Logging Out of the Web Application


To log out of the web application:
1. Click on the user panel on the top-right corner of the web application.
2. Click on SIGN OUT to log out from the web application.

Figure 4.5. Sign Out Window

4.3. Changing the Password


To change your password:
1. Click on the user panel on the top-right corner of the web application.

Figure 4.6. Change Password Option

2. Click on CHANGE PASSWORD.

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

18 FPGA-AN-02039-1.2
Lattice sensAI Studio User Guide
Application Note

Figure 4.7. Change Password Window

3. Provide the new password and click on Change.

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-AN-02039-1.2 19
Lattice sensAI Studio User Guide
Application Note

5. Dashboard
The Dashboard module has different kinds of graphs available. These graphs allow you and the admin user to have a
better visualization of the statistics. By default, you are redirected to the Dashboard page after logging into the web
application. You can also go to Dashboard at any time by clicking on Dashboard from all the page links available in the
main menu.
These graphs provide different information about projects, workspaces, datasets, jobs, memory usage, and so on.

Figure 5.1. Dashboard Page

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

20 FPGA-AN-02039-1.2
Lattice sensAI Studio User Guide
Application Note

6. Project
The Project page contains all the existing projects you created.

Figure 6.1. Project Page

6.1. Viewing Project Details


To view the project details:
1. Go to the Project page.

2. Click on the icon to view the project details based on project type, as shown in Figure 6.2.

Figure 6.2. View Project

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-AN-02039-1.2 21
Lattice sensAI Studio User Guide
Application Note

6.2. Updating Project Details


To update a project:
1. Go to the Project page.

2. Click on the icon .

Figure 6.3. Update Project

3. Modify the required details and click on Update.

6.3. Adding Project


To create a new project, click on Add Project. It shows the available options to add a project.
• Model Zoo – From the Lattice Model Zoo repository.
• Local Model – You can upload your own model to create project.
• Other Repo – Currently, this option is not supported.

Figure 6.4. Add New Project

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

22 FPGA-AN-02039-1.2
Lattice sensAI Studio User Guide
Application Note

6.3.1. Model Zoo


To create a project from Model Zoo:
1. Enter the Project Name and Project Description.
2. If a project is part of a group, select the group from the drop-down menu. If no group is selected, it is created in
your private space or in a selected group space.
3. Select Model Zoo as project type.
4. Click on Clone on the project that should be created. You can also filter the list using the Advance Search option.

Figure 6.5. Clone Project

5. Once the project is created, it is reflected in the Project page.

Pulling the Latest Code


You can pull the latest changes from the Model Zoo by clicking on the icon from the Project page.

Figure 6.6. Pull Project

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-AN-02039-1.2 23
Lattice sensAI Studio User Guide
Application Note

6.3.2. Local Model


To create a project from your local repository:
1. Enter the Project Name and Project Description.
2. If a project is part of a group, select the group from the drop-down menu. If no group is selected, it is created in
your private space or in a selected group space.
3. Select Local Model as project type.
4. Enter the Model Name and select the Target Framework, Framework Version, and Dataset Type from the
drop-down menu.
5. Open and select the JSON file. This is optional since the details from the JSON file are automatically populated if
the JSON is uploaded. You can download the sample JSON for reference, if required.
6. Click on Attach Files to upload the Project Zip.

Figure 6.7. Create Local Model Project

7. To create new sections, click on Add Section. These sections are reflected in the section list of the job created, and
can also be used for Training, TensorBoard, and Freezing. For each section, you can provide the command to run by
adding the Command field.
The command can also have other required/optional parameters. These parameters can be provided by adding
parameters under a specified section.
To add a new parameter, click on Add Parameter.
Note: There are different fields of parameters:
• Visible name – This is the name which is provided to user during Job creation.
• Argument – The argument provided with section command (--<argument>).
• Default Value – Default value shown for parameter during the job creation.
• Type – Type of argument. The possible options are listed below:
• file – Used for execution script. This file parameter can be a log file, the input files needed while running
section command.
• str – String Parameter
• int – Integer Parameter
• dir – Directory. When dir is selected as type, the framework automatically selects the path based on the
Sub Type.
• If Sub Type is dataset, the framework automatically identifies the path of the selected dataset.

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

24 FPGA-AN-02039-1.2
Lattice sensAI Studio User Guide
Application Note

• If Sub Type is artefacts, the framework automatically identifies the path of the job artefacts. This is
used for model output directory.
• If Sub Type is checkpoint, the framework automatically identifies the path of the job artefacts. This is
used for checkpoint path. The artefacts and checkpoint path can be the same. Checkpoint subtype is
used for transferring learning where pre-trained checkpoints are copied to.
• transfer learning – You can provide the link to the pre-trained checkpoint zip for transfer learning. This
shows the option to do transfer learning from pre-trained checkpoints while creating the job.
• checkboxes – This can be used for layer freezing. This option can only be used from JSON upload. Refer to
the sample JSON format for details on how to use this. All option values are passed to the command
separated by a comma.
• workspace – This option is used for sections other than training (for example, freezing). This provides you
an option to select the workspace other than the training workspace.
Figure 6.8, Figure 6.9, and Figure 6.10 show the reference examples for the Training, TensorBoard and Freezing
sections.
8. Once all settings are completed, click on Save to create the project.

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-AN-02039-1.2 25
Lattice sensAI Studio User Guide
Application Note

Figure 6.8. Local Project Training Example

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

26 FPGA-AN-02039-1.2
Lattice sensAI Studio User Guide
Application Note

Figure 6.9. Local Project TensorBoard Example

Figure 6.10. Local Project Freeze Example

Download Project JSON


You can also download the JSON for the local project. To do this, view the project details as mentioned in Viewing
Project Details, and then click on Download JSON.

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-AN-02039-1.2 27
Lattice sensAI Studio User Guide
Application Note

Figure 6.11. Download Model JSON

6.3.3. Other Repo


This option is not supported in the current release.

6.4. Project Graph


Project graphs helps user to compare runs of different jobs for the given project.
Follow below steps to view graphs of project for different job runs:
1. Go to the Project page.

2. Click on the icon for the required project.


3. Select the Metrics parameter(s).
4. Click on Apply. This shows the project graph as below (Figure 6.12).

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

28 FPGA-AN-02039-1.2
Lattice sensAI Studio User Guide
Application Note

Figure 6.12. Project Graph

6.5. Download Project Zip


To download project files:
1. Go to the Project page.

Figure 6.13. Project Page

2. Click on the icon to download project zip.


Note: For AutoKeras model, refer to Lattice sensAI Neural Network Compiler User Guide (FPGA-UG-02052).

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-AN-02039-1.2 29
Lattice sensAI Studio User Guide
Application Note

7. Workspace
The Workspace page contains all the workspaces you created. Initially, there is a default workspace.

Figure 7.1. Workspace Page

7.1. Adding Public Workspace


To add a public workspace:
1. Click on Add Workspace.
2. Select the Public Workspace option.
3. Enter the Workspace Name and Description. Click on Create Workspace on the workspace that is to be created.
4. If a workspace is part of a group, select the group from the drop-down menu. If no group is selected, it is created in
your private space or in a selected group space.
5. Once the workspace is created, it is reflected in the Workspace page.

Figure 7.2. Add Public Workspace

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

30 FPGA-AN-02039-1.2
Lattice sensAI Studio User Guide
Application Note

7.2. Adding Custom Workspace


To add a custom workspace:
1. Click on Add Workspace.
2. Select the Custom Workspace option.
3. Enter the Workspace Name and Description.
4. If a workspace is part of a group, select the group from the drop-down menu. If no group is selected, it is created in
your private space or in a selected group space.
5. Enter the Framework, Framework Version, and Docker Image URL.
6. Click on Save to create the workspace.

Figure 7.3. Add Custom Workspace

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-AN-02039-1.2 31
Lattice sensAI Studio User Guide
Application Note

8. Dataset
The Dataset page contains all the datasets you created.

Figure 8.1. Dataset Page

8.1. Viewing Dataset Details


To view the dataset details:
1. Go to the Dataset page.

2. Click on the icon to view the dataset details, as shown in Figure 8.2.

Figure 8.2. Dataset View

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

32 FPGA-AN-02039-1.2
Lattice sensAI Studio User Guide
Application Note

8.2. Adding Dataset


To add a dataset, click on Add Dataset. It shows the different types of datasets available:
• Public – Create dataset from dataset directory, which is shared while starting the web application in the
docker-compose file.
• Upload Dataset – Create dataset by uploading file.
• URL – Download dataset from using link to dataset.
• AWS/GCP – Not supported in current release.

Figure 8.3. Dataset Type Selection

8.2.1. Public
To add a Public dataset:
1. Select the Public option.
2. Provide the Dataset Name and Description.
3. Select Label Type and Dataset Path. If you do not select Other as label type, click on Validate to validate the
dataset.
4. Click on Save.
5. Once the dataset is created, it is reflected in the Dataset page.

Figure 8.4. Create Public Dataset

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-AN-02039-1.2 33
Lattice sensAI Studio User Guide
Application Note

8.2.2. URL
To add dataset using Dataset URL:
1. Select the URL option.
2. Provide the Dataset Name and Description.
3. Select Label Type and provide the Dataset URL. If you do not select Other as label type, click on Validate to
validate the dataset.
4. Click on Save.
5. Once the dataset is created, it is reflected in the Dataset page.

Figure 8.5. Create Dataset from URL

8.2.3. AWS
To add dataset from AWS S3 bucket:
1. Select the AWS option.
2. Provide Dataset Name and Description.
3. Select Label Type.
4. Provide Bucket Name and Dataset Path (Dataset path inside the AWS bucket).
5. Click on Attach Files and upload the credential file in .txt format. The format for the same is given below.

Figure 8.6. AWS Credential File

6. Tick the Copy dataset to local option to download the dataset (It might take some time depending on dataset size
and network bandwidth). This is optional and required if you want to copy dataset to the user volume (to save
training time).
7. Click on Validate to validate the dataset.
8. Click on Save.
9. Once the dataset is created, it is reflected in the Dataset page.

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

34 FPGA-AN-02039-1.2
Lattice sensAI Studio User Guide
Application Note

Figure 8.7. AWS Dataset Creation

8.2.4. GCP
To add the dataset from Google Cloud Platform (GCP):
1. Select the GCP option.
2. Provide Dataset Name and Description.
3. Select Label Type.
4. Provide Bucket Name and Dataset Path (Dataset path inside the GCP bucket).
5. Click on Attach Files and upload the service account credentials in .json format. The format of the same file is given
below.

Figure 8.8. GCP Credential File

6. Tick the Copy dataset option to download the dataset (It might take time depending on dataset size and network
bandwidth). This is optional and required if you want to copy dataset to the user volume (to save training time).
7. Click on Validate to validate the dataset.
8. Click on Save.
9. Once the dataset is created, it is reflected in the Dataset page.

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-AN-02039-1.2 35
Lattice sensAI Studio User Guide
Application Note

Figure 8.9. GCP Dataset Creation

8.2.5. Uploading Dataset


To upload dataset from your local machine:
1. Select the Upload Dataset option.
2. Provide the Dataset Name and Description.
3. Select Label Type. If you do not select Other as the label type, click on Validate to validate the dataset.
4. Click on Attach Files to attach the dataset file and wait for the upload to complete.
5. Click on Save.
6. Once the dataset is created, it is reflected in the Dataset page.

Figure 8.10. Upload Dataset

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

36 FPGA-AN-02039-1.2
Lattice sensAI Studio User Guide
Application Note

8.2.6. Stream Dataset


To upload dataset from your local machine:
1. Select the Stream Dataset option.
2. Provide the Dataset Name and Description.

8.2.6.1. Manual Capture


1. Click on Start Manual Capture.
2. Select the camera from the drop-down menu. If the camera is not displayed in the menu, refer to Not Able to See
Camera for Stream Dataset for troubleshooting.
3. Capture the required number of images.
4. Click on Save to save dataset.

Figure 8.11. Manual Capture

8.2.6.2. Continuous Capture


1. Provide values for Capture interval, Capture Duration and Capture Duration Interval.
2. Click on Start Continuous Capture.
3. Click on Save to save dataset.

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-AN-02039-1.2 37
Lattice sensAI Studio User Guide
Application Note

Figure 8.12. Continuous Capture

8.2.6.3. Video Stream


1. Select Stream Type as Video Stream.
2. Provide values for Capture interval and Capture Duration Interval.
3. Provide Stream Url as a working video link.
4. Click on Save to save dataset.

Figure 8.13. Video Stream

8.3. Editing Dataset


User can edit datasets to:
• Change train/val/test splitting ratio.
• Add more images to existing Upload Dataset option.
This option is only available for KITTI and VOC datasets.
Follow the steps below to edit a dataset:

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

38 FPGA-AN-02039-1.2
Lattice sensAI Studio User Guide
Application Note

1. Go to the Dataset page.


2. Click on the Edit icon from the dataset tile.

Figure 8.14. Edit Dataset

3. Attach zip file if the dataset needs to be appended.


4. Change the train/test/val split ratio if required.
5. The attached zip needs to be validated.
6. Click on Save to save the dataset.

Figure 8.15. Dataset Edit Options

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-AN-02039-1.2 39
Lattice sensAI Studio User Guide
Application Note

8.4. Visualizing the Dataset


Once a dataset is created, you can use this option to visualize the dataset. This option is available for KITTI and VOC
datasets.
To visualize the dataset:
1. Go to the Dataset page.
2. Click on the View dataset icon, as shown in Figure 8.16.

Figure 8.16. View Dataset

3. Click on Visualization.

Figure 8.17. Dataset Visualization Option

The window displays the dataset images with bounding boxes drawn, based on labels from the dataset (Figure 8.18).
Different classes are displayed with different colors.

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

40 FPGA-AN-02039-1.2
Lattice sensAI Studio User Guide
Application Note

Figure 8.18. Dataset Visualization Window

You have a different option to save the modified annotation, change the class, draw new boxes, resize the existing box,
or delete the existing box in the top panel (Figure 8.19).

Figure 8.19. Dataset Visualization Top Panel

You can also jump to a specific image by clicking on the Go to image option. Different classes are marked with different
box colors, which can be viewed from bottom panel.

Figure 8.20. Dataset Visualization Bottom Panel

8.4.1. Changing Existing Class


To change the existing class:
1. Click on the bounding box of the class you want to change. The drop-down menu of existing labels appears on the
top panel, as shown in Figure 8.21.
2. Select the new class from the drop-down menu on the top-right corner of the page. If the required label is not
present, click on Create New to create a new label.
3. Click on Save.

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-AN-02039-1.2 41
Lattice sensAI Studio User Guide
Application Note

Figure 8.21. Change Class Label in Dataset Visualization

8.4.2. Resizing Existing Bounding Box


To resize an existing bounding box:
1. Click on the icon on the top panel if it is not selected.
2. Hover the mouse on any corner of the bounding box. Once the corner is detected, drag and drop the corner to
resize the box.
3. Click on Save.

Figure 8.22. Resize Box in Dataset Visualization

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

42 FPGA-AN-02039-1.2
Lattice sensAI Studio User Guide
Application Note

8.4.3. Drawing a New Bounding Box


To draw a new bounding box:
1. Click on the icon on the top panel if it is not selected.
2. Draw a box on the image.
3. Change the label as mentioned in Changing Existing Class section.
4. Click on Save.

8.4.4. Deleting a New Bounding Box


To delete a new bounding box:
1. Click on the icon on the top panel of the screen.
2. Delete the box which needs to be deleted by clicking on the X symbol.
3. Click on Save on the top panel to save changes.

Figure 8.23. Dataset Visualization Delete Box

8.5. Auto-Annotation
You can create a dataset from plain images and generate auto-annotated dataset using pre-trained models. To do this,
perform the steps below:
1. Create a compressed file (zip/tar) from the images, and create a dataset as mentioned in the GCP section:
a. Select the GCP option.
b. Provide Dataset Name and Description.
c. Select Label Type.
d. Provide Bucket Name and Dataset Path (Dataset path inside the GCP bucket).
e. Click on Attach Files and upload the service account credentials in .json format. The format of the same
file is given below.

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-AN-02039-1.2 43
Lattice sensAI Studio User Guide
Application Note

Figure 8.24. GCP Credential File

2. Tick the Copy dataset to local option to download the dataset (It might take some time depending on the dataset
size and network bandwidth). This is optional and required if you want to copy dataset to the user volume (to save
training time).
3. Click on Validate to validate the dataset.
4. Click on Save.
5. Once the dataset is created, it is reflected in the Dataset page.

Figure 8.25. GCP Dataset Creation

6. Click on View Dataset on the dataset tile.


7. Expand the Dataset Auto Annotation option.

Figure 8.26. Dataset Auto-Annotation

8. Click on Start. It displays the available models for auto-annotation.

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

44 FPGA-AN-02039-1.2
Lattice sensAI Studio User Guide
Application Note

Figure 8.27. Dataset Auto-Annotation Model Selection

9. Select the required model from the list and click on Start. Once annotation starts, the progress can be seen, as
shown in Figure 8.28.

Figure 8.28. Start Dataset Auto-Annotation

You can refresh the status by clicking on the Refresh icon.


Note: The current model only works on plain images. The dataset type may change after annotation based on the
model script. Therefore, re-annotation may not work once auto-annotation is done.

8.6. Converting Annotation


You can convert dataset from one type to another type. To convert an annotation:
1. Click on View Dataset on the dataset tile.
2. Expand the Annotation Conversion option.

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-AN-02039-1.2 45
Lattice sensAI Studio User Guide
Application Note

Figure 8.29. Convert Dataset Annotation

3. Select the current label format as From and the required label format as To.
4. Click on Start.

Figure 8.30. Dataset Annotation Conversion Options

5. Once conversion starts, the progress can be seen in the window, as shown in Figure 8.31.

Figure 8.31. Dataset Annotation Start Conversion

You can refresh the status by clicking on the Refresh icon.

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

46 FPGA-AN-02039-1.2
Lattice sensAI Studio User Guide
Application Note

8.7. Downloading Dataset


To download the dataset, click on Download from the dataset tile, as shown in Figure 8.32.

Figure 8.32. Dataset Download

8.8. Pre-Processing the Dataset


There are different options available in the framework to process data like augmentation, conversion, and annotation.
However, if you want to do some custom processing which is not available in framework, you can do it using the
Dataset Pre-processing option.
For custom dataset pre-processing, you need to have a docker image with the required scripts and environment to do
the processing. You can create a custom workspace; for the details, refer to the Adding Custom Workspace section.
Once the workspace is created, you can perform the steps below to pre-process the dataset:
1. Open the dataset, and go to the Dataset Preprocessing option.
2. Select the workspace, which has the required script and environment to run it.
3. Enter the command to run. Note that only a single command can be run at a time. You can run multiple commands
by using shell options, like combining multiple commands with && or providing them with semicolon separated.
4. Click on Start.

Figure 8.33. Dataset Pre-Processing

You can click on the Refresh icon to refresh the processing status. You can also visualize logs by clicking on the Log
button (if required).

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-AN-02039-1.2 47
Lattice sensAI Studio User Guide
Application Note

Notes:
• User Dataset is available in /lattice/dataset/<dataset-name> location. For example, if dataset name is
human_dataset, the root directory is /lattice/dataset/human_dataset.
• You can use your projects paths from this option using the same way as a dataset (for example,
/lattice/projects/<proj-name>).

8.8.1. Dataset Conversion Sample


The dataset pre-processing can also be used for auto-annotation. You can bring your workspace with the required
script, which can convert dataset from one format to another.
To do auto-annotation using your own model, perform the steps below:
1. Create the docker image, with the modifications below, and host it on the public docker repository:
• You need to create a script to do the conversion from one format to another format. This script should be able
to take the input path and output path as input so it can work on any dataset name used to create the dataset
in the Lattice sensAI Studio.
Note: You can skip this if a script is already present somewhere (for example, projects and any other workspace) and
have all the packages to run the script.
2. Create the custom workspace in the Lattice sensAI studio from the custom docker image.
3. Create the dataset in the Lattice sensAI studio with input format by using the Upload Zip option.
4. Click on the Dataset Pre-processing option.
5. Select the workspace and provide the command to run for annotation.
6. Run the dataset pre-processing.
Note: Dataset format maintained in framework is not changed during the dataset processing. For example, if the script
converts the dataset from X format to Y format, the format specified during dataset creation does not change.

8.8.2. Dataset Auto-Annotation Sample


The dataset pre-processing can also be used for auto-annotation. You can bring your workspace with the required
model and script which can label images.
To perform auto-annotation using your own model, perform the steps below:
1. Create docker image, with the modifications below, and host it on the public docker repository:
• Frozen model for inference
• You need to create a script to do inference on input images and create labels in required format. This script
should be able to take input path and output path as input so it can work on any dataset name used to create
dataset in Lattice sensAI Studio.
Note: You can skip this if pb/script is already present (for example, in projects/jobs and any other workspace) and have
all the packages to run the script.
2. Create the custom workspace in the Lattice sensAI studio from the custom docker image.
3. Create the dataset in the Lattice sensAI studio with plain images by using the Upload Zip option.
4. Click on the Dataset Pre-processing option.
5. Select the workspace and provide the command to run for annotation.
6. Run the dataset pre-processing.
For example, the Human Count SSD MV2 model needs a dataset conversion from VOC to pickle format. This project
contains script to convert it. You can do the conversion by selecting Default Workspace with command as python3
/lattice/projects/mv2ssd/process_dataset.py –data-dir /lattice/dataset/voc/, where mv2ssd is the project name and
voc is the dataset name.
Note: The dataset format maintained in the framework is not changed during the dataset processing. For example, if
the script converts the dataset from plain images to Kitti format, the format specified during the dataset creation does
not change.

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

48 FPGA-AN-02039-1.2
Lattice sensAI Studio User Guide
Application Note

8.9. Dataset Augmentation


You can perform augmentation on the dataset by using the Dataset Augmentation option. You have the flexibility to
select the types of augmentation, percentage of images to augment, and output dimension.
Once the workspace is created, you can perform the steps below to pre-process the dataset:
1. Open the dataset.
2. Select the Dataset Augmentation option.
3. Click on Output Dimension and enable/disable the types of augmentation as required.
4. Provide/Update the default values of augmentation, such as percentage of images to augment and other
parameters.
5. Click on Start to begin the augmentation.

Figure 8.34. Dataset Augmentation

You can click on the Refresh icon to refresh the processing status.

8.10. Dataset append and train/val/test split


Refer to the Editing Dataset section to append dataset to change train/val/test split.

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-AN-02039-1.2 49
Lattice sensAI Studio User Guide
Application Note

8.11. Dataset Merging


The user can merge multiple datasets into one dataset.
Note: Only datasets with the same label type can be merged (Currently only KITTI/VOC supported).
To merge datasets:
1. Select the datasets to be merged.
2. Click on Merge on the top right corner, as shown in Figure 8.35.

Figure 8.35. Merge Dataset in the Dataset Page

3. Add Dataset Name.


4. Click on Merge Dataset, as shown in Figure 8.36.

Figure 8.36. Merge Dataset Dialog

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

50 FPGA-AN-02039-1.2
Lattice sensAI Studio User Guide
Application Note

9. Jobs
The Jobs Page contains all the jobs you created.

Figure 9.1. The Job Page

9.1. Adding Training Job


To add a training job:
1. Click on Add Jobs to create a new job.
2. Provide the Name.
3. Select the Project from the created projects, Workspace from the created workspaces, and Dataset from the
created datasets. Based on the project, different job types are displayed in the Job Type drop-down menu.
4. Select Training for model training. Based on the selected project, the parameters are displayed with default values.
You can change the parameter values, if required.
For example, to change the checkpoint interval, change Checkpoint Interval parameter to required value.
5. Click on Save to start the job. To run the job later sometime, click on Save and Run.

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-AN-02039-1.2 51
Lattice sensAI Studio User Guide
Application Note

Figure 9.2. Add Job

9.1.1. Logs
Once the job is started, the log appears in the log tab as shown in Figure 9.3. To stop the training (if required), click on
Stop.

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

52 FPGA-AN-02039-1.2
Lattice sensAI Studio User Guide
Application Note

Figure 9.3. Job Logs

9.1.2. Freeze
To freeze the model:
1. Go to the Freeze tab in the Job window and it shows the freeze command.
2. Click on Freeze to freeze the model and run the freezing command, which displays the logs, as shown in Figure 9.5.

Figure 9.4. Freeze Button

Figure 9.5. Freeze Logs

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-AN-02039-1.2 53
Lattice sensAI Studio User Guide
Application Note

9.1.3. TensorBoard
To open the TensorBoard, select the TensorBoard tab in the Job window and click on Launch. This displays the
TensorBoard window, as shown in Figure 9.7.

Figure 9.6. TensorBoard Launch Button

Note: If TensorBoard is not launched upon clicking on Launch, click on Refresh.

Figure 9.7. TensorBoard Window

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

54 FPGA-AN-02039-1.2
Lattice sensAI Studio User Guide
Application Note

9.2. Adding Neural Network Compiler Job


You can utilize the Lattice Neural Network Compiler features using the training framework web application. You can
also analyze, compile, and simulate the trained model by choosing NN Compiler as the job type.
To add a neural network compiler job:
1. Select NN Compiler as Job Type.
2. Click on the appropriate command to run from the following options: Analyze, Compile, and Simulate. The Analyze
option is a mandated step for compile and simulate.
3. Provide values for the other parameters required by NN Compiler such as Framework, Class, Device, Model Files,
and so on. The user can also provide encrypted model file (elpb/elh5) as the input model file.

Figure 9.8. Neural Network Compiler Job

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-AN-02039-1.2 55
Lattice sensAI Studio User Guide
Application Note

Figure 9.9. Neural Network Compiler Options

4. Click on Save to run the job. Based on the selected command, the application generates the firmware binary for
the selected device and/or run hardware simulation. You can also see the logs in the log window, as shown in
Figure 9.10.

Figure 9.10. Neural Network Compiler Logs

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

56 FPGA-AN-02039-1.2
Lattice sensAI Studio User Guide
Application Note

9.3. Adding Download Job Artefacts


To download job artefacts, click on Download. This allows you to select the file to download. You can also select the
appropriate file and save it on your chosen folder.

Figure 9.11. Download Artefacts

Figure 9.12. Select Download File

The user has the option to download the model as Encrypt Model (pb/h5) by selecting encryption and providing
encryption password, as shown in Figure 9.13.

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-AN-02039-1.2 57
Lattice sensAI Studio User Guide
Application Note

Figure 9.13. Encrypt Model

This encrypted model can be passed to Lattice sensAI compiler as it is to generate firmware binary. Lattice sensAI
compiler can decrypt the model and parse it.

9.4. Download Complete Job


The user can download the complete job (dataset, project and artefacts).
To download the complete job, click on Download, select Full job checkbox, and click on Download. A zip file with the
complete job containing a run.sh file is downloaded.

Figure 9.14. Download Complete Job

9.5. Modifying a Job


In case you make a mistake while creating a job, you have the option to edit the job instead of creating a new one. This
helps avoid multiple unrequired jobs and save manual efforts in deleting these unrequired jobs.

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

58 FPGA-AN-02039-1.2
Lattice sensAI Studio User Guide
Application Note

9.6. Scheduling a Job


The user can schedule a job to be based on the specific pattern such as daily/weekly/yearly on a specific time and with
a recurrence pattern of n days.
To schedule a job:
1. Click on Add Jobs to create a new job.
2. Provide the Name.
3. Select project from the created projects, workspace from the created workspaces, and dataset from the created
datasets. Based on the project, different job types are displayed in the Job Type drop-down menu.
4. Select Training for model training. Based on the selected project, the parameters are displayed with default values.
You can change the parameter values, if required.
5. For example, to change the checkpoint interval, change Checkpoint Interval parameter to required value.
6. Click on the Schedule.
7. Add details for scheduling such as Start Time, Recurrence Pattern, and the Range of recurrence for scheduling.
8. Click on Schedule.

Figure 9.15. Schedule a Job

9.7. Job Metrics


Metrics are useful to extract information such as loss, accuracy from training, and/or inference outputs. These metrics
can be provided as regular expression by users. The Lattice sensAI Studio parses logs and store those metrics in the
database. You can also compare metrics of different jobs using the job comparison feature.

9.7.1. Adding Metrics


Several Lattice-provided models have sample metrics added by default. You can add, modify, and/or remove
depending on the requirement. You can also add metrics for your custom model while creating a model or running a
job. Refer to the Local Model section to know more on how to add metrics during local project creation. For details on
how to add metrics during job creation, refer to Adding Training Job.

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-AN-02039-1.2 59
Lattice sensAI Studio User Guide
Application Note

9.8. Comparing Jobs


To compare jobs:
1. Go to the Job page.
2. Select the checkboxes for the jobs you want to compare. Maximum of three jobs can be selected for comparison.
3. Click on Compare. This opens the comparison window as shown in Figure 9.16. It compares job information such as
Job Details, Parameters, and Metrics.

Figure 9.16. Compare Jobs

Figure 9.17. Job Details Comparison

Figure 9.18. Job Parameter Comparison

Figure 9.19. Job Metrics Comparison

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

60 FPGA-AN-02039-1.2
Lattice sensAI Studio User Guide
Application Note

9.9. Job Graphs


Job graphs help the user to visualize the statistics of job run. The user can provide metrics to extract while creating a
job and use those metrics to create graphs. Refer to the Job Metrics section for details on how to add metrics to job.
Once a job is complete, follow the steps below to create graphs:
1. Go to the Job page.
2. Click on the icon to open the graph window.
3. Select X-axis parameter and Y-axis parameters.
4. Click on Apply to generate the graph.

Figure 9.20. Job Graphs

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-AN-02039-1.2 61
Lattice sensAI Studio User Guide
Application Note

10. Code
The Code page can be used to explore and update/modify the training code in projects. To navigate the Code page, the
JupyterHub instance is launched and the interface is displayed as shown in Figure 10.1.
Note: If JupyterHub is not launched, click on Refresh.

Figure 10.1. JupyterHub for Code

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

62 FPGA-AN-02039-1.2
Lattice sensAI Studio User Guide
Application Note

11. User Management


The Admin User has the access to create other users and can assign different roles to users based on requirements. The
possible roles are:
• Admin – All permissions and access.
• Manager – Permission to read and create things (projects, workspaces, datasets, and so on) in its own space or
groups.
• Viewer – Read-only access (not supported).
Note: Creating more than one admin is possible, but is not fully-supported.

11.1. Creating a New User


To create a new user:
1. Click on the user panel on the top-right corner of the web application and select USER MANAGEMENT.

Figure 11.1. User Creation Option

2. Click on Add User.

Figure 11.2. Add User Option

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-AN-02039-1.2 63
Lattice sensAI Studio User Guide
Application Note

3. Provide the required details and click on Save to create the user.

Figure 11.3. Create User

11.2. Removing a User


To remove a user:
1. Click on the user panel on the top-right corner of the web application and select USER MANAGEMENT.
2. Click on the Remove icon of the user you want to delete.

Figure 11.4. Remove User Option

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

64 FPGA-AN-02039-1.2
Lattice sensAI Studio User Guide
Application Note

11.3. Updating the User Details


To update user details:
1. Click on the user panel on the top-right corner of the web application and select USER MANAGEMENT.
2. Click on the Edit icon of the user you want to modify the details, as shown in Figure 11.5.
3. Change the details needed and click on Update.

Figure 11.5. Update User Option

Figure 11.6. Update User Details

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-AN-02039-1.2 65
Lattice sensAI Studio User Guide
Application Note

12. Group Management


The Group feature allows you to share projects, dataset, workspaces, and jobs with other users. The entities created in
a group are stored in the group volume instead of your private volume. All members in a group can access them. When
creating projects, dataset, workspaces, and jobs, you can choose where you want to create it. If no group is selected, it
is created in your private space or in a selected group space.
Note: It is advised to use a shared project for shared jobs. Otherwise, all UI elements may not be visible properly to
other users who do not have access to the project used in a shared job.

12.1. Creating a Group


To create a group:
1. Click on the user panel on the top-right corner of the web application and select GROUP MANAGEMENT. This
displays all the groups available for the user.

Figure 12.1. Group Management Option

2. Click on Add Group to create the group.

Figure 12.2. Group List

3. Enter the group name and description. Select the users, depending on the requirement, from the drop-down
menu.
4. Click on Save to create the group.

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

66 FPGA-AN-02039-1.2
Lattice sensAI Studio User Guide
Application Note

Figure 12.3. Create a Group

12.2. Adding and Removing Group Members


After creating a group, you can add or remove users when necessary.
To add or remove a user in a group:
1. Click on the user panel on the top-right corner of the web application and select GROUP MANAGEMENT.
2. Click on the Edit icon of the group you want to modify, as shown in Figure 12.4.

Figure 12.4. Update Group Option

3. Select the users to add or remove, and click on Update as shown in Figure 12.5.

Figure 12.5. Adding and Removing Group Members

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-AN-02039-1.2 67
Lattice sensAI Studio User Guide
Application Note

12.3. Deleting a Group


To delete a group:
1. Click on the user panel on the top-right corner of the web application and select GROUP MANAGEMENT.
2. Click on the Delete icon of the group you want to remove.

Figure 12.6. Delete Group Option

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

68 FPGA-AN-02039-1.2
Lattice sensAI Studio User Guide
Application Note

13. Memory Management


The Memory Management feature allows the periodic cleanup of past projects, datasets, and jobs. Based on
configuration, older data gets deleted to free up space on the deployment server. This feature is configurable by the
admin only and is very flexible. Data older than the configurable period is cleaned up automatically. This period can be
configured globally or per user. It is also possible to keep selected items (project, dataset, and job) continually.
Note: This feature is currently in beta phase for this release.

13.1. Configuring Memory Management


To configure memory management:
1. Click on the user panel on the top-right corner of the web application and select SETTINGS. This opens the
Memory Management Configuration window as shown in Figure 13.2.

Figure 13.1. Memory Management Option

Figure 13.2. Memory Management Global Configuration

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-AN-02039-1.2 69
Lattice sensAI Studio User Guide
Application Note

2. To enable or disable the memory management feature, toggle Global Setting.


3. To enable or disable the memory management for project, dataset, and jobs respectively, toggle Project, and/or
Dataset, and/or Job switches.
4. Select Duration period on how long you want to keep the items. Selecting Forever does not delete anything.
5. Check the Run cleanup on save box if you want to run the cleaner upon saving. Click on Save. By default, the
cleanup scheduler runs once a day. Checking Force settings for all users remove user specific settings if already
configured and applies global configuration for all users.
6. Click on Save. This runs the cleaner scheduler if Run cleanup on save is checked.

13.2. Memory Management User Setting


You can select a different period for individual users through the User Setting feature.
To set a different period:
1. Select the user from the drop-down menu. Click on Add to add user to the configuration list.

Figure 13.3. Memory Management User Configuration

2. Select duration and click on the icon to save the configuration. The icon switches to Edit as shown in
Figure 13.4.

Figure 13.4. Memory Management Global Configuration Update

Once configuration is completed, the admin can remove the configuration by clicking on the Delete icon or modify it by
clicking on the Edit icon.

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

70 FPGA-AN-02039-1.2
Lattice sensAI Studio User Guide
Application Note

14. AWS NFS Setup Guide


To setup NFS in AWS cloud, follow the links below:
1. Create a S3 bucket on AWS.
• The user needs to create a S3 bucket to be used as storage for NFS (The user can also use volume/tape
storage).
2. Create a S3 file Storage Gateway on AWS.
• The user needs to create a relevant file gateway based on the above step for NFS creation.
• Select m5.xlarge ec2 instance at least if using ec2 as compute option for Gateway. Also, allow access for ports
2049 and 22 while creating ec2 instance.
3. Create an NFS file share on AWS.
• The user needs to create an NFS file share.
• To access this NFS, the user needs to set the IP address of the ec2 instance and not file share.

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-AN-02039-1.2 71
Lattice sensAI Studio User Guide
Application Note

15. AWS Kubernetes (EKS) Cluster Creation


• Create an Amazon EKS cluster.
• After the cluster creation, set up the service account and roles for the pod access from sensAI Studio. Connect to
master node and run the following commands:
• Create a file roles.yaml as given below. Change username, namespace, and other aspects as per requirement,
same needs to be configured in Create an NFS file share on AWS.

Figure 15.1. Kubernetes Service Account, Roles and Namespace

• Run kubectl create -f roles.yaml.


• This creates a role, namespace and user.

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

72 FPGA-AN-02039-1.2
Lattice sensAI Studio User Guide
Application Note

Figure 15.2. Kubernetes Service Account, Roles and Namespace Creation

• Get the api token by running these commands.


• SERVICEACCOUNT=<user-name>
• NAMESPACE=<name-space>
• APISERVER=$(kubectl config view --minify -o jsonpath='{.clusters[0].cluster.server}')
• SECRET_NAME=$(kubectl get serviceaccount -n $NAMESPACE $SERVICEACCOUNT -o
jsonpath='{.secrets[0].name}')
• TOKEN=$(kubectl get secret -n $NAMESPACE $SECRET_NAME -o jsonpath='{.data.token}' | base64 --
decode)
• curl $APISERVER/api/v1/namespaces/$NAMESPACE/pods --header "Authorization: Bearer $TOKEN" -
insecure
• echo $TOKEN
• Copy the $TOKEN that is used for authentication.

Figure 15.3. API Token

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-AN-02039-1.2 73
Lattice sensAI Studio User Guide
Application Note

16. Troubleshooting

16.1. Database Connection Failed during Boot


The user might see the messages below during boot for fresh setup. This is expected, as database initialization takes
time. This error should go away after some time.

Figure 16.1. Boot Error during Database Setup

16.2. Dataset Validation Fails


Dataset validation fails if the uploaded dataset has a wrong folder structure. The user should upload the correct folder
structure based on dataset type. The user can click on the Tool Tip icon (Figure 16.2) to get information on required
structure.

Figure 16.2. Dataset Structure Tool Tip

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

74 FPGA-AN-02039-1.2
Lattice sensAI Studio User Guide
Application Note

Figure 16.3 shows a Kitti example.

Figure 16.3. Dataset Structure Example – Kitti

Note: The user needs to make sure there is no extra folder when creating zip. In the above example, the user needs to
create zip from ImageSets and the training folder (not from the KITTI_Dataset folder). So when extracted, it contains
ImageSets and training in main dataset folder.

16.3. Not Able to Open Application Web Page


• Make sure IP address in ltf.config is correct.
• It should be IP address of system in which application is deployed.
• Make sure not to use localhost or local IP (for example, 127.x.x.x) as IP address.

16.4. Model Zoo Projects or Workspaces Are Not Available


Model Zoo projects and workspaces are listed based json maintained on GitHub. Make sure your system allows
http/https file downloads.
You can run the command below to make sure it is accessible:
# wget https://raw.githubusercontent.com/LatticeSemi/Model_Info/dev/model_info.json
If you are unable to download it, contact your IT team to allow required access.

16.5. Not Able to See Camera for Stream Dataset


If sensAI studio is running in http mode, some browsers may not allow the access of microphone/camera. If the site is
running on https, this issue should not be observed.
There might be workaround provided by browser to allow access of microphone/camera over non-secure connection
as well. For example, to ignore Chrome secure origin policy, follow these steps.

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-AN-02039-1.2 75
Lattice sensAI Studio User Guide
Application Note

• Navigate to `chrome://flags/#unsafely-treat-insecure-origin-as-secure` in Chrome.


• Find and enable the `Insecure origins treated as secure` section (see below).
• Add any address you want to ignore the secure origin policy for. Remember to include the port number too (if
required).
• Save and restart Chrome.

Figure 16.4. Chrome Treat Origin as Secure

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

76 FPGA-AN-02039-1.2
Lattice sensAI Studio User Guide
Application Note

Technical Support Assistance


Submit a technical support case through www.latticesemi.com/techsupport.
For frequently asked questions, refer to the Lattice Answer Database at
www.latticesemi.com/en/Support/AnswerDatabase.

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

FPGA-AN-02039-1.2 77
Lattice sensAI Studio User Guide
Application Note

Revision History
Revision 1.2, June 2023
Section Change Summary
Introduction Updated the description of the hardware component required by the software in the
Hardware Requirement section.

Revision 1.1, June 2022


Section Change Summary
All Updated for latest release.

Revision 1.0, May 2021


Section Change Summary
All Initial release.

© 2022-2023 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal.
All other brand or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.

78 FPGA-AN-02039-1.2
www.latticesemi.com

You might also like