W11 Ecs7020p
W11 Ecs7020p
7 Dec 2023
Matched and Wiener filtering
2/35
Data never beats what is known!
3/35
Agenda
Deployment options
ML project management
4/35
Building and deployment
Hypothesis
Learner
Data
Model (knowledge)
Deployment
New data (production, Prediction/Action
inference)
5/35
Technological products: Non-ML
https://www.ifixit.com/Teardown/iPhone+12+and+12+Pro+
Teardown/137669
6/35
Technological products: ML
7/35
Technological products: ML
https://doi.org/10.13026/9cft- hg92
8/35
Lifecycle of a typical product
Develop and
tes
t
n
sig
De
Deploy
Plan
in
a
Mo i nt
n it o r Ma
9/35
Lifecycle of a machine learning model
Build and
tes
a t
at
,d
gn
Desi
Deploy
Plan
in
a
Mo i nt
n it o r Ma
10/35
Machine learning systems as computing products
11/35
Lifecycle of a machine learning model
Build and
tes
a t
at
,d
gn
Desi
o ---- r d
Deploy
ha
a
M ore d at
rd
ha
Plan
o
To
M or To
e data
in
a
Mo i nt
n it o r Ma
12/35
The data flywheel
13/35
Agenda
Deployment options
ML project management
14/35
Machine learning modelling in context
The role of the deployment stage depends on the specific context of the
machine learning project. Deployment can be a goal, but also seen as
one component of the development strategy.
15/35
Pipelines, not models
16/35
The notion of environment
17/35
Deploying in data science environments
Models can be deployed in the data science environment where they have
been built. In this case, the data science environment provides both the
building and the deployment runtime. Typical scenarios include:
Research.
Analytics.
18/35
Deploying in other environments
19/35
Executables
20/35
Packaging
Packaging can be used for deployment purposes but also with a focus on
reproducibility and collaboration. Packaging options include:
Models as dependencies (e.g. Python modules).
Models and other dependencies in the environment (e.g. conda).
Containers (e.g. Docker) packaging code, runtime, dependencies as
as images (not pictures!).
Packaging the environment (conda and docker) ensures that the model
and its environment are replicated exactly when deployed.
21/35
Cloud deployments
22/35
Machine learning systems: More than machine learning
https://papers.nips.cc/paper/2015/file/86df7dcfd896fcaf2674f757a2463eba- Paper.pdf
23/35
Agenda
Deployment options
ML project management
24/35
Machine learning systems
Machine learning systems are more than a machine learning model, they
consists of infrastructure (hardware, networking, OS), application
software, data and other components.
25/35
Machine learning system specs
26/35
Accuracy vs speed
27/35
Accuracy vs speed
28/35
Memory footprint
29/35
Test in production
30/35
Agenda
Deployment options
ML project management
31/35
Machine learning deployment time: A survey
Many tech companies deploy thousands of new models every day. Others
report that deploying one single model takes them:
32/35
Managing machine learning projects
33/35
Versioning and tracking
34/35
MLOps: End-to-end machine learning management
35/35