Deepdive on Amazon SageMaker

and AWS re:Invent New Features

Belton Kwong, Solutions Architect

Amazon Web Services

© 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
 Our Mission at AWS

Put AI/ML into the hands

of every IT professional

© 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
 The AWS AI/ML Stack
Broadest and most complete set of AI/ML capabilities
AI SERVICES HEALTH AI INDUSTRIAL AI ANOMALY DETECTION CODE AND DEVOPS

NEW NEW NEW NEW NEW NEW NEW

Amazon Amazon
Amazon Transcribe Comprehend AWS Panorama Amazon Amazon Lookout Amazon Lookout Amazon Lookout Amazon Amazon
HealthLake Medical Medical + Appliance Monitron for Equipment for Vision for Metrics DevOps Guru CodeGuru

VISION SPEECH TEXT SEARCH CHATBOTS PERSONALIZATION FORECASTING FRAUD CONTACT CENTERS

Contact Lens
Amazon Amazon Amazon Amazon Amazon Amazon Amazon Amazon Amazon Amazon Amazon
Rekognition Polly Transcribe Comprehend Translate Textract Kendra Lex Personalize Forecast Fraud Detector Voice ID
+Medical +Medical For Amazon Connect

ML SERVICES
SAGEMAKER STUDIO IDE

NEW NEW NEW NEW

Label Aggregate & Store & share Detect Visualize in Pick Train Tune Debug & Deploy in Manage NEW Human
Auto ML Spark/R CI/CD
Amazon data prepare data features bias notebooks algorithm models parameters profile production & monitor review
SageMaker
NEW: SageMaker JumpStart

NEW: Model management for edge devices

FRAMEWORKS & INFRASTRUCTURE

Deep
Learning GPUs & Elastic
Trainium Inferentia Greengrass
AMIs & CPUs Inference
RL C o a ch DeepGraphLibrary
Containers

© 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
 State of AI / ML

INCREASED FROM PILOTING TO ML HANDLES REAL- INTEGRATING ML

SPENDING OPERATIONALIZING WORLD TASKS INTO DEVOPS

By 2023, spending on AI By the end of 2024, 75% of Driven by advancements in GPUs and ML is now part of mainstream
systems will reach $97.9B, enterprises will shift from compute, availability of data, new DevOps process, not a set of
up 2.5x from $37.5B in 2019 piloting to operationalizing AI algorithms and the cloud specialized, isolated projects
— IDC — Gartner — Gartner

© 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
 Machine learning development can be complex and costly

Label Collect and Store Check Visualize in

data prepare data features for bias notebooks

Pick Train Tune Deploy in Manage CI/CD

algorithm models parameters production and monitor

© 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
 Amazon SageMaker: Built to make ML more accessible
SageMaker Studio IDE

Label Collect and Store Check Visualize in

data prepare data features for bias notebooks

Pick Train Tune Deploy in Manage CI/CD

algorithm models parameters production and monitor
MODEL MANAGEMENT FOR EDGE DEVICES

© 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
 Amazon SageMaker Overview
Amazon SageMaker
PREPARE BUILD TRAIN & TUNE DEPLOY & MANAGE

SageMaker Ground Truth SageMaker Studio Notebooks Managed Training Managed Deployment
Label training data for machine learning Jupyter notebooks with elastic compute Distributed infrastructure Fully managed, ultra low latency,
and sharing management high throughput
SageMaker Data Wrangler NEW
Aggregate and prepare data for Built-in and Bring SageMaker Experiments Kubernetes & Kubeflow
machine learning your-own Algorithms Capture, organize, and compare Integration
Dozens of optimized algorithms or bring every step Simplify Kubernetes-based
SageMaker Processing your own machine learning
Built-in Python, BYO R/Spark Automatic
Local Mode Model Tuning Multi-Model Endpoints
SageMaker Feature Store NEW Test and prototype on your local machine Hyperparameter optimization Reduce cost by hosting multiple models
Store, update, retrieve, and share features per instance
SageMaker Autopilot Distributed Training
SageMaker Clarify NEW Automatically create machine learning Libraries NEW SageMaker Model Monitor
Detect bias and understand models with full visibility Training for large datasets Maintain accuracy of deployed models
model predictions and models
SageMaker JumpStart NEW SageMaker Edge Manager NEW
Pre-built solutions for common use cases SageMaker Debugger NEW Manage and monitor models on
Debug and profile training runs edge devices

Managed Spot Training SageMaker Pipelines NEW

Reduce training cost by 90% Workflow orchestration and automation

SageMaker Studio
Integrated development environment (IDE) for ML

© 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
 Amazon SageMaker Studio
Fully Integrated Development Environment (IDE)
for machine learning

© 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
 Amazon SageMaker Studio
Fully Integrated Development Environment (IDE) for machine learning

Collaboration Easy Automatic Higher quality Increased

at scale experiment model ML models productivity
Share notebooks
management generation Automatically debug Code, build, train,
without tracking errors, monitor deploy, and monitor
Organize, track, and Get accurate models
code dependencies models, and maintain in a unified
compare thousands with full visibility and
high quality visual interface
of experiments control without
writing code

© 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
 Amazon SageMaker JumpStart
Easily and quickly bring machine learning
applications to market

© 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
 15+ pre-built solutions for common ML use cases
Solutions can be used out-of-the-box or can be customized for a specific
business problem

SageMaker
JumpStart Accelerate time to deploy over 150 open
source models
Use one-click deployable ML models and algorithms from popular
Easily and quickly bring model zoos

machine learning
applications to market Get started with just a few clicks
Easily bring ML applications to market using pre-built solutions, ML models,
and algorithms from popular model zoos, and getting started content

© 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
© 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
 Amazon SageMaker
Data Wrangler
Fastest way to prepare data

© 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
 80% of time spent on data prep

What data scientists spend the most time doing

4% 3%
5%
Cleaning and organizing data
9%
Collecting data sets

Mining data for patterns

Other

Refining algorithms
19% 60%
Building training sets

Source: Forbes survey of 80 data scientists, March 2016

© 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
 SageMaker Data Wrangler is the fastest way to prepare data

Amazon SageMaker Select & Query Cleanse & Enrich Visualize Understand Operationalize
Data Wrangler Select and query Cleanse and explore data, Graphically understand Use a sample dataset Export data preparation
A faster, visual way data from a variety of perform feature data and detect outliers to quickly estimate workflow to a notebook or
to aggregate and prepare data sources such as engineering with built-in with pre-configured model performance, code script to bring the
data for machine learning Amazon Athena, Amazon data transforms, and visualization templates accuracy, and diagnose workflow into production
Redshift, detect statistical bias potential issues
AWS Lake Formation, and with SageMaker Clarify
Amazon S3

Integrate data Export prepared

Import data from a preparation data into Amazon
feature store such as workflow with SageMaker
Amazon SageMaker Amazon Feature Store
Feature Store SageMaker
Pipelines

© 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
© 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
 Amazon SageMaker
Feature Store
Store, discover, and share features
for machine learning

© 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
 Feature drift

Challenges
of separate Feature duplication
feature stores

Slow model
development/deployment

© 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
 Amazon SageMaker Feature Store
Store, discover, and share features for machine learning

Real time
inference
Online
Streaming feature
store
Batch
inference
Offline
Batch feature
Amazon SageMaker store
Feature Store Model
training
Store, discover, and share
features for machine
Raw data Feature processing learning Ingest data Store Serve
Data in its original Transform raw data into Move streaming features Online and offline feature Features for real-time and
form that has not meaningful features for or batch features to a central stores maintaining batch applications, and for
been processed better models repository consistency and accuracy model training

© 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
© 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
© 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
 Machine Learning at Scale

© 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
 Machine Learning at Scale
Balance between ML agility and IT governance can be achieved with SageMaker

ML Builders Cloud IT
and Business Stakeholders and DevOps

Innovate with the speed and agility of AWS Govern and enable with central controls
Self-service access Security
Experiment fast Compliance
Respond quickly to change Operations
Spend management

© 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
 ML is an Iterative and Collaborative Process

Feature generation Prototyping Model Model management

and engineering and experimentation training and deployment

Automation of
ML production model deployment
“recipe” via CI/CD
=
data flow
pipeline

Data scientists / ML engineers DevOps / MLOps engineers

© 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
 Security
Security features to help you meet strict security

Amazon requirements of ML workloads

SageMaker
Compliance
PCI, HIPAA, SOC 1/2/3, FedRAMP, and ISO
9001/27001/27017/27018

is DevOps ready
ML workflows
Create automated workflows in minutes to support
thousands of models

Scalability
Train complex models with massive datasets

Orchestration
Automatic scheduling and execution of jobs with managed infrastructure

© 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
 Amazon SageMaker Pipelines
Managed machine learning CI/CD service

© 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
 Amazon SageMaker Pipelines
Managed machine learning CI/CD service

Centrally manage Share and Choose from Compare

each step of re-run built-in workflows
the workflow workflows templates visually

© 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
 See pipeline execution details and metrics in real-time

Follow completed steps and

monitor steps in progress

Understand the output

from each step with the output
logs

Monitor, change, and manage the

parameters
for each step

© 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
 Approve models for production

© 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
 Why customers choose Amazon SageMaker

EASE-OF-USE SCALE AND PERFORMANCE REDUCE COSTS

SINGLE IDE Lower TCO

Perform all ML steps in Better GPU
a web-based interface utilization and efficiency
Cost reduction for
data labeling using
ONE-CLICK Performance increases Ground Truth
model training and deployment from model optimization
with Neo
Cost reduction for inference
TRAIN ONCE with
run anywhere Elastic Inference and Inf1
SECURITY AND COMPLIANCE
DEVOPS READY Cost reduction
with SageMaker Pipelines SOC, PCI/DSS, ISO, HIPAA, C5, with
and Kubeflow integration support OSPAR, HITRUST CSF, GDPR, FIPS Managed Spot Training

© 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
 Thank You!

Belton Kwong
Solutions Architect
AWS

© 2021, Amazon Web Services, Inc. or its Affiliates. All rights reserved.