SER3052BU

How VMware vSphere

t i o n
and NVIDIA GPUs i s tr ibu
or d
Accelerate Your t ion
bli c a
p u
Organization o t fo r
nt: N
o n te
17 C
2 0
w orld
V M
Raj Rao, NVIDIA GRID Product Management
Ziv Kalmanovich, vSphere ESXi Product Management

#VMworld #SER3052BU
Disclaimer
• This presentation may contain product features that are currently under development.
• This overview of new technology represents no commitment from VMware to deliver these
features in any generally available product. t i o n
tribu
r dis purchase orders, or
• Features are subject to change, and must not be included in contracts,
o
sales agreements of any kind.
a t i on
c u b li
• Technical feasibility and market demand will affect o p
r final delivery.
o t f
• n
Pricing and packaging for any new technologiest : N or features discussed or presented have not
ont e
been determined.
17 C
2 0
o r ld
VMw

CONFIDENTIAL 2
Aligning To Your Strategic Priorities

Your Strategic VMware’s Vision

Your Strategic IT Priorities
IT Priorities
t i o n
Any Device
F

i s tr ibu
Modernize Integrate Empower
o d
r Transform
Transform VMware Workspace ONE™
o n
Desktop Mobile Identity

cati
Security Data Public Digital
li
Security
Centers Clouds Workspaces
p u b
Any Application
t f or
Empower
:N o
Traditional Apps Cloud-Native Apps SaaS Apps
Digital
t e n t
Workspaces
C on VMware Cross-Cloud Architecture™
1 7
20
Private Cloud Hybrid Cloud Public Cloud
Integrate
o r ld VMware Cross-Cloud Services™

VMw
Public
Clouds
Any Cloud
VMware vRealize® Cloud Management
Modernize
Data VMware Cloud Foundation™
Centers
Software-Defined Data Center VMware Cloud Provider Partners

3
Universal App Platform

Test / Dev / Business-Critical Desktop 3D Deep Learning Big SAP Cloud-Native

Tier 2/3 Apps Virtualization Graphics with GPU Data HANA Applications

t i o n
i s tr ibu
or d
t ion
bli c a
r p u
o t fo
nt: N
o n te vSphere

1 7 C Integrated
2 0 Containers

w o rld
V M

4
Virtualizing HPC, Big Data and ML Workloads with GPUs

High Performance AI and Big Data &

Traditional Enterprise Applications Computing Deep Learning Analytics

t i o n
i s tr ibu
or d
t ion
bli c a
r p u Application Compatibility
t f o
:N o
t e n t Near Native Performance
C o n
Efficiency017 Efficiency
r l d 2
w o
VM Agility Agility

Resiliency Resiliency

Security Security

CONFIDENTIAL 5
GPU Compute on vSphere with DirectPath IO – Benefits

VM level QoS
t i o n
Workload Isolation
i s tr ibu
or d
t io n
a
VM

ubli c
r p
o t fo
GPU GPU

nt: N GPU GPU

te
GPU GPU

o n
C Reproducibility
GPU GPU

0 1 7
rld 2
Mw o
vSphere V
vSphere

GPU GPU GPU GPU GPU GPU GPU GPU

Near Bare Metal Performance HW Isolation

6
CONFIDENTIAL
GPU Compute on vSphere with DirectPath IO – Machine Learning
1:1 X:1

t i o n
ibu
CUDA CUDA Data Data
Developer Developer Scientist Scientist
i s tr
or d
t io n
a
VM

ubli c
r p
o t fo
GPU GPU

nt: N GPU GPU

te
GPU GPU

C o n GPU GPU

0 1 7
rld 2
Mw o
vSphere V
vSphere

GPU GPU GPU GPU GPU GPU GPU GPU

7
CONFIDENTIAL
GPU Compute on vSphere with DirectPath IO – HPC and Big Data

t i o n
i s tr ibu
or d Researchers

t io n
a
VM
Worker Worker Worker Worker
ubli c Worker

r p
o t fo
GPU GPU

nt: N Worker Master GPU GPU

onte
GPU GPU

node

17 C
GPU GPU

d 2 0
orl
vSphere VMw vSphere

GPU GPU GPU GPU GPU GPU GPU GPU

8
CONFIDENTIAL
GPU Compute on vSphere with DirectPath IO – Benefits

VM level QoS
t i o n
Workload Isolation
i s tr ibu
or d
t io n
a
VM

ubli c
r p What’s Missing?
o t fo
GPU GPU

nt: N GPU GPU

GPU sharing
te
GPU GPU

o n
C Reproducibility GPU acc VM Resiliency
GPU GPU

0 1 7
rld 2 GPU QoS
Mw o
vSphere V GPU Resource Scheduling
vSphere

GPU GPU GPU GPU GPU GPU GPU GPU

Near Bare Metal Performance HW Isolation

9
CONFIDENTIAL
 t i o n
i s tr ibu
or d
n your DataCenter
tioto
Why Compute Workloads important
b l i c a
pu
AGENDA : o t f or
What are the keyNrequirements that Compute Workloads bring
t e n t
C on
2 0
How 17does NVIDIA GPU Virtualization enable you to host these Workloads
w orld
V M

10
 t i o n
i s tr ibu
or d
t ion
WHY IS SUPPORT FOR COMPUTE bli
r p
cu
a
WORKLOADS
t fo
IMPORTANT
nte n t : N o
C o
0 1 7
l d 2
or
VMw

11
 GPU ACCELRATED APPLICATIONS
BROAD RANGE OF INDUSTRIES TRANSFORMED
Computational Structural Medical Imaging Weather and Climate
Visual Computing
Mechanics
t i o n
i s tr ibu
or d
t ion
bli c a
r p u
Electric Design Automation fo
Computational Fluid Dynamics
N o t Computational Finance Numerical Analytics

n t :
o n te
1 7 C
r ld 20
Mw o
V
Computational Chemistry Data Science Defense Machine Learning

For a complete list go to: http://www.nvidia.com/object/gpu-applications.html

12
 THE EVOLUTION OF MODERN WORKFLOWS
VISUAL LARGE DATA
t i o n
ibu
MOBILITY COLLABORATION VR PHOTOREALISM AI
WORKSPACE INTERACTIVE
i s tr
HPC
or d
t ion
bli c a
r p u
o t fo
nt: N
o n te
17 C
2 0
w o rld
V M

Information Workers/Students VISUAL COMPUTING SPECTRUM Designers/Scientists

13
 t i o n
i s tr ibu
or d
t ion
WHAT ARE THE KEY oREQUIREMENTS
bli
r
c
pu TO
a
t f
CONSIDER
nte n t : N o
C o
0 1 7
l d 2
or
VMw

14
 VIRTUALIZING MIXED WORKLOADS
Requirements
Performance
t i o n
i s tr ibu
or d
t i
Guaranteed
a on QoS
Virtual Machine Virtual Machine Virtual Machine
u ic
bl
Guest OS Guest OS Guest OS
or p
Apps Apps Apps
o t f
NVIDIA Driver
n t : N
te
NVIDIA Driver NVIDIA Driver

C o n Insight
d 2 017
or l
vGPU VMw vGPU vGPU
Hypervisor NVIDIA vGPU manager Fully Accelerate every Application

CPUs NVIDIA
Server GPU
15
 t i o n
i s tr ibu
or d
t ion
bli c a
u
GRID AUGUST
: N o t 2017
fo r p
RELEASE
t e n t
C on
1 7
r ld 20
Mw o
V

16
 t i o n
ibu tr
NVIDIA VIRTUAL PASCAL HW SUPPORT
o n o r d i s
END TO END MANAGEMENT
t i
GPU SW - or p u b l i c a
t f
AUGUST 2017 ntent : No
C o
RELEASErld 201 7
Mw o
V

GPU SCHEDULER ADVANCEMENTS COMPUTE IN ALL vDWS

PROFILES
17
 UNDERSTANDING HARDWARE SPECS
Planning for Performance

Video Memory Graphics t i o n

Compute
Performance i s tr ibuPerformance
or d
t ion
bli c a
r p u
o t fo
nt: N
o n te
17 C
2 0
w o rld
V M 3DMark 11 - DX
Frame Buffer Size Peak TFLOPS – DP/SP
SPECviewperf 12 - OGL
Scalability & Flexibility Memory Bandwidth
Decoding/Encoding

18
 GRID PERFORMANCE OPTIMIZED
TESLA M60 TESLA P40
GPUs Dual GM204 Single GP102

CUDA Cores 4,096 (2,048 per GPU) 3,840

Memory Size 16 GB GDDR5 (8 GB per GPU)

t i o nGB GDDR5
24

Form Factor PCIe 3.0 Dual Slot

i s tr ibu PCIe 3.0 Dual Slot
or d
on
Thermal passive / active passive
a t i
Power
u b lic
300W / 240W 250W

r p
o t fo
Max Concurrent Users
n t : N 32 (0.5GB FB) 24 (1GB FB)

ont e 0Q, 1Q, 2Q, 4Q, 8Q 1Q, 2Q, 3Q, 4Q, 6Q, 8Q, 12Q, 24Q

1 C
Profile Options
7 0B, 1B 1B

r ld 20 0A, 1A, 2A, 4A, 8A 1A, 2A, 3A, 4A, 6A, 8A, 12A, 24A

Mw o H.264 1080p30 Streams 36 24*

V
3DMark 11 13,732 25,000*

~2x
for
SPECviewperf 12 62 110*
Virtual Data Center
SGEMM TFLOPS 2x 3.8 10.6
Workstations
Memory Bandwidth 2x 160 GB/s 347 GB/s
* estimate
19
 STANDARD SCHEDULER
BEST EFFORT SCHEDULING
6 4 2 1
VM 1 Timesliced Round i o n Robin
Scheduler i b u t
is t r
o r d
on
5 7 6 5 4 3 2 1
8 7 Round 8
VM 2
c a t i Tasks generally execute
ubli
Robin
Scheduler o r p within a timeslice
GPU
o t fEngine
nt: N
3
VM 3
n te Best Effort Scheduling
C o
0 1 7
rld 2
Mw o
SHARE OF GPU CYCLES
V
VM3

VM1
VM2

20
 SCHEDULING LONG RUNNING TASKS
ROOT-CAUSE FOR QoS ISSUES
1
VM 1 Compute Tasks i o n can be long
running i b u t
is t r
o r d
on
1
8 6 4 2 Round
VM 2
c a t i Round Robin Scheduler fails
ubli
Robin
Scheduler o r p when a single task does not
GPU
o t fEngine complete within a reasonable
nt: N
7 5 3
VM 3
o n te time
C
17
2 0
r ld OF GPU CYCLES
SHARE
o Starves other VMs
V Mw
Injects the “noisy neighbor”
symptom

VM1

21
 INTRODUCING : EQUAL SHARE SCHEDULER
GUARENTEE DETERMISINSTIC QoS
1 New Advanced Scheduling
VM 1
ion Share
modeib:utEqual
Equal i s tr
Scheduler
d (Available on
8 6 4 2 Share 8 1 7 6 1 5 4 1 3 2 1 o r
n Pascal HW)
VM 2 a t i o
Round
u b lic
Robin o r p
o t
GPUf Engine Long Running Tasks are pre-
7 5 3 Scheduler
n t : N empted and context saved
VM 3
on t e
7 C to be resumed when
2 0 1 rescheduled
r l d
o SHARE OF GPU
V Mw
Deterministic share of GPU
VM3 VM1
cycles per VM
All running vGPU enabled
VM2 VMs get equal share of GPU
cycles 22
 END-TO-END MANAGEMENT
Taking GPU visibility to a new level with application monitoring

t i o n
i s tr ibu
or d
t ion
bli c a
r p u
o t fo End user experience

n t: N Session monitoring monitoring

on t e
1 7 C
r ld 20
Mw o
Guest monitoring V Performance
troubleshooting
Accurate sizing
Host monitoring

New

App monitoring

23
 CUDA enabled app

GPU ACCELERATED APPLICATIONS

t i o n
- CUDA 9.0 i s tr ibu
or d
a t i on Quadro Virtual
- OCLb2.0
u lic
o r p Data Center
o t f Workstation
t : N - Quadro Value-Add
e n
C ont
1 7 - Vulkan 1.0
r ld 20
Mw o
V - Shader Model 5.0
GRID Virtual
- OGL 4.5 PC

- DX 9, 10, 11, 12

24
NVIDIA GRID GPU VIRTUALIZATION PLATFORM
Industry standard virtualization platform

Quadro virtual
t i o n
vPC Data Center Rendering CUDA Compute
HPC u
Support
ib tr AI
d i s
(Workstation Apps, Rendering, HPC, DL, AI)
r
Workstation
on o
c a t i
u b li
vGPU Monitoring, Insight
o p
r and Management
o t f
n
Data Centert N
: and/or Cloud Accessible
t e
17 Con
2 0 Hypervisor

w o rld
V M NVIDIA Virtualization Software

NVIDIA Tesla GPU

M60, M6, M10 (graphics/sharing only) P40, P6, P100, P4

25
 WRAP UP

t i o n
r ibu
dist
- GPU Accelerated Apps is transforming a Broad Range of Industries
o r
t io n
- Virtualizing Mixed Workloads in a multi-tenantbenvironment
li c a requires
r p u
t f o
- Performance : N o
te n t
o n
- Deterministic QoS 17 C
d 2 0
- Insight Mwo
rl
V
- Full Acceleration for every Application

- NVIDIA GPU Virtualization Platform delivers key requirements to host mixed

workloads
26
 Roadmap

Vision – Extend all vSphere Benefits to NVIDIA GRID™ vGPU

t i o n
i s t r ibu
or d
t io
vGPU
vGPU n
bli c a vGPU
vGPU

r p u
o t fo
nt: N
Suspend
o n te
&Resume
Snapshots vMotion
1 7 C DRS
2 0 See @booths

vSphere w o rld
V M vSphere Cloud
Platform - New
Workloads
Shared Resources NVIDIA GRIDtm
EUC 3D Experience
CPU Mem GPU GPU NVIDIA GRID

The information in this presentation is intended to outline our general product direction and should not be relied on in making a purchasing decision. It 27
is for informational purposes only and may not be incorporated into any contract.
Considered Milestones for VMware vSphere with NVIDIA GRID
Virtual High Performance
Virtual PC Machine Learning
Workstation Computing
t i o n
i s t r ibu
or d
t io n
bli c a
r p u
o t fo
nt: N
Suspend&Resume nte
Snapshots
CoRoadmap
vSphere vMotion vSphere DRS
0 1 7
Tech Preview

rld 2 Roadmap Roadmap

M w o
V
See @booths See @booths

VMW Cloud Platform - VMW Cloud Platform -

New Workloads New Workloads

VMW EUC 3D VMW EUC 3D

Experience Experience

NVIDIA GRID NVIDIA GRID

The information in this presentation is intended to outline our general product direction and should not be relied on in making a purchasing decision. It 28
is for informational purposes only and may not be incorporated into any contract.
 Roadmap

See @booths

vSphere with NVIDIA GRID – Simplified Maintenance VMW Cloud Platform -

New Workloads

VMW EUC 3D
Experience

tion
NVIDIA GRID

t r i b u
o r dis
on
CUDA CUDA Data Data CUDA CUDA Data Data
Developer Developer Scientist Scientist Developer Developer Scientist Scientist

c a t i
VM VM
u b li
or p
vGPU vGPU

o t f vGPU

:N
vGPU vGPU vGPU vGPU vGPU
vGPU

t
vGPU

t e n
C on
1 7
r ld 20
Mw o
V
VMware vSphere NVIDIA GRID VMware vSphere NVIDIA GRID
Remediate
GPU GPU GPU GPU GPU GPU GPU GPU

The information in this presentation is intended to outline our general product direction and should not be relied on in making a purchasing decision. It 29
is for informational purposes only and may not be incorporated into any contract.
 Tech Preview

vSphere with NVIDIA GRID – 24h Utilization

CUDA Data Data
CUDA Scientist Scientist
Developer Developer
VM

t i o n
vGPU vGPU vGPU vGPU

i s t r ibu
Develop/VDI/
Inference by day or d
Data Data
a t i on
licnight
CUDA vGPU vGPU
CUDA
ML Trainingbby
Developer Scientist Scientist

u
Developer
VM
o r p
N o tf
vGPU vGPU vGPU vGPU

ent :
on t
17 C See @Booths
CUDA CUDA
2
Data

d 0 Data Same Infrastructure

orl
Developer Developer Scientist Scientist vGPU vGPU VMW Cloud Platform-

VMw
VM New Workloads

vGPU vGPU vGPU vGPU

VMW EUC

NVIDIA
VMware vSphere NVIDIA GRID

GPU GPU GPU GPU

The information in this presentation is intended to outline our general product direction and should not be relied on in making a purchasing decision. It is 30
for informational purposes only and may not be incorporated into any contract.
 Roadmap

Vision - Extend vSphere Benefits to NVIDIA GPUs with DirectPath

IO (passthrough) GPU workloads

t i o n
i s t r ibu
r
oGPUd
GPU
t io n
bli c a
r p u
o t fo
nt: N
Suspend
o n te vSphere DRS
&Resume
Snapshots vMotion
1 7 C DRS
HA Placement
2 0
vSphere w o rld
V M
Shared Resources
CPU Mem GPU GPU

The information in this presentation is intended to outline our general product direction and should not be relied on in making a purchasing decision. It is
CONFIDENTIAL 31
for informational purposes only and may not be incorporated into any contract.
Overview - Considered Milestones for all NVIDIA GPU Enablement

Virtual High Performance

Virtual PC (VDI) Machine Learning
Workstation Computing
t i o n
i s t r ibu
or d
t io n
bli c a
r p u
o t fo
n t N
: for
Suspend&Resume Snapshots
e
nt GRID
vSphere vMotion
o
for NVIDIA GRID NVIDIA
vSphere DRS vSphere HA for
DirectPath IO GPU
vSphere DRS for
DirectPath IO GPU
For NVIDIA GRID
17 C for NVIDIA GRID
Tech Preview Roadmap
2 0 Roadmap Roadmap Roadmap Roadmap

o r ld
See @booths V Mw See @booths

VMW Cloud Platform - VMW Cloud Platform -

New Workloads New Workloads

VMW EUC 3D VMW EUC 3D

Experience Experience

NVIDIA GRID NVIDIA GRID

The information in this presentation is intended to outline our general product direction and should not be relied on in making a purchasing decision. It is for 32
informational purposes only and may not be incorporated into any contract.
Introducing vSphere Scale-Out for Big Data and HPC Workloads
New package that provides all the core features required for scale-out workloads at an attractive
price point
t i o n
• Hypervisor, vMotion, vShield Endpoint, i sibu
r
tStorage
o d vMotion,
rI/O Controls & SR-IOV,
Features Storage APIs, Distributed Switch,
t n
ioand more
i c a
Host Profiles / Auto Deploy
bl
p u
t f or
:N o
t e n t
on
Packaging 2017 C • Sold in Packs of 8 CPU at a cost-effective price point
ld o r
VMw

Licensing • EULA enforced for use w/ Big Data/HPC/ML workloads only

33
Value of vSphere Scale-Out for Big Data and HPC

Flexibility & Reduced Data Governance

Operational
andnControl of
Agility Efficiency Complexity u t i o
i s tr ib Sensitive Data
• Infrastructure on • CapEx and OpEx • Simple operations o r d • Host and VM
t n
iothat IT
demand Saving a
using tools
c
bli with
security for your
• Iterate faster • Cluster p
is u
familiar customer data
• Scale out more Consolidation t f or
• Live workload • Security isolation
: N o
rapidly
e n
• Increase Server t mobility for Master • Hypervisor Guests
• Multi-tenancy C ont
Utilization nodes have low privileges
enables different 2 0 17 • Reference by default
multiple distros on orld architecture and
the same setVof Mw best practices
server

Faster time to results and insights at a lower cost

34
Key Takeaways

• NVIDIA GRID and VMware vSphere provide the operational benefits of virtualization with near native
performance (95%) for GPU accelerated HPC, Big Data and Machine Learning
t i o n
ibu i s tr
• VMware's vision is seamless integration of NVIDIA GPU technologies as native o d
r resources of VM
t io n
infrastructure
bli c a
p u
t f or
: N o
• New vSphere Scale-Out SKU; new package twith e n tattractive price point for Big Data/HPC/ML dedicated
C on
17
infrastructure virtualization. http://blogs.vmware.com/vsphere/2017/09/vsphere-scale-now-available.html
2 0
o r ld
VMw
 t i o n
i s tr ibu
or d
t ion
bli c a
u r p
Contact us!
t f o
o
t et: Nand challenges.
We’d like to learn about your use cases
n
Co n
2 0 17
o r
Raj Rao, NVIDIA GRID ld Product Management – [email protected]
VMwvSphere Product Management – [email protected]
Ziv Kalmanovich,
Recommended Additional Resources at VMWorld

Expo Booths GPU Sessions @VMworld

t i o n
ibuistr
VMware Cloud Platform GPU Enabled Linux VDI [VMTN6636U]r
o d
New Workloads t i o n
c
Machine Learning and DeepliLearning
b a on VMware vSphere: GPUs Are
r
Invading the Software-Definedp u Data Center [VIRT1997BU]
o t fo
VMware End User Empowering n t :
the N digital workspace: balancing tomorrow’s trends with
Computing 3D Experience today’s o n te [UEM3332PUS]
needs
1 7 C
d 2 0Wringing Maximum Performance from vSphere for Extremely
w o rl
NVIDIA VM Demanding Workloads and Customers [FUT2020BU]

CONFIDENTIAL 37
 t i o n
i s tr ibu
or d
t ion
bli c a
r p u
o t fo
nt: N
o n te
17 C
2 0
w orld
V M