Virtualization
Introductio
n Providing an interface to
Virtualization:
software that maps to some underlying
system.
A one-to-one mapping between a guest
and the host on which it runs
Virtualized system should be an “efficient,
isolated duplicate” of the real one.
Process virtual machine just supports a process;
system virtual machine supports an entire
system.
Why
Virtualize?
Reasons for Virtualization
Hardware Economy
Versatility
Environment
Specialization
Security
Safe Kernel Development
OS Research
Process
Virtualization
VM interfaces
single process
with
Application sees
“virtual machine” as Application
address space,
Virtualization Layer
registers, and
instruction set .
OS
Examples:
Multiprogramming Hardware
Emulation for binaries
High-level language
VMMs (e.g., JVM)
System
Virtualization
Application
Application
OS
OS
Virtualization Layer
Virtualization Layer
OS
Hardware
Hardware
Classical Hosted
Virtualization Virtualization/
Emulation
System
Virtualization
Interfaces with operating system
OS sees VM as an actual machine—memory,
I/O, CPU, etc
Classic virtualization: virtualization layer
runs atop the hardware.
Usually found on servers (Xen, VMWare ESX)
Hosted or whole-system virtualization:
virtualization runs on an operating system
Popular for desktops (VMWare Workstation,
Virtual PC)
Emulatio
n to a system so that it can
Providing an interface
run on a system with a different interface
Lets compiled binaries, OSes run on
architectures with different ISA (binary
translation)
Performance usually worse than classic
virtualization.
Example: QEMU
Breaks CPU instructions into small ops, coded in C.
C code is compiled into small objects on native ISA.
dyngen utility runs code by dynamically
stitching objects together (dynamic code
generation).
Some Important
Terms
Virtual Machine (VM): An instance of of an
operating system running on a virtualized
system. Also known as a virtual or guest OS.
hypervisor: The underlying virtualization
system sitting between the guest OSes and
the hardware. Also known as a Virtual
Machine Monitor (VMM).
Requirements of a
VMMby Popek & Goldberg in 1974:
Developed
1. Provides environment identical to
underlying hardware.
2. Most of the instructions coming from the guest
OS are executed by the hardware without being
modified by the VMM.
3. Resource management is handled by the
VMM (this all non-CPU hardware such as
memory and peripherals).
Guest OS
Model
Hypervisor exists as a
layer between the
operating systems and
the hardware.
Apps Apps Apps
Performs memory
management and Guest OS Guest OS
scheduling required Guest OS
to coordinate
multiple operating
systems. Hypervisor
(Host)
May also have a
separate controlling Hardware
interface.
Virtualization
Challenges
Privileged Instructions
Handling architecture-imposed instruction
privilege levels.
Performance Requirements
Holding down the cost of VMM activities.
Memory Management
Managing multiple address spaces efficiently.
I/O Virtualization
Handling I/O requests from multiple
operating systems.
CPU Virtualization
x86 architecture has
four privilege levels
(rings).
The OS assumes it will
be executing in Ring 0.
Many system calls
require 0-level
privileges to execute.
Any virtualization
strategy must find a
way to circumvent this.
Image Source: VMWare White Paper, “Understanding Full Virtualization, Paravirtualization, and
Hardware Assist”, 2007.
Binary Translation with Full Virtualization
“Hardware is
functionally identical
to underlying
architecture.”
Typically accomplished
through interpretation
or binary translation.
Advantage: Guest OS
will run without any
changes to source code.
Disadvantage:
Complex, usually
slower than
paravirtualization.
Image Source: VMWare White Paper, “Understanding Full Virtualization, Paravirtualization, and
Hardware Assist”, 2007.
Paravirtualizatio
n
Replace certain
unvirtualized sections
of OS code with
virtualization-friendly
code.
Virtual architecture
“similar but not
identical to the
underlying
architecture.”
Advantages: easier,
lower virtualization
overhead
Disadvantages: requires
Image Source: VMWare White Paper, “Understanding Full Virtualization, Paravirtualization, and
Hardware Assist”, 2007.
modifications to guest
OS
Performanc
e
Modern VMMs based
around trap-and-emulate .
Guest
When a guest OS executes CPU_INST OS
a privileged instruction,
control is passed to VMM
(VMM “traps” on
instruction), which decides TRAP VMM
how to handle instruction
. CPU_INST1
VMM generates
instructions to handle EXEC
trapped instruction
(emulation).
Non-privileged instructions
do not trap (system stays CPU_INST
in guest context).
Trap-and-Emulate
Problems
Trap-and-emulate is expensive
Requires context-switch from guest OS mode to
VMM.
x86 is not trap-friendly
Guest’s CPL privilege level is visible in hardware
registers; cannot change it in a way that the guest
OS cannot detect .
Some instructions are not privileged, but
access privileged systems (page tables, for
example) .
Hardware-Assisted
Virtualization
Hardware virtualization-assist released in 2006 .
Intel, AMD both have technologies of this type.
Introduces new VMX runtime mode.
Two modes: guest (for OS) and root (for VMM).
Each mode has all four CPL privilege levels available .
Switching from guest to VMM does not require changes
in privilege level.
Root mode supports special VMX instructions.
Virtual machine control block contains control flags and
state information for active guest OS.
New CPU instructions for entering and exiting VMM
mode.
Does not support I/O virtualization.
VMWare
FullVirtualization
virtualization implemented through
dynamic binary translation.
Translated code is grouped and stored in
translation caches (TCs).
Callout method replaces traps with stored
emulation functions.
In-TC emulation blocks are even more efficient.
Adaptive binary translation rewrites translated
blocks to minimize PTE traps .
Direct execution of user-space code further
reduces overhead .
Xen
Virtualization
Xen occupies privilege level 0; guest OS
occupies privilege level 1.
OS code is modified so that high-privilege
calls (hypercalls) are made to and trapped by
Xen .
Xen traps guest OS instructions using table
of exception handlers.
Frequently used handlers (e.g., system calls)
have special handlers that allow guest OS to
bypass privilege level 0 .
Approach does not work with page faults.
Handlers are vetted by Xen before being
stored.
Applications of VT-
X Intel VT-x to host fully-virtualized
Xen uses
guests alongside paravirtualized guests .
System has root (VMM) and non-root (guest)
modes, each with privilege levels 0-3.
QEMU/Bochs projects provide emulations
VMWare does not make use of VT technology .
VMWare’s software-based VMMs
significantly outperformed VT-X-based
VMMs .
VT-X virtualization is trap-based, and DBT tries
to eliminate traps wherever possible.
Memory Virtualization
Virtualization software must find a way to handle paging
requests of operating systems, keeping each set of pages
separate.
Memory virtualization must not impose too much overhead,
or performance and scalability will be impaired.
Guest OS must each have an address space, be convinced
that it has access to the entire address space.
SOLUTION: most modern VMMs add an additional layer
of abstraction in address space .
Machine Address—bare hardware address.
Physical Address—VMM abstraction of machine address, used
by guest Oses.
Guest maintains virtual-to-physical page tables.
VMM maintains pmap structure containing physical-to-machine
page mappings.
Memory
virtua
Problem
physica physical
machine
l l
a b b c
fram
e
Page Table Pmap
for Program structure
m on VM n. in VMM.
That’s a lot of
lookups!
Shadow Page
Tables
Shadow page tables map virtual memory to
machine memory .
One page table maintained per guest OS.
TLB(Translation Look aside Buffer) caches results
from shadow page tables.
Shadow page tables must be kept consistent
with guest pages.
VMM updates shadow page tables when
pmap (physical-to-machine) records are
updated.
VMM now has access to virtual
addresses, eliminating two page table
lookups.
Shadow Page
virtua
Tables
physica physical virtua machin
machine
l l l e
a b b c a c
Page Table Pmap Shadow
for Program structure page table
m on VM n. in VMM. in VMM.
Gues VMM
t
Shadow Page
Table
Updates are expensive
Drawbacks
On a write, the VMM must update the VM and
the shadow page table.
TLB must be flushed on world switch.
TLB from other guest will be full of machine
addresses that would be invalid in the new context.
Direct
Direct accessAccess
to hardware is not permitted by the
Popek and Goldberg model .
VMWare and Xen both bend this rule, allow guests
to access hardware directly in certain cases.
Xen uses validated access model .
Fine-grained control over direct access.
VMWare allows user-mode instructions to bypass BT,
go straight to CPU .
Memory accesses are sometimes batched to
minimize context switches.
Memory
Overcommitment
Overcommitment: committing more total
memory to guest OSes than actually exists on
the system .
Guest memory can be adjusted according to workload.
Higher-workload servers get better performance than
with a simple even allocation.
Requires some mechanism to reclaim memory
from other guests .
Poor page replacement schemes can result in
double paging .
VMM marks page for reclamation, OS immediately
moves reclaimed page out of memory
Most common in high memory-usage situations.
I/O
Virtualization
Performance is critical
for virtualized I/O
Guest OS
Many I/O devices Gues
are time- sensitive or t
require low latency . Drive
r
Most common Virtua
method: device l
emulation Devic
VMM presents guest eVMM
OS with a virtual
device Virtua
Preserves security, l
handles concurrency, but Drive
imposes more overhead. r
Physical Device
I/O Virtualization
Problems
Multiplexing
How to share hardware access among multiple
OSes.
Switching Expense
Low-level I/O functionality happens at the VMM
level, requiring a context switch.
I/O Rings,
continued
Xen VMWare
Rings contain memory Ring buffer is
descriptors pointing to
I/O buffer regions constructed in and
declared in guest managed by VMM.
address space.
If VMM detects a great
Guest and VMM deposit deal of entries and exits,
and remove messages
using a producer- it starts queuing I/O
consumer model . requests in ring buffer .
Xen 3.0 places device Next interrupt triggers
drivers on their own
virtual domains, transmission of
minimizing the effect of accumulated
driver crashes. messages.
Summar
y
Current VMM implementations provide safe,
relatively efficient virtualization, albeit often at
the expense of theoretical soundness .
The x86 architecture requires a) binary translation,
b) paravirtualization, or c) hardware support to
virtualize.
Binary translation and instruction trapping costs
are currently the largest drains on efficiency .
Management of memory and other resources
remains a complex and expensive task in modern
virtualization implementations.
Reference
1.
2.
s
Singh, A. “An Introduction To Virtualization”, www.kernelthread.com, 2004.
VMWare White Paper, “Understanding Full Virtualization, Paravirtualization,
and Hardware Assist”, 2007.
3. Barham, P. et al. “Xen and the Art of Virtualization”, SOSP 2003.
4. Waldspurger, C. “Memory Resource Management in VMware ESX Server”, OSDI
2002.
5. Adams, K. and Agesen, O. “A Comparison of Software and Hardware Techniques
for x86 Virtualization”, ASPLOS 2006.
6. Pratt, I. et al. “Xen 3.0 and the Art of Virtualization”, Linux Symposium 2005.
7. Sugerman, J. et al. “Virtualizing I/O Devices on Vmware Workstation’s Hosted
Virtual Machine Monitor”, Usenix, 2001.
8. Popek, G. and Kgoldberg, R. “Formal Requirements for Virtualizable Third-
Generation Architectures”, Communications of the ACM, 1974.
9. Mahalingam, M. “I/O Architectures for Virtualization”, VMWorld, 2006.
10. Smith, J. and Nair, R. Virtual Machines, Morgan Kaufmann, 2005.
11. Bellard, F. “QEMU, a Fast and Portable Translator”, USENIX 2005.
12. Silberschatz, A., Galvin, P., Gagne, G. Operating System Concepts, Eighth Edition.
Wiley & Sons, 2009.
