VIRTUALIZATION AS THE EVOLUTION OF

OPERATING SYSTEMS
Tudor Zaharia
Vrije Universiteit Amsterdam, The Netherlands
[email protected]

ABSTRACT
Although virtualization has been around for more than 50 years, the subject is hotter
than ever. Initiated to simulate multiple machines using hardware and software
techniques, virtualization today brings server consolidation, security and isolation for
multiple operating systems running on the same hardware. This article’s main objective
is to show that virtual machines are just another step in the evolution of operating
systems. I will argue that in fact a virtual machine monitor is a resource manager just
like an operating system is. Important points like CPU and memory management are
going to be addressed. Although not exhaustive, this paper will discuss the most
important attributes of virtual machine monitors as operating systems and my view on
how the field should evolve in this new light.

Keywords: virtualization, operating systems, VMM, hypervisor

1. Introduction machine monitor (VMM) is a resource manager. It

The "magic" word "virtualization" is on allows multiple operating systems to share the same
everybody's lips these days. It seems to provide a hardware and this looks very similar to the multi-user,
solution to many of the problems that computer multi-program support offered by operating systems. I
scientist have had. From server consolidation all the will argue that virtualization is an evolution of
way to running a Real-Time Operating System operating systems and it should be treated like that.
together with a General Purpose Operating system on This means that some of the functionalities provided
a single core chip in a mobile phone [6], virtualization by the operating systems should be moved in the
has proven its values time and time again. In High VMM and inherently the OSes should be moved
Performance Computing, virtualization improves higher on the stack (a thing that actually happened
productivity, performance, reliability, availability and already).
security and decreases software complexity [8]. For Following is an outline of the different types of
embedded systems virtualization means decreased virtualization. Short details are given about each one.
prices for manufacturing, since a decrease in bill of
materials is expected [10]. Moreover, live migration Virtualization can be achieved in several ways:
[5], [2] improves reliability by enabling an entire 1. Emulation/simulation. A software emulator
running operating system to migrate on another allows computer programs to run on a
machine with very little downtime. platform (computer architecture and/or
In datacenters, the Intel 48 core [4] which was operating system) other than the one for
demonstrated last year, could finally find its use which they were originally written. Multiple
beyond very intensive computational applications. such emulators were released under public
This is because although novel programming licenses [16]. The terms emulator and
languages such as SAC [3], Occam [11] and others, simulator are very close in meaning, and it's
for one reason or another, did not became beyond this paper's scope to argue about
mainstream. This means that programmers still use which one is the most appropriate.
traditional imperative languages extended with 2. Partial virtualization simulates multiple
Pthreads [9] and MPI [12] which makes parallel instances of much (but not all) of an
programming very difficult and does not take full underlying hardware environment.
advantage of the multicore processing power of recent 3. OS-level virtualization enables multiple
processors. Instead, this power could be leveraged by isolated and secure virtualized servers to run
the virtualization engines. on a single physical server.
But virtualization provides a lot of features that 4. Paravirtualization presents a software
operating systems also provide. Above all, a virtual interface to virtual machines that is similar
but not identical to that of the underlying
20
 hardware. This means that the "guest" (and might as well never be done).
operating system needs to be modified in
order to run in the paravirtualized hardware 2.1 Operating systems management
(to support the para-API). The term A virtual machine manager deals with multiple
"paravirtualization" was first used in operating system instances, which can vary a lot with
association with the Denali virtual machine regards to the functionalities that they provide and/or
[19]. the way they are implemented. But so do processes
5. Full virtualization provides a complete that run inside an operating system. To run in an
simulation of the underlying hardware. The operating system, processes make system calls which
result is a system in which all software form the API of an operating system. To run in a
capable of execution on the raw hardware (virtual) machine, operating systems make calls that
can be run in the virtual machine. form the API of that (virtual) machine. The VMM has
Increased productivity, performance, reliability just implemented another level of abstraction that
and all of the words are used alongside virtualization. allows multiple operating systems to share the same
But they used to be the qualities of operating systems resources (i.e. to run at the same time on the same
as well. So what did exactly change why is a VMM machine).
different from an Operating System? Considering this, operating systems lost the
In this paper I will argue that virtual machines access to the privileged instruction in favor of the
are just the evolution of the operating systems. For VMM that now runs in ring0 as it is called by
this I will start by looking at the definition of an VMware or dom0 in XEN.
operating system. [15] defines the operating systems If virtual machines calls could someday be
regarding two individual aspects: standardized, every operating system developer could
an extension of a machine's functionality provide its version for that "architecture". One could
a resource administrator argue that this calls are already standardized in a way
These are also the main characteristics of a (by the x86 architecture), and operating systems
virtual machine. In fact, virtualization is just the latest should use that. But this comes against the trend that
development in the operating system's continuous has moved the operating system higher on the stack
adaptation to different forms of multitenancy. This and its place was taken by the virtual machine
started as the need to provide multiuser support and manager.
protected memory. I will argue that, against all recent In fact, even without the standardized interface,
trends which favor full virtualization, a new paradigm developers should port their operating systems to
in operating systems is needed that will change the these diverse virtual machines architectures (just as
face of the operating systems. This is because they did by porting them to various CPU
virtualization should take over some functionality architectures). It would just seem fair since the
provided by the operating systems since it is already application developers were constrained to port their
providing them. application to particular operating systems (as POSIX
This evolution does not exclude the recent standard was not adopted by all the operating system
advances in virtualization support provided by the developers and therefore application developers had
CPUs, but the evolution should be made by rethinking to port their code to each OS).
each component's functionalities and together provide Of course, since the world of computer science
improved performance. depends on the business models as well, it is very
improbable that we will see redesigned OSes so that
2. The virtual machine monitor is a they give up functionality in favor of VMMs.
resource manager I believe this happens because the major players
The extension of a machine's functionalities is in operating systems do not want a standardized API
made by the operating system through its process, for virtualization nor they want to provide less
memory and I/O management. A virtual machine functionality. Microsoft still has a very big share of
manager does the same things, but instead of process the market [7] and it also entered the virtualization
management, it is doing operating system field. It is expected that most of their operating
management. A virtual machine manager multiplexes systems users would prefer to acquire their
resources in two modes: in time and in space, exactly virtualization products.
as an operating system does [15]. On the other hand, VMware, the biggest
This section will describe in detail each one and virtualization company wants to impose its own API.
how some of the functionality actually moved from I believe these are the main reasons that hold
the operating system side to the virtual machine back the rethinking the operating systems in general.
monitor. I will argue that the implementation of these Naturally, another reason is backward compatibility,
functionalities should be rethought in order to provide for which full virtualization is the preferred choice.
better usage of resources. I will also try to give some
answers on why these changes were not made yet
21
 2.2 CPU access management we would not consider that it runs its own drivers.
Operating systems have a scheduling algorithm Currently there are three techniques used for I/O
that decides which process/thread runs next. management, although only the first two are
Similarly, a VMM should decide which operating equivalent:
system runs next. User space device emulation.
Nowadays the most common thing to have is a Hypervisor-based device emulation.
hybrid, where a host operating system runs a type 2 Device passthrough.
(hosted) hypervisor and the guest operating systems In the first case, the device emulation is
are installed on top of this hypervisor. New solutions implemented in user space. QEMU [1] provides
that make a VMM behave more like an OS are device emulation and it is used by a large number of
already available. WMware's variant is called ESX independent hypervisors like KVM [13] and
server. These solutions provide significantly higher VirtualBox [18]. Device emulation is totally
performance [17]. independent from the hypervisor, which makes this
An important penalty hit in virtualization comes solution more secure than hypervisor-based device
from multiple switches between the virtual machine emulation.
monitor and operating system. This is because some The second solution, although faster, it burdens
calls are only allowed in ring 0, the most privileged of the hypervisor with this functionality.
all. This is similar to operating systems, where some Device passthrough can be used when only one
instructions are only allowed to be made from a virtual machine needs access to a particular device. In
certain context. If the operating systems would be this case sharing becomes more efficient, as the
built keeping in mind that they are running over virtualization engine provides isolation of devices to a
virtualized hardware, a lot of calls that trap to the given guest operating system so that the device can be
VMM could be avoided. used only by the designated guest. Improved
performance and isolation of devices that cannot be
2.3 Memory management shared are the main benefits of this approach.
Virtual memory was introduced as an automatic Intel and AMD both provide support for device
alternative to the overlays used by a programmer to passthrough in their recent CPU architecture
run software that would not fit in the machine's main developments. "Virtualization Technology for
memory [14]. A Memory Management Unit translates Directed I/O" (VT-d) from Intel and "I/O Memory
physical addresses into machine addresses. In a Management Unit" (IOMMU) from AMD provide the
virtualized system, memory is virtualized by the means to map PCI physical addresses to guest virtual
hypervisor, which is another level introduced. It is machines addresses. This mapping ensures that the
called Shadow Page Tables (SPTs). These pages access is exclusively granted to that particular virtual
provide a level of indirection between virtual and machine which can use it as if it was a non-virtualized
machine addresses. system.
Intel and AMD both developed hardware I think this solution could be adapted so that it is
support for these SPTs. Intel has called it's system used by the memory manager of a hypervisor. I can
"Extended page tables" and AMD uses the name imagine a system with multiple hosts where each
"Rapid Virtualization Indexing". Another level of operating system has "passthrough" access to a part of
indirection only increases overhead, especially since the memory. In this case paging should (and probably
this level could be entirely moved from the operating will) be kept also on the OS level. An analogous
system to the VMM. Since the hypervisor is the one example is a system with multiple (physical) USB
that runs in privileged mode, it should be the one that ports.
deals completely with the hardware resources. This These are individually isolated to given domains,
means that operating systems should be, again, so each virtual machine has exclusive access to
designed with this in mind. As it is right now, hacks certain ports.
like "ballooning" [17] are used by the VMM in order Further evolution of the virtualization could
to reclaim memory. These again, introduce more bring virtualization-aware devices that would
overheard. eliminate I/O virtualization overhead by employing
In my opinion SPTs are (another) step made in the adequate hardware support. The devices should
the wrong direction. The reasons are again business export multiple interfaces that can be mapped to
based. The OS developers do not want to modify their virtual interfaces inside the virtual machines.
products while CPU manufacturers are always eager Communication between the operating system and the
to support new features requested by the software. device would be made directly, without trapping into
the VMM. This is like DMA in a way, but not from
2.4 I/O management the device to the memory, but from the operating
It is somewhat ironic that device drivers live on system to the device.
the application facing side (in the guest OS) which
makes the hypervisor look like a microkernel. This if
22
 3. Conclusions [3] Clemens Grelck and Sven-Bodo Scholz. Sac: a functional array
language for efficient multi-threaded execution. Int. J. Parallel
Virtualization provides a lot of benefits to its Program, 34(4):383–427, 2006.
user. Most significant, it brings server consolidation. [4] intel.com. Futuristic intel chip could reshape how computers
Since the development of live VM migration, are built, consumers interact with their pcs and personal devices:
http://www.intel.com/pressroom/archive/releases/20091202comp
automatic load balancing is trivial and a robust model sm.htm.
for dealing with hardware failures is real. [5] Christopher Clark Keir, Christopher Clark, Keir Fraser, Steven
The benefits are visible also in mobile phones, H, Jacob Gorm Hansen, Eric Jul, Christian Limpach, Ian Pratt, and
where virtualization can decrease the costs of devices. Andrew Warfield. Live migration of virtual machines. In In
Proceedings of the 2nd ACM/USENIX Symposium on Networked
Virtualization is a solution to provide true isolation Systems Design and Implementation (NSDI, pages 273–286, 2005.
among operating systems. [6] linuxfordevices.com. Singlecore linux phone hits the market.
But there's more to virtualization. In this paper I http://www.linuxfordevices.com/c/a/News/SinglecoreLinuxphonehi
have shown how virtualization is in fact a step in the tsthemarket/.
[7] marketshare.hitslink.com. Os market share.
operating systems' evolution. The main argument was http://marketshare.hitslink.com/os-market-share.aspx?qprid=9.
that a virtual machine monitor is first of all a resource [8] Mark F. Mergen, Volkmar Uhlig, Orran Krieger, and Jimi
manager that multiplexes access to resources to Xenidis. Virtualization for high-performance computing. SIGOPS
multiple operating systems. I have argued that since Oper. Syst. Rev., 40(2):8–11, 2006.
[9] Institute of Electrical and Inc. Information Technology
the operating systems have moved higher on the Electronic Engineers. Portable Operating Systems Interface
stack, they should be designed accordingly. The main (POSIX) Part: System Application Program Interface (API)
points that I discussed covered operating system, Amendment 2: Threads Extension [C Language]. IEEE, IEEE,
CPU, memory and I/O management made by the 1995.
[10] ok labs.com. Ok labs enables world’s first virtualized
VMM. smartphone, with mobile virtualization solution : Open kernel labs.
But the trend is actually the opposite. The same http://www.ok-labs.com/releases/release/
functionality continues to be provided at the operating ok-labs-enables-worlds-first-virtualized-smartphone-with-mobile-
system's level and at the virtual machine monitor's virtualizat.
[11] A. W. Roscoe and C. A. R. Hoare. The laws of occam
one. Advances in CPU designs are made to move programming. Theor. Comput. Sci., 60(2):177–229, 1988.
some of the burden into the hardware instead of [12] Marc Snir and Steve Otto. MPI-The Complete Reference: The
rethinking the whole hardware and software stack. MPI Core.
This might be because Operating Systems failed to MIT Press, Cambridge, MA, USA, 1998.
[13] Sun Microsystems Inc. The k virtual machine (kvm). White
provide the real process isolation, and the problem paper, 1999.
was solved through virtualization. Moving [14] Andrew S. Tanenbaum. Structured Computer Organization
functionalities from the operating system to the VMM (5th Edition). Prentice-Hall, Inc., Upper Saddle River, NJ, USA,
could open up old wounds and, moreover, turn 2005.
[15] Andrew S. Tanenbaum. Modern Operating Systems. Prentice
operating systems into "bags of drivers" as Marc Hall Press, Upper Saddle River, NJ, USA, 2007.
Anddressen once said. [16] Jeffrey van der Hoeven, Bram Lohman, and Remco Verdegem.
Emulation for digital preservation in practice: The results.
References: International Journal of Digital Curation, 2(2), 2007.
[1] Fabrice Bellard. Qemu, a fast and portable dynamic translator. [17] Carl A. Waldspurger. Memory resource management in
In ATEC ’05: Proceedings of the annual conference on USENIX vmware esx server.
Annual Technical Conference, pages 41–41, Berkeley, CA, USA, [18] Jon Watson. Virtualbox: bits and bytes masquerading as
2005. USENIX Association. machines. Linux J., 2008(166):1, 2008.
[2] Brendan Cully, Geoffrey Lefebvre, Dutch Meyer, Mike Feeley, [19] Andrew Whitaker, Marianne Shaw, and Steven D. Gribble.
Norm Hutchinson, and Andrew Warfield. Remus: high availability Denali: Lightweight virtual machines for distributed and networked
via asynchronous virtual machine replication. In NSDI’08: applications. In Proceedings of the USENIX Annual Technical
Proceedings of the 5th USENIX Symposium on Networked Conference, 2002.
Systems Design and Implementation, pages 161–174, Berkeley,
CA, USA, 2008. USENIX Association.

23