The Memory Pool System

Thirty person-years of memory management development goes Open Source

Richard Brooksby Nicholas Barnes
Ravenbrook Limited Ravenbrook Limited
PO Box 205 PO Box 205
Cambridge, United Kingdom Cambridge, United Kingdom
[email protected] [email protected]

ABSTRACT developing the MPS. It contains many innovative techniques and

The Memory Pool System (MPS) is a very general, adaptable, flex- abstractions which have hitherto been kept secret. In 1997, Richard
ible, reliable, and efficient memory management system. It permits Brooksby, the manager and chief architect of the project, and Nicholas
the flexible combination of memory management techniques, sup- Barnes, a senior developer, left Harlequin to form their own con-
porting manual and automatic memory management, in-line alloca- sultancy company, Ravenbrook Limited, and in 2001, Ravenbrook
tion, finalization, weakness, and multiple concurrent co-operating acquired the MPS technology from Global Graphics.
incremental generational garbage collections. It also includes a li- Our goals in going open source are that as many people as pos-
brary of memory pool classes implementing specialized memory sible benefit from the hard work that the members of the (now de-
management policies. funct) Memory Management Group put in to the MPS design and
The MPS represents about thirty person-years of development implementation. We also hope to develop the MPS further, through
effort. It contains many innovative techniques and abstractions commercial licensing and consultancy.
which have hitherto been kept secret. We are happy to announce This paper gives an overview of the MPS, with particular em-
that Ravenbrook Limited is publishing the source code and doc- phasis on innovative things that it does.
umentation under an open source licence. This paper gives an
overview of the system. 2. BACKGROUND
Categories and Subject Descriptors 2.1 History
D.2.3 [Software Engineering]: Coding Tools and Techniques; D.3.3 The original Memory Management Group, set up in 1994, con-
[Programming Languages]: Language constructs and features— sisted of Richard Brooksby and P. Tucker Withington. Richard had
dynamic storage management; D.3.4 [Programming Languages]: previously worked in the ML Group and implemented the MLWorksTM
Processors—memory management; D.4.2 [Operating Systems]: memory manager and garbage collector. Tucker joined from the ail-
Storage Management ing Symbolics Inc, where he maintained the Lisp Machine’s mem-
ory systems.
The initial brief of the group was to provide a memory manager
General Terms for Harlequin’s new Dylan system. Harlequin was also interested in
Algorithms, Design, Reliability a broader set of memory management products, and in absorbing
the memory managers of other products, such as ScriptWorksTM
Keywords (the high-end PostScript R language compatible raster image pro-
cessor), LispWorksTM , and MLWorksTM . Initial prototyping and
Garbage collection, memory management, software engineering design work concentrated on a flexible memory management frame-
work which would meet Dylan’s requirements but also be adaptable
1. INTRODUCTION to other projects, and form a stand-alone product.
The Memory Pool System (MPS) is a flexible, extensible, adapt- Richard’s concerns about the subtlety of a generic memory man-
able, and robust memory management system, now available under agement interface, the pain of debugging memory managers, and
an open source licence from Ravenbrook Limited. the complexity of the Dylan implementation, led him to push for a
Between 1994 and 2001, Harlequin Limited (now Global Graph- fairly formal requirements specification. This set the tone for the
ics Software Limited) invested about thirty person-years of effort group’s operations, and led to extensive use of formal software en-
gineering techniques such as inspections. At its height, the group
was operating a Capability Maturity Model level 3 process [12]. As
a result, the MPS was very robust and had a very low defect rate.
This enabled the group to concentrate on development.
Permission to make digital or hard copies of all or part of this work for The Memory Management Group collaborated with the Labora-
personal or classroom use is granted without fee provided that copies are tory for the Foundations of Computer Science at Edinburgh Uni-
not made or distributed for profit or commercial advantage and that copies versity, with the goal of formally verifying some of the algorithms
bear this notice and the full citation on the first page. To copy otherwise, to [16].
republish, to post on servers or to redistribute to lists, requires prior specific
permission and/or a fee. The MPS was incorporated into the run-time system of Harlequin’s
Copyright 2002 Ravenbrook Limited. DylanWorksTM compiler and development environment (now avail-
able as Functional Developer from Functional Objects, Inc). Later, an object is not white). Other arena classes work in real memory
it replaced the highly optimized memory manager in Harlequin’s machines, such as printer controllers.
ScriptWorksTM , improving performance and reliability to this day The MPM co-ordinates the activities of the pools, interfaces with
as part of Global Graphics’ Harlequin RIP R . the client, and provides abstractions on which the memory manage-
ment policies in the pools are implemented.
2.2 Requirements This architecture gives the MPS flexibility, its primary require-
The MPS had a fairly large and complex set of requirements ment, by allowing an application of the memory manager to com-
from the beginning. The Harlequin Dylan project was formed from bine specialized behaviour implemented by pool classes in flexi-
highly experienced Lisp system developers who knew what they ble configurations. It also contributes to adaptability because pool
wanted [10]. The requirements of ScriptWorksTM were even more classes are less effort to implement than a complete new memory
complex [8]. On top of this, we were always striving to anticipate manager for each new application. Reliability is enhanced by the
future requirements. fact that the MPM code can be mature code even in new appli-
This section describes the overall architectural requirements that cations. However, efficiency is reduced by the extra layer of the
guided all aspects of the design [11]: MPM between the client code and the memory management pol-
icy. This problem is alleviated by careful critical path analysis and
Adaptability The MPS has to be easy to modify to meet new optimization of the MPM, and by providing abstractions that allow
requirements. This makes the MPS suitable for new applica- the MPM to cache critical information.
tions and ensures it has long and useful life.

Flexibility The MPS must fit into a number of different products 4. IMPLEMENTATION
and meet differing requirements in diverse environments. It The MPS is about 62 Kloc of extremely portable ISO standard C
must do this with as little modification as possible, so that [1]. Except for a few well-defined interface modules, it is freestand-
it can be deployed at low cost. Flexibility gives the MPS ing (doesn’t depend on external libraries1 ). We have been known
broad application, and reduces the need to maintain special to port to a new operating system in less than an hour.
versions of the MPS for different clients. Code re-use also The code is written to strict standards. It is heavily asserted, with
leads to robustness through use testing. checks on important and subtle invariants. Every data structure
has a run-time type signature, and associated consistency check-
Reliability Memory management defects are very costly. In de- ing routines which are called frequently when the MPS is compiled
velopment they are difficult to find and fix, and once de- in “cool” mode2 . Much of the code has been put through formal
ployed they are virtually impossible to reproduce. The MPS code inspection (at 10 lines/minute or less) by between four and
may be shipped to third and fourth parties, further increasing six experienced memory management developers [15]. It was de-
the cost of a defect. Reliability is therefore very important to veloped by a team working at approximately Capability Maturity
the viability of the MPS. Model level 3 [CMMI1.02]. As a result, it is extremely robust, and
has a very low defect rate.
Efficiency Efficiency will always be required by clients; after all, The MPS is designed to work efficiently with threads, but is
memory management is about the efficient utilization of re- not currently multi-threaded. Fast allocation is achieved by a non-
sources to meet requirements. However, the tradeoffs be- locking in-line allocation mechanism (see section 5.2).
tween those requirements will differ from application to ap-
plication, hence the need for adaptability and flexibility. A
generally efficient system will make it easier to meet these 5. KEY FEATURES AND ATTRIBUTES
requirements.
5.1 Flexible combination of memory manage-
3. ARCHITECTURE ment techniques
The MPS consists of three main parts: The most important feature of the MPS is the ability to com-
bine memory management policies efficiently. In particular, mul-
1. the Memory Pool Manager (MPM) tiple instances of differing garbage collection techniques can be
combined. In the Harlequin Dylan system, for example, a mostly-
2. the pool classes, and copying main pool is combined with a mark-sweep pool for han-
dling weak key hash-tables, a manually-managed pool containing
3. the arena classes. guardians implementing user-level weakness and finalization, and a
mark-sweep pool for leaf objects. The same codebase is used with
See Figure 1. a very different configuration of pools in the Harlequin R RIP3 .
Each pool class may be instantiated zero or more times, creating An overview of the abstractions that allow flexible combination
a pool. A pool contains memory allocated for the client program. can be found in section 6.
The memory is managed according to the memory management

policy implemented by its pool class. For example, a pool class Ironically, a lot of clever design went into the interfaces (MPM
may implement a type of garbage collection, or manage a partic- and the plinth) to make robust and efficient binary interfaces for a
ular kind of object efficiently. Each pool can be instantiated with closed-source
 MPS library.
different parameters, creating variations on the policy. The MPS can be compiled with various flags to give different va-
The arena classes implement large-scale memory layout. Pools rieties. The “cool” varieties are intended for debugging and testing.
The “hot” varieties are for delivery. Some level of internal consis-
allocate tracts of memory from the arena in which they manage tency checking is present in all varieties.

client data. Some arena classes use virtual memory techniques to The details of the Harlequin R RIP configuration and the the RIP-
give control over the addresses of objects, in order to make mapping specific pool class implementations are confidential, and not avail-
from objects to other information very efficient (critically, whether able under an open source licence.
 Client Memory Pool
Inter- Pool manage Pools
Program Manager (MPM)
face Classes

contained by

Plinths faults

Arena Classes Arenas

Threads &
manage
Locks
Libraries Protections
VMs contained by

Memory /
Operating System manage Address
Space

Figure 1: The MPS Architecture.

5.2 Efficient in-line allocation practice.

The MPS achieves high-speed multi-threaded allocation using The AP protocol relies on atomic ordered access to words in
the abstraction of allocation points backed by allocation buffers [9]. memory, and some care must be taken to prevent some processors
The allocation point protocol also allows garbage collection to take from re-ordering memory accesses.
place without explicit synchronization with the mutator threads. The design of allocation buffers was inspired by the Symbolics
An allocation point (AP) consists of three pointers: init, al- Lisp Machine allocator, which supports a similar protocol in mi-
loc, and limit. Before allocation, init is equal to alloc. The crocode, but requires atomic initialization of objects.
thread owning the AP reserves a block by increasing alloc by the
size of the object. If the result exceeds limit, it calls the MPS to 5.3 A library of pool classes
complete the reservation, but otherwise it can initialize the object at
init. Once the object is initialized, the thread commits it, by set- 5.3.1 A: Allocate only
ting init equal to alloc, checking to see if limit is zero, and A simple pool class which only supports in-line allocation. This
calling the MPS if it is. At this point the MPS may return a flag in- is useful when objects need to be allocated rapidly then deleted
dicating that the object has been invalidated, and must be allocated together (by destroying the pool). Allocation is very fast. This pool
and initialized again. Both the reserve and commit operations take is not currently in the open sources.
very few instructions, and can be inlined.
The exact implementation of the AP protocol depends on the 5.3.2 AMC : Automatic Mostly Copying
pool from which the thread is allocating. Some pools may guar- The most complex and well-developed pool class, this was orig-
antee that an object is never invalidated, for example, and so the inally designed as the main pool for Harlequin’s implementation
commit check can be omitted. Most pools implement APs by back- of Dylan, but is a general purpose moving pool. It implements a
ing them with allocation buffers, fairly large contiguous blocks generational mostly-copying algorithm [5].
of memory from which they can allocate without ever calling the
MPS. 5.3.3 AMS : Automatic Mark Sweep
The AP protocol allows in-line allocation of formatted objects This is the general-purpose non-moving counterpart of AMC.
(see section 5.5) containing references that need tracing by a garbage Not generational.
collection. The MPS knows that objects up to init have been
initialized and can therefore be scanned. It also knows that an ob- 5.3.4 AWL : Automatic Weak Linked
ject that is half-initialized (somewhere between reserve and com- A specialized pool originally designed to support weak key hash
mit) when a flip occurs (see section 6.7) can’t be scanned, and may tables in Dylan. In a weak key hash table, the value is nulled out
therefore contain bad references, that is, references which have not when the key dies (see section 6.4). The pool implements mark
been fixed (see section 6.5). Hence the commit check, and the re- sweep collection on its contents.
allocation protocol. The chances of a flip occurring between a re-
serve and commit are very low, and re-allocation rarely happens in 5.3.5 LO : Leaf Object
 This pool stores leaf objects (objects not containing references). The MPS includes no object format information; rather it allows
It was originally designed for use with the Dylan foreign function each MPS client to specify one or more object formats, by provid-
interface (FFI), guaranteeing that the objects will not be protected ing a small set of methods which operate on pieces of memory.
by the MPS under a hardware read or write barrier, because inter- When a client creates a pool of a formatted pool class, it speci-
actions with foreign code would be unpredictable. fies an object format for the objects allocated in that pool. The pool
class is then able to invoke these methods as necessary to perform
5.3.6 MFS : Manual Fixed Small format-specific operations. In particular, the scan and fix methods
A simple pool which allocates objects of fixed (regular) small of the pool class interact closely with the format methods (see sec-
(much less than a page) size, though each instance of the pool can tion 6.5).
hold a different size. Not garbage collected. Used internally to the Different pool classes will use formats in different ways. For
MPS for the management of some of its own dynamic structures. example, a copying garbage-collected pool may need methods to:

calculate the size of an object;
5.3.7 MRG : Manual Rank Guardian

This pool is used internally in the MPS to implement user-level copy an object;
finalization of objects in other pools. Some techniques from [14] 
scan an object;
were used in its design.

replace an object with a “broken heart” (containing a for-
5.3.8 MV : Manual Variable warding pointer).
A manually managed pool for variable sized objects. This is the
pool class used internally by the MPS as the control pool for many A non-moving mark-and-sweep pool, on the other hand, may
MPS data structures. It is designed to be very robust, and, like only need methods to calculate the size of an object and to scan an
many other MPS pools, keeps all of its data structures away from object.
the objects it manages. It is first-fit, with the usual eager coalescing. The format methods include the following[4]:

skip Skips a pointer over an object.

5.3.9 MV2 : Manual Variable 2
An unfinished manually managed pool for variable sized objects, copy Makes a copy of the object in another location. Objects are
using bitmaps and crossing maps. Designed for high-performance usually copied byte by byte, but some uncommon object for-
freelists with subtle theory of block reuse. MVFF is bascially a mats might contain relative pointers that have to be fixed up
light version of the same that just uses the high performance freelist when the object is moved.
and first fit.
pad Fills a block of memory with a dummy object. This should
work just like a real object in all the other methods, but con-
5.3.10 MVFF : Manual Variable First Fit tain no data. This method is used by the MPS to fill in odd
A general-purpose manually-managed pool for variable sized ob- corners that need to be scannable.
jects, implementing address-ordered first-fit, but with in-line worst-
fit allocation. It is optimized for high performance when there is fwd Replaces an object with a “broken heart” of the same size,
frequent deallocation in various patterns. containing a forwarding pointer. It is used when the MPS
has moved an object to a new location in memory.
5.4 Support for location dependency isfwd distinguishes between a broken heart and a real object, re-
Some data structures and algorithms use the address of an ob- turning the forwarding pointer of a broken heart.
ject. This can be a problem if the memory manager moves objects
around. The MPS provides an abstraction called location depen- scan locates all references in a contiguous set of objects and tells
dency (LD) which allows client code to depend on the locations of the MPS where they are. The objects may include dummy
moving objects. objects and broken hearts.
The design of LDs was inspired by the Symbolics Lisp Machine
align is an integer value defining the alignment of objects allo-
which has hardware support for something similar, and an evolution
cated with this format.
of an algorithm developed by P. Tucker Withington for simulating
the LispM hardware on stock hardware when writing the Lisp Ma- Note that a single client may use more than one object format,
chine emulator. even within the same pool class. A client may choose to have many
formatted pools, specializing format methods to the kind of object
5.5 Client object formats which is allocated within a given pool.
Garbage collectors always have some information about the for-
mat of the objects they manage. For example, a non-conservative 5.6 Multiple arenas
garbage collector must be able to find and interpret references within Memory managers, and garbage collectors especially, usually
the objects it manages. A garbage collector must be able to com- work well with exactly one client. They take over the whole mem-
pute the size of an object given a reference to that object. If the ory infrastructure of a process and provide a single instance of an
object allocation is entirely handled by the client (as in the MPS; abstraction to a single client. For instance, they often assume that
see section 5.2), the size information is encoded somehow in the they have total control over memory protection and memory map-
object format. pings.
This format information is usually entangled deeply in the source In today’s modular software world, such an approach has obvi-
code of the collector, for instance in the innermost scan/fix loop. ous drawbacks. How do you link together two components, possi-
Adapting such a collector to a new object format may be difficult bly written in different languages, with different memory manage-
and may introduce some very complex defects. ment infrastructures?
 The design of the MPS avoids such assumptions about the en- ing the reference partition invariant by moving objects from to
vironment in which it runs. It abstracts its entire interface with a where necessary, until we end up with a final reference partition.
client into an arena object. There is no “global state” of the MPS The key observation here is that any number of partitions can
(apart from the set of arenas). Separate arenas are managed entirely exist for a graph, and so there’s no theoretical reason that multiple
independently. All MPS operations (e.g., allocation, pool creation, garbage collections can’t happen simultaneously.
garbage collection) are per-arena. A second important observation is that because reference parti-
There is more than one way to provide the underlying memory tions can be defined for any property, one could have, for example,
which the MPS manages: it may be a dynamic amount obtained a reference partition with respect to a certain size of objects. The
from a virtual memory subsystem, or it may be a fixed amount MPS uses the reference partition abstraction to implement some-
of client memory (for instance, in an embedded controller or an thing equivalent to “remembered sets” [20] by maintaining refer-
application which needs a constant memory footprint). These are ence partitions with respect to areas of address space called zones.
implemented as distinct arena classes. Note that the MPS may This is described further in section 6.2.
manage arenas from more than one arena class simultaneously. Reference partitions can be usefully combined. If  and ! are
For testing purposes, there is also an arena class which obtains reference partitions, then we can define reference partitions #"$!
memory from the C standard library malloc function. as &% ' )(*+,-% ".-(/102,3%." 3(4 3% ".2(5 and 3'6!
as  % "7 ( +, % "7 ( 80# % "$ (  % '9 (  .
6. THE TRACER
The Tracer co-ordinates the garbage collection of memory pools.
6.2 Induced graphs
The Tracer is designed to drive multiple simultaneous garbage Given a directed graph and an equivalence relation on the nodes
collection processes, known as traces, and therefore allows several we can define an equivalence-class induced graph whose nodes are
garbage collections to be running simultaneously on the same heap. the equivalence classes. If there’s an edge between two nodes in the
Each trace is concerned with refining a reference partition (section graph, then there’s an edge between the equivalence classes in the
6.1) using a five-phase garbage collection algorithm that allows for induced graph. We can define reference partitions on the induced
incremental generational non-moving write-barrier type collection graph, and do refinement on those partitions in just the same way as
(possibly with ambiguous references) combined with incremental for the original graph. We can garbage collect the induced graph.
generational moving read-barrier type collection, while simulta- In fact, you can think of conventional garbage collectors as do-
neously maintaining generational and inter-pool remembered sets. ing this all the time. Consider a language in which an object may
Furthermore, the Tracer co-ordinates garbage collection across pools. have sub-objects (inlined within its representation in memory), and
A trace can include any set of pools. The Tracer knows nothing of an object may refer directly to sub-objects of other objects. You
the details of the objects allocated in the pools. can represent the sub-object relationship with implicit references
This section describes the abstractions used to design such a gen- between sub-objects. Then there is a graph on the sub-objects, in-
eral system. The definitions are rather abstract and mathematical, cluding both the usual references and these implicit references as
but lead to some very practical bit twiddling. The current MPS im- edges. The graph of objects we normally discuss is induced from
plementation doesn’t make full use of these abstractions. Nonethe- this graph, the equivalence classes being the sets of sub-objects of
less, they were critical in ensuring the that the MPS algorithms were separate objects. Theoretically we could garbage collect individual
correct. It is our hope that they will be of great use to future design- sub-objects by tracing this lower level graph, and reclaim the mem-
ers of garbage collection algorithms. ory occupied by parts of objects! The MPS is general enough to
support this, though we have not implemented a pool class which
6.1 Reference Partitions does it.
The MPS was originally based on a theory of reference parti- Given two equivalence relations : and ; on a directed graph ,
tions, which we developed to generalize the familiar idea of “tri- we can define an equivalence-class relation induced by : and ; ,
colour marking” [13]. Subsequently the MPS was refined to in- which is a binary relation between the equivalence classes of : and
clude more varied barrier techniques[19], but we present the basic the equivalence classes of ; . :.;<=>? is in the relation if the
theory here to give a flavour of the MPS. graph includes an edge from  to = in the graph.
A reference partition is a colouring of the nodes in a directed The MPS divides address space into large areas called zones. The
graph of objects. Every object is either “black”, “grey”, or “white” set of zones is called @ . The number of zones is equal to the number
in any reference partition. A partition    of a directed of bits in the target machine word, so any set of zones (subset of
graph is a reference partition if and only if there are no nodes in @ ) can be represented by a word. Given an equivalence relation

 which have a reference to any node in , that is, nothing black : , the equivalence-class relation induced by : and @ is called the

can refer to anything white. summary of each equivalence class of : . Roughly speaking, it
An initial partition is one with no black nodes:    . All summarizes the set of zones to which that class refers. This is a
initial partitions are trivially reference partitions. BIBOP-like technique adapted from the Symbolics Lisp Machine’s
A final partition is one with no grey nodes:   . hardware assisted garbage collection [18].
For a predicate  , we say some reference partition    The Tracer groups objects into segments, and maintains a con-
is a reference partition with respect to  if and only if everything servative approximation of the summary of each segment. By do-
with  is in , that is,  . ing this, it is maintaining a reference partition with respect to each
If we can determine a final reference partition such that the client zone. Segments which don’t have a zone in their summary are
process roots are contained in  then is unreachable by the “black” for that zone, segments which do are “grey” if they aren’t in
mutator, cannot affect future computation, and can be reclaimed. the zone, and “white” if they are. The Tracer uses this information
Tracing is a way of finding final reference partitions by refine- to refine traces during phase 1 of collection; see section 6.7.
ment. We start out by defining an initial reference partition with
respect to a “condemned” property, such as being a member of a 6.3 Segments
generation. We then move reachable objects from to  , preserv- The Tracer doesn’t deal with individual client program objects.
All details of object allocation and format is delegated to the pool. 6.4 Reference Ranks for Ambiguity, Exact-
The Tracer deals with areas of memory defined by pool classes ness, Weakness, and Finalization
called segments. The segment descriptor contains these fields im- The rank of a reference controls the order in which references
portant to tracing: are traced while refining the reference partition. All references of
white The set of traces for which the segment is white. A superset lower numbered rank are scanned before any references of higher
of the union of the trace whiteness of all the objects in the rank. The current MPS implementation supports four ranks:
segment. More precisely, if a trace is not in the set, then the
segment doesn’t contain white objects for that trace. 1. ambiguous An ambiguous reference is a machine word which
may or may not be a reference. It must be treated as a refer-
grey The set of traces for which the segment is grey. (See “white” ence by the MPS in that it preserves its referent if it’s reach-
above.) able from the client process roots, but can’t be updated in
case it isn’t a reference, and so its referent can’t be moved.
summary A summary (see section 6.2) of all the references in the
Ambiguous references are used to implement conservative
segment.
garbage collection [7].
ranks A superset of the ranks of all the references in the segment
(see section 6.4). 2. exact An exact reference is definitely a reference to an object
if it points into a pool. Depending on the pool, the referent
In addition, the Tracer maintains a set of traces for which the may be moved, and the reference may be updated.
mutator is grey (and assumes it’s black for all other traces), and
a summary for the mutator. The mutator is a graph node which 3. final A final reference is just like an exact reference, except
consists of the processor registers, and any references in memory that a message (sometimes called a “near death notice”) is
that can’t be protected against transfers to and from the registers. sent to the client process if the MPS finds no ambiguous or
This is not usually the same as the root set. exact references to the referent. This mechanism is used to
Memory barriers4 are used to preserve the reference partitions implement finalization [17, 14].
represented by the traces in the face of mutation by the client pro-
4. weak A weak reference is just like an exact reference, except
cess. These invariants are maintained by the MPS at all times:
that it doesn’t preserve its referent even if it is reachable from
 the client process roots. So, if no reachable ambiguous, ex-
Any segment whose grey trace set is not a subset of the mu-
tator’s grey trace set is protected from reads by the mutator. act, or final references are found, the weak reference is sim-
This prevents the mutator from reading a reference to a white ply nulled out. This mechanism is used to implement weak-
object when the mutator is black. If the mutator reads the ness.
segment, the MPS catches the memory exception and scans
the segment to turn it black for all traces in the difference be- Note that the pool which owns the reference may implement ad-
tween the sets. (An theoretical alternative would be to “un- ditional semantics. For example, when a weak reference is nulled
flip”, making the mutator grey for the union of the sets, but out, the AWL pool nulls out an associated strong reference in order
this would seriously set back the progress of a trace.) to support weak key hash tables.

Ranks are by no means a perfect abstraction. Parts of the Tracer
Any segment whose grey trace set is not a superset of the have to know quite a bit about the special semantics of ambiguous
mutator’s grey trace set is protected from writes by the mu- references. The Tracer doesn’t take any special action for final and
tator. This prevents the mutator from writing a reference to a weak references other than to scan them in the right order. It’s the
white object into a black object. If the mutator writes to the pools that implement the final and weak semantics. For example,
segment, the MPS catches the memory exception and makes the MRG pool class is one which implements the sending of near
the segment grey for the union of the sets. (An alternative death notices to the client process.
would be to “flip”, making the mutator black for the differ- The current implementation of the Tracer does not support seg-
ence between the sets, but this would generally be premature, ments with more than one rank, but is designed to be extended to
pushing the colletion’s progress along too fast.) do so.

Any segment which has a summary which is not a superset of The current MPS ordering puts weak after final, and is equivalent
the mutator’s summary is protected from writes by the muta- to Java’s “phantom references”. It would be easy to extend the
tor5 . If the mutator writes to the segment, the MPS catches MPS with additional ranks, such as a weak-before-final (like Java’s
the memory exception and unions the summary with the mu- “weak references”).
tator’s summary, removing the protection. Abstractly, this is
the same invariant as for the grey trace set (see above), be-
6.5 Scanning and Fixing
cause the summaries represent reference partitions with re- In order to allow pools to co-operate during a trace the MPS
spect to zones. The barrier prevents the mutator writing a needs a protocol for discovering references. This protocol is the
pointer to a white object (the zone) into a black object (which most time critical part of the MPS, as it may involve every object
doesn’t refer to the zone). that it is managing. The MPS protocol is both abstract and highly
A optimized.
The MPS uses memory protection (hardware memory barrier) for Each pool class may implement a scan and fix method. These
this, but could easily be adapted to a software barrier (if we have are used to implement a generic scan and generic fix method which
control over the compiler). The MPS abstraction of memory barri- dispatch to the pool’s method as necessary. The scan method maps
ers distinguishes between read and write barriers, even if the spe-
cific environment cannot. the generic fix method over the references in a segment. The fix
B
The current implementation of the MPS assumes that the muta- method preserves the referent of a reference (except when it is ap-
tor has a universal summary. In other words, it assumes that the plied to weak references), moving it out of the white set for one or
mutator could refer to any zone. This could be improved. more traces.
 The most important optimization is part of the generic fix method large number of grey segments. This is how the MPS implements
which is inlined into the scan methods. The generic fix method first remembered sets.
looks up the reference in the interesting set (see section 6.7), which In a similar way, the MPS could also use the current status of
takes about three instructions. This eliminates almost all irrelevant other traces to refine the new trace. Imagine a large slow trace
references, such as references to generations which aren’t being which is performing a copying collection of three generations. A
collected, or references to objects not being managed by the MPS. fast small trace could condemn the old space of just one of the
A second level optimization in the generic fix method checks to generations. Any object which is black for the large trace is also
see if the reference is to a segment managed by the MPS, and then black for the small trace. Such refinement is not implemented in
whether the segment is white for any of the traces for which the the MPS at present.
reference is being fixed. This eliminates many more references. Note that these refinement steps could be applied at any time:
Only if a reference passes these tests is the pool’s fix method they are just refinements that preserve reference partitions. The
called. MPS currently only applies them during the condemn step.
A pool need not implement both scan and fix methods. A pool
which doesn’t contain references, but does contain garbage col- 6.7.2 Phase 2: Grey Mutator Tracing
lected objects, will have a “fix” method but no “scan” method. Note This phase most resembles a write-barrier non-moving garbage
that such objects are either black or white. A pool which contains collector [2]. Any segment “blacker” than the mutator is write pro-
references involved in tracing, but not garbage collected objects, tected (see section 6.3).
will have a “scan” method but no “fix” method. Such a pool would At this point the mutator is grey for the trace. Note that, at any
be a pool of roots, its objects either grey or black. A pool with stage, the mutator may be grey or black for different traces inde-
neither method is not involved in tracing, for example, a manually pendently. In addition, newly allocated objects are grey, because
managed pool storing strings. they are being initialized by the mutator.
An object can be moved provided that it is white for any trace for
6.6 Roots which the mutator is black, because the mutator can’t see references
Roots are objects declared to the MPS by the client process as to that object. [What about ambiguous references?]
being a priori alive. The purpose of tracing is to discover objects During phases 2 and 4 the Tracer makes progress by scanning
which aren’t referenced by transitive closure from the roots and segments which are grey for one or more traces (see section 6.5) in
recycle the memory they occupy. order to make them black. Thus we make progress towards a final
The MPS supports various kinds of roots. In particular, a thread reference partition (see section 6.1).
can be declared as a root, in which case its stack and registers are
scanned during a collection. 6.7.3 Phase 3: Flip
Roots have “grey” and “summary” fields just like segments, and Flipping for a set of traces means turning the mutator black for
may be protected from reads and writes by the mutator using the those traces. This may entail scanning the client process thread
same rules. However, roots are never white for any trace, since registers and any unprotectable data. The mutator can’t be running
their purpose is to be alive. while this is happening, so the MPS stops all mutator threads.
This is also the point at which the MPS sets the limit fields of
6.7 Five phase collection any formatted allocation points to zero, so that unscannable half-
The Tracer runs each trace through five phases designed to al- allocated objects are invalidated (see section 5.2).
low pools to co-operate in the same trace even though they may 6.7.4 Phase 4: Black Mutator Tracing
implement very different kinds of garbage collection.
This phase most resembles a read-barrier possibly-moving garbage
6.7.1 Phase 1: Condemn collector [6]. Any segment “greyer” than the mutator is read pro-
tected (see section 6.3).
The set of objects we want to try to recycle, the condemned set
At this point the mutator is black for the trace. In addition, newly
is identified, and a newly allocated trace is added to the white trace
allocated objects are black, and don’t need to be scanned.
set for the segments containing them.
At this stage, all segments containing any references (even the 6.7.5 Phase 5: Reclaim
white ones) are assumed to be grey for the trace, because we don’t
When the grey set for a trace is empty after flip then it represents
know whether they contain references to objects in the white set.
a final reference partition. The Tracer looks for segments which are
The roots are made grey for the trace, because they are a priori
white for the trace and calls the owning pool to reclaim the space
alive. The mutator is also assumed to be grey for the trace, because
occupied by remaining white objects within.
it has had access to all the grey data. Thus we start out with a valid
It’s up to the pool to decide whether to return the reclaimed space
initial reference partition (see section 6.1).
to its own free list, or to the arena.
We then use any existing reference partitions to reduce the num-
ber of grey segments for the trace as much as possible, using this
rule: Let  be the set of reference partitions whose white sets are 7. FUTURE DIRECTIONS
supersets of the new white set. Any node which is in the union of The Memory Management Group at Harlequin was whittled away
the black sets of  cannot refer to any member of the new white to nothing as Harlequin slid into financial trouble. Parts of the sys-
set, and so is also black with respect to it. tem are incomplete or have unclear status. A large amount of design
In practical terms, we work out the set of zones occupied by the documentation exists, but it is fairly disorganized and incomplete.
white set. We call this the interesting set. We can then make any We would like to organize all this information to make the MPS a
segment or root whose summary doesn’t intersect with the interest- more useful resource.
ing set black for the new trace. This is just a bitwise AND between The MPS was designed around many abstractions that make it
two machine words. A pool will usually arrange for a generation very adaptable, but it is not very well packaged and is unlikely to
to occupy a single zone, so this refinement step can eliminate a work in new applications without some modification. We would
like to improve the MPS to make it easier to apply without modifi- [1] American National Standards Institute. American National
cation. Standard for Information Systems: Programming Language
The MPS is currently commercially licensed to Global Graphics C, Dec. 1989.
Software Limited for use in the Harlequin RIP R , and to Configura [2] A. W. Appel, J. R. Ellis, and K. Li. Real-time concurrent
Sverige AB for use in their Configura R business system. We are collection on stock multiprocessors. ACM SIGPLAN Notices,
seeking further licensees and consultancy. 23(7):11–20, 1988.
[3] G. Attardi, T. Flagella, and P. Iglio. A customisable memory
8. AVAILABILITY management framework for C++. Software Practice and
Experience, 28(11):1143–1183, Nov. 1998.
The MPS project tree is available on the web at <http://
www.ravenbrook.com/project/mps/>. It includes all of [4] N. Barnes. MPS Format Protocol. MPS Project
the non-confidential source code and design documentation. This Documentation, November 2001.
is a mirror of the tree in Ravenbrook’s configuration management http://www.ravenbrook.com/project/mps/
master/mmdoc/protocol/mps/format/.
repository, so it will continue to reflect the development of the
MPS. [5] J. F. Bartlett. Mostly-Copying garbage collection picks up
generations and C++. Technical note, DEC Western
Research Laboratory, Palo Alto, CA, Oct. 1989. Sources
9. RELATED WORK available in ftp://gatekeeper.dec.com/pub/DEC/CCgc.
The MPS resembles the Customisable Memory Management (CMM) [6] H.-J. Boehm, A. J. Demers, and S. Shenker. Mostly parallel
framework for C++ [3] and shares some of its design goals. The garbage collection. ACM SIGPLAN Notices, 26(6):157–164,
MPS was not designed for C++, but as a memory manager for dy- 1991.
namic language run-time systems. In fact, it was specifically de- [7] H.-J. Boehm and M. Weiser. Garbage collection in an
signed not to require C++. uncooperative environment. Software Practice and
The Memory Management Reference Bibliography contains all Experience, 18(9):807–820, 1988.
the papers that we collected during the project, and can be found on [8] R. Brooksby. MM/EP-Core Requirements. MPS Project
the web at <http://www.memorymanagement.org/bib/>. Documentation (Confidential), November 1995.
[9] R. Brooksby. Allocation Buffers and Allocation Points. MPS
10. CONCLUSIONS Project Documentation, September 1996.
During our stay at Harlequin we were often frustrated by confi- http://www.ravenbrook.com/project/mps/
dentiality. We were not able to reveal ideas and techniques which master/mminfo/design/mps/buffer/.
we believed were both innovative and useful. The MPS contains [10] R. Brooksby. Dylan Requirements. MPS Project
many such ideas – the results of hard work by many people (see Documentation, October 1996.
11). Now, at last, we can reveal almost all. http://www.ravenbrook.com/project/mps/
The MPS is a highly portable, robust, extensible, and flexible master/mminfo/req/dylan/.
system for memory management, based on very powerful abstrac- [11] R. Brooksby. The architecture of the MPS. MPS Project
tions. It contains many more useful concepts and abstractions not Documentation, January 1997.
covered in this paper. http://www.ravenbrook.com/project/mps/
We hope that the ideas, techniques, and code of the MPS will master/mminfo/design/mps/arch/.
be useful. We also hope that companies will license the MPS or [12] CMMI Product Development Team. CMMISM for Systems
engage us to extend and develop it further. Engineering/Software Engineering, Version 1.02
(CMMI-SE/SW, V1.02) (Staged Representation). Technical
report, Software Engineering Institute, 2000.
11. ACKNOWLEDGEMENTS [13] E. W. Dijkstra, L. Lamport, A. J. Martin, C. S. Scholten, and
The authors would like to thank all the members of the Mem- E. F. M. Steffens. On-the-fly garbage collection: An exercise
ory Management Group for their contributions to the design and in cooperation. In Lecture Notes in Computer Science, No.
implementation of the MPS. 46. Springer-Verlag, New York, 1976.
The members of the Memory Management Group were: [14] R. K. Dybvig, C. Bruggeman, and D. Eby. Guardians in a
generation-based garbage collector. In Proceedings of
Name Period of membership
SIGPLAN’93 Conference on Programming Languages
Design and Implementation, volume 28(6) of ACM
Nick Barnes 1995-08/1997-11
SIGPLAN Notices, pages 207–216, Albuquerque, NM, June
Richard Brooksby 1994-02/1997-11
1993. ACM Press.
Nick “Sheep” Dalton 1997-04/1998-07
[15] T. Gilb and D. Graham. Software Inspection.
Lars Hansen 1998-06/1998-09
Addison-Wesley, 1995.
David Jones 1994-10/1999-06
Richard Kistruck 1998-02/1999-06 [16] H. Goguen, R. Brooksby, and R. M. Burstall. Memory
Tony Mann 1998-02/2000-02 management: An abstract formulation of incremental
Gavin Matthews 1996-08/1998-11 tracing. In Types for Proofs and Programs, International
David Moore 1995-04/1996-04 Workshop TYPES’99, pages 148–161. Springer, 2000.
Pekka P. Pirinen 1997-06/2001-04 [17] B. Hayes. Finalization of the collector interface. In
Richard Tucker 1996-10/1999-09 Y. Bekkers and J. Cohen, editors, Proceedings of
P. Tucker Withington 1994-02/1998-11 International Workshop on Memory Management, volume
637 of Lecture Notes in Computer Science, Stanford
12. REFERENCES
 University, USA, 16–18 Sept. 1992. Springer-Verlag.
[18] D. A. Moon. Garbage collection in a large LISP system. In
G. L. Steele, editor, Conference Record of the 1984 ACM
Symposium on Lisp and Functional Programming, pages
235–245, Austin, TX, Aug. 1984. ACM Press.
[19] P. P. Pirinen. Barrier techniques for incremental tracing. In
R. Jones, editor, ISMM’98 Proceedings of the First
International Symposium on Memory Management, volume
34(3) of ACM SIGPLAN Notices, pages 20–25, Vancouver,
Oct. 1998. ACM Press. ISMM is the successor to the IWMM
series of workshops.
[20] D. M. Ungar. Generation scavenging: A non-disruptive high
performance storage reclamation algorithm. ACM SIGPLAN
Notices, 19(5):157–167, Apr. 1984. Also published as ACM
Software Engineering Notes 9, 3 (May 1984) — Proceedings
of the ACM/SIGSOFT/SIGPLAN Software Engineering
Symposium on Practical Software Development
Environments, 157–167, April 1984.