.. HYSICAL .

REVIEW

VOLUME 74 2 JANUARY 1995 NUMBER 1

Monte Carlo Algorithms with Absorbing Markov Chains: Fast Local Algorithms for
Slow Dynamics
M. A. Novotny
Supercomputer Computations Research Institute, FIorida State University, Tallahassee, Florida 32306-4052
(Received 3 June 1994)
A class of Monte Carlo algorithms which incorporate absorbing Markov chains is presented. In
a particular limit, the lowest order of these algorithms reduces to the n-fold way algorithm. These
algorithms are applied to study the escape from the metastable state in the two-dimensional square-
lattice nearest-neighbor Ising ferromagnet in an unfavorable applied field, and the agreement with
theoretical predictions is very good. It is demonstrated that the higher-order algorithms can be many
orders of magnitude faster than either the traditional Monte Carlo or n-fold way algorithms.

PACS numbers: 02.70.Lq, 05.50.+q, 64.60.My, 75.40.Mg

Monte Carlo (MC) methods [1] have become indispen- In this Letter, a different class of accelerated MC
sible tools for nonperturbative calculations in numerous algorithms is presented that does not change the origi-
fields, including materials science, high-energy physics, nal MC dynamic and can therefore be used to simulate
chemistry, biology, engineering, and economics. These time series with very large separations in time scales.
methods are used for two fundamentally different pur- These algorithms incorporate into the standard MC al-
poses: to calculate time-independent quantities (statics) gorithm absorbing Markov chains (AMC). The acronym
and to simulate time series (dynamics). In the former MCAMC stands for Monte Carlo with absorbing Markov
case, the slow relaxation observed, e.g. , near phase tran- chains. With some additional approximations, the lowest-
sitions (critical slowing down) and at low temperatures, order MCAMC algorithm corresponds to the n-fold way
is merely a nuisance that has been overcome by a num- algorithm [7]. We demonstrate that MCAMC algorithms
ber of new MC algorithms, including cluster algorithms can be many orders of magnitude faster than either the
[2], vertex algorithms [3], multicanonical algorithms [4], standard MC or n-fold way algorithms.
and hybrid MC algorithms [5]. Such algorithms can be In a Markov process the probability of going from
many orders of magnitude faster than standard MC meth- the current state of the system to another state depends
ods. However, they all replace the standard MC dynamic only on these two states, not on the history of how the
with a different dynamic. Consequently, although such al- current state was reached. Standard MC algorithms are
gorithms may be very efficient in the calculation of static typical examples of Markov processes. A Markov process
quantities, information about the kinetics of the original with a finite state space and a discrete time step is a
MC dynamic cannot be obtained. There are many in- Markov chain. The Markov chain is often written as a
stances where the kinetics, rather than just the statics, is of Markov matrix M the elements of which are the transition
physical importance. Recently, MC methods using con- probabilities between the states. The time evolution of
strained cluster-Gipping algorithms have been proposed in the probability distribution vector vT is then given by
order to obtain information about the long-wavelength ki- v (m + 1) = vr(m)M. An AMC is one in which at least
netics of a system [6]. However, in such methods the local one state has the property that transitions out of that state
dynamic is modified, and universality arguments must be are forbidden.
made to relate the results to the dynamic of the original The Markov matrix associated with an AMC with q
system. absorbing states and s transient states is given by [8]

0031-9007/95/74(1)/1(5)$06. 00 1994 The American Physical Society

VOLUME 74, NUMBER 1 PH YS ICAL REVIEW LETTERS 2 JANUARY 1995

gXq QX5' a different AMC with different transient and absorbing

subspaces, and the process is iterated.
To illustrate the MCAMC algorithm, we apply it to
where I is the identity matrix, 0 is the zero matrix, and the
the square-lattice nearest-neighbor Ising ferromagnet in
subscripts show the size of each submatrix. The elements =
of the transient submatrix T are the transition probabilities
a magnetic field. The Hamiltonian is given by 9f
—1 P~;, s;s, —H g; s;, where the spins s; = ~ I. The
between the s transient states of M. The elements of the
sums run over a11 nearest-neighbor pairs and over all N =
&

submatrix R are the transition probabilities from each of

L sites, respectively. Periodic boundary conditions are
the s transient states to the q absorbing states. Starting
from an initial vector vI, the probability of still being in
any of the s transient states after m time steps is given
by J )
used. The isotropic two-body coupling constant is given
0 and the magnetic field by H. To study the decay
of a metastable phase, we apply a negative magnetic
by vI T e, where e is the vector with all elements equal
to 1. Although vI could be any normalized probability
field at a temperature below the critical temperature T„
and start with the configuration of all spins +1 (which
distribution over the transient states, in this work we will
we call the C+ configuration). Standard droplet theory
take vI to represent one particular transient state. It is
(for a recent review on metastability see Ref. [9]) shows
easy to show that one can obtain the time m for exiting that in order for the metastable phase C+ to decay,
to one of the q absorbing states from the solution of the one or more critical droplets must nucleate via the
equation random superposition of microscopic fluctuations. Since
the critical droplet is of a certain size, the average time
before its creation, and thus the metastable lifetime, is
where r is a random number uniformly distributed on
much longer than the microscopic time scales. Although
the interval (0, 1]. Note that the stochastic variable m
any local dynamic can be used within the MCAMC
is independent of which one of the q states the AMC
framework, in this Letter only Metropolis updates [1] are
ends up in. The elements of the vector of q unnormalized
performed. We measure the number of Monte Carlo steps
probabilities
per spin (MCSS) until a configuration is reached which
has an equal number of +1 and —1 spins. The average
Q =v, I+ gT" R, (3) lifetime 7- of the metastable state is found by averaging
),
I

, over a number of starts from the C+ configuration.

give the probability of exiting to each corresponding state, A standard MC algorithm randomly chooses a spin and
given that the system has exited from the transient sub- then decides whether or not to flip it. Each spin in the
space in m time steps. After normalization, this proba- L X L lattice, with periodic boundary conditions, is in
bility distribution can be used to pick a particular state one of 10 possible energy classes which are determined
into which the AMC exits. This is done by using another by how the spin is oriented with respect to the applied
uniformly distributed random number. Equations (2) and field and the nearest-neighbor spins [7]. The number of
(3) are the only equations needed to include absorbing spins in class i is n;, and the probability of flipping a spin
Markov chains within a Monte Carlo simulation. In prac- in class i once it has been chosen is p(i).
tice, it is computationally sound policy to obtain all the The s = 1 MCAMC algorithm is defined to have a
eigenvalues and eigenvectors of T and utilize the spectral single transient state, which is the current state of the
decomposition of T within the AMC portion of the algo- spin configuration. When spins belonging to each class
rithm to numerically solve Eqs. (2) and (3). are grouped together the submatrix Ri&&ip = (nip(1), . . . ,
The MCAMC algorithm has the following steps: nipp(10))/N. Define Qp = 0 and Q, = g';=& n;p(i).
(0) The system is in an initial state represented by vI . Since M is a Markov matrix, its row sums are unity,
(1) Write an s X s transient matrix that includes at least so the transient matrix is Ti&&i = 1 —Qip/N. The time
the initial configuration, but ideally will include other
states (which may represent many configurations of the
increment to flip a spin is found from Eq. (2) to be m
In(r)/ ln(1 —Qip/N) ~ m — 1. If one relaxes the con-
)
system that are related by symmetry). The additional straint that m is an integer, and expands to lowest order for
states included as transient states should be those the cur- small Qip/N, this equation becomes m = N ln(r)/Qip, —
rent configuration has the largest probability of exiting to. and one has the standard n-fold way algorithm [7]. In
(2) All configurations that the transient states can exit to both the s = 1 MCAMC and the n-fold way algorithm,
another random number is used to choose which of the 10
in one time step must be included in the absorbing states.
(3) Use Eq. (2) to find the time spent in the transient classes to pick via Q, i ( rQip ~ Q, , and a spin from
state subspace, and Eq. (3) to find a new configuration class j is randomly picked and flipped. As shown in
for the system. (4) Iterate the procedure using the new Fig. 1, at low temperatures the s = 1 MCAMC algorithm
configuration as the next initial configuration. Thus, once can be many orders of magnitude faster than the standard
the system has exited to one of the q absorbing states, MC algorithm. At J/T = 3 the speed improves by a
this decides the initial configuration of the system for factor of about 107.
VOLUME 74, NUMBER l PH YS ICAL REVIEW LETTERS 2 JANUARY 1995

The MCAMC algorithm offers a further advantage, in

~~, 020 102 that the statistical error in the average lifetime ~ is smaller
than from standard MC. The average lifetime of an
1015 1p80 AMC that starts from state k is given by vl, Ne, where
the fundamental matrix is defined by N = (I —T) ' [8].
g
C4 1010 1p15
For s ) l, the initial configuration in which the AMC
starts is C+ only once, since it must reenter C+ from
M a configuration C+ with one overturned spin. Since the
IX
105 1p10 M ~ contribution to the lifetime that comes from the escape
from C+ is exact, this reduces the total error in r.
.~ 1OP 1O5 It has been shown that at low enough temperatures 7.
is related to the height of the lowest energy barrier which
C4
must be reached by flipping one spin at a time starting
from C+ [10]. In the limit of low temperature and
large L, the discreteness of the lattice gives an important
FIG. 1. The average CPU time for escape from the metastable contribution to r. Theorem 3 of Ref. [10] states that then
state is shown as a function of the inverse temperature. This
is for a 24 X 24 lattice in a field of (H~//J = 0.75. For the
standard Monte Carlo algorithm (x) the CPU time is directly
proportional to the average lifetime ~ of the metastable state, (4)
and r (in units of Monte Carlo steps per spin) is plotted on the
right-hand axis. All values of ~ were calculated by averaging where 8, = [2J/~H~] and the notation [x] denotes the
over 10 escapes from the metastable state. The symbol x, smallest integer larger than x. This result, which is re-
with error estimates, for J/T ~ 1 is calculated from the values
of r from the MCAMC algorithms, while for J/T (
time for a standard Monte Carlo algorithm is plotted.
1 the CPU
The
stricted to 2J/~H) not an integer and to (H( ~ 4J, is
shown in Fig. 1 as a solid line. It is in qualitative agree-
CPU time from the n-fold way algorithm, which corresponds ment with the measured values of ~. A detailed com-
to s = 1 MCAMC, is given by the symbol O. The timings parison with predictions from Ref. [10] will be published
for s = 2 and s = 3 MCAMC algorithms are given by the elsewhere [11].
symbols + and, respectively. The vertical arrow marks the
Using similar reasoning, it is possible to estimate at low
exact critical temperature. The solid line is the low-temperature
discrete droplet result [10] for r with 4, = 3 from Eq. (4). temperatures the temperature dependence of the different
The dashed lines are estimates, described in the text, for the MCAMC algorithms. By assuming that most of the CPU
CPU times for the MCAMC algorithms. The horizontal arrows time is spent in the s = 1 portion of the algorithm, at low
indicate that the solid line and symbols x are related to both enough temperatures the CPU time should be proportional
vertical axes, whereas the other points and lines are only for the
left-hand axis. The curvature near J/T = 1 is near the value to exp[[1 (H, J) —I o(H, J)]/ksT). Here I o is 2J times
where H, ~2 = 3J/4 for this system size. Note the spectacular the surface area minus 2~H~ times the volume of the
increases in speed with the MCAMC algorithms. largest compact lattice animal included in T. The dashed
lines in Fig. 1 have these slopes and are drawn to go
At low temperatures in the s = 1 MCAMC algorithm, through the data point at the lowest temperature available
once a spin has fIipped from the C+ configuration, it for a particular algorithm.
is extremely probable that in the next iteration this spin Figure 2 shows values for r as a function of ~H~ for
will be chosen again to fIip, and one returns to the C+ two low temperatures. Note how well Eq. (4) fits the
configuration. This problem can be addressed by going results. Equation (4) is only valid if the nucleation of
to the s = 2 MCAMC algorithm. Whenever the starting a single droplet is responsible for flipping the lattice into
configuration is the C+ configuration or a configuration the stable phase. A reasonable estimate for the crossover
with only one spin overturned from the C+ configuration, field (which has been called the dynamic spinodal field
one includes these two states in the transient matrix T. [12]) out of the single-droplet regime is H~/2, the field at
The normal s = 1 MCAMC algorithm is used if the spin which the standard deviation of the lifetime is r/2 [12].
configuration is other than one of the transient states. As Figure 2 shows that this is a reasonable estimate for the
shown in Fig. 1, for J/T = 3 the speed improves by a point at which the results deviate from Eq. (4) at strong
factor of about 10 over the n-fold way algorithm. fields.
One can continue to increase the number of states in At higher temperatures, the discreteness of the lattice
T. Figure 1 also includes results from s = 3 MCAMC, becomes less important, and ~ should be given by the
where the three states in T were C+, C+ with one over- continuum droplet-theory prediction [13—16]
turned spin, and C+ with two nearest-neighbor overturned =
ln(r) (T)/(H[ —(b + c) ln fH[ + in[A(T)]. (5)
spins. At J/T = 3 this s = 3 MCAMC algorithm im-
proves the speed by a factor of about 102 compared to the Here (T) is related to the equilibrium zero-field sur-
s = 2 MCAMC algorithm. face tension, the equilibrium droplet shape [17], and the
VOLUME 74, NUMBER l PH YSICAL REVIEW LETTERS 2 JANUAR& l995
I I I I I I I I I 1
Q
20— I

1025—

1020
K f T=0.2J
15— $7=0.4J
M
CG 1015
t-

1010
10—
105—
O ()r)()

100
0
55m~
4
5
0
I. . . , I
()

r- —,
O
, I

FIG. 2. The average lifetime r, as a function of lHl, for FIG. 3. The temperature times the derivative of the logarithm
escape from the metastable state is shown for a 24 X 24 lattice of the average lifetime with respect to lH ' is shown as a
with T/J = 0.2 (T/T, =0,088. ) (x) and T/J = 0.4 (Q). Error function of lHl. This is for a 24 x 24 lattice with T = 0.2J
l

estimates are plotted for each point. The solid lines are the (x) and T = 0.4J (Q). The solid curves are from the low-
low-temperature discrete droplet estimates for r from Eq. (4). temperature predictions [10] from Eq. (4). Only results in
The diagrams show the nucleating droplet, reading from the the single droplet regime, lHl ~ H~t2, are shown. The dot-
strongest fields with 4,. of 1, 2, and 3. The dashed lines show dashed and dashed horizontal lines correspond to the exact
the boundary between the various 4,. regions. The vertical [17] zero-field value for (T)/T for T = 0.2J and T = 0 4J, .
arrows mark the crossover field H, t, [12], which separates the respectively. The dashed inclined straight line includes the
single-droplet regime (lHl ~ H~tq) and the multidroplet regime theoretical value b + c = 3 in Eq. (5) for T = 0.4J.
(Hit2 ~ IHl) for the two temperatures for this lattice size.

which are rapidly fluctuating, and major rearrangements

spontaneous magnetization. A(T) is a nonuniversal func- of spins occur very infrequently. Examples would be
tion. Field-theoretical and numerical calculations give the kinetics of spin-glass models, where the spins that
b = 1 [14, 16, 18]. For dynamics described by a Fokker- fluctuate rapidly are the ones that find themselves in a
Planck equation c = 2 [15,16], and a recent MC study is local environment with zero energy difference between
consistent with b + c = 3 [12]. the possible spin orientations. Another example is phase-
Differentiating Eqs. (4) and (5) with respect to lHl ordering kinetics in which a few particles undergo rapid
allows for direct comparisons with both the discrete random walks on long fiat portions of interfaces, while
droplet and continuum droplet predictions. This is shown the interfaces move extremely slowly. Similar reasoning
in Fig. 3. For T = 0.4J a crossover is observed at weak also applies to simulations of molecular-beam epitaxial
lHl to the continuum droplet prediction using b + c = 3 growth, where the n-fold way algorithm has been redis-
and the exact zero-field value for (T) [17]. At weak [Hl covered at least twice [23,24]. In all such cases, using
Fig. 3 shows that the T = 0.4J data are consistent with higher-order MCAMC algorithms should substantially de-
the theoretical predictions of Eq. (5) with the theoretical crease the CPU time required to obtain the static and dy-
values for (T) and b + c. namic information about the system without making any
MCAMC algorithms can be utilized to study other approximations to the dynamics.
interesting low-temperature effects in lattice-gas types The author wishes to thank J. Lee and P. A. Rikvold
of models. An example is the recent prediction that at for useful discussions and suggestions. This research was
low enough temperatures in the anisotropic Ising model supported in part by the Florida State University Super-
nucleation occurs through square droplets, rather than computer Computations Research Institute, which is par-
through rectangular equilibrium Wulff droplets [19]. tially funded through Contract No. DE-FC05-85ER25000
Additional applications of MCAMC algorithms may by the U. S. Department of Energy.
include studies of equilibrium properties of systems where
the n-fold way algorithm has been utilized. These include
the anisotropic Ising model [20] (which is related via
[1] K. Binder and D. Stauffer, in Applications of the Monte
a Trotter-Suzuki decomposition to a quantum model
Carlo Method in Statisticah Physics, edited by K. Binder
in one lower dimension [21]), and simulated annealing in Topics in Current Physics, Vol. 36, (Springer-Verlag,
approaches to minimization [22]. Berlin, 1984).
However, the real strength of MCAMC algorithms lies [2] For a review see, A. D. Sokal, in Lattice '90 [Nucl. Phys.
in the study of slow dynamics in models with a limited B (Proc. Suppl. ) 20, 216 (1991)].
number of local states. MCAMC algorithms will be par- [3] H. G. Evertz, G. Lana, and M. Marcu, Phys. Rev. Lett. 70,
ticularly useful when there exists a small number of spins 875 (1993).
VOLUME 74, NUMBER j, PH YS ICAL RE VIE% LETTERS 2 JANUARY 1995

[4] B. A. Berg and T. Celik, Phys. Rev. Lett. 69, 2292 (1992). [14] J. S. Langer, Ann. Phys. (N. Y.) 41, 108 (1967).
[5] S. Duane, A. D. Kennedy, B.J. Pendleton, and D. Roweth, [15] J. S. Langer, Ann. Phys. (N. Y.) 54, 258 (1969).
Phys. Lett. B 195, 216 (1986). [16] N. J. Giinther, D. A. Nicole, and D. J. Wallace, J. Phys. A
[6] G. T. Barkema and J. F. Marko, Phys. Rev. Lett. 71, 2070 13, 1755 (1980).
(1993). [17] R. K. P. Zia and J. E. Avron, Phys. Rev. B 46, 8886
[7] A. B. Bortz, M. H. Kalos, and J. L. Lebowitz, J. Comput. (1992).
Phys. 17, 10 (1975). [18] C. C. A. Giinther, P. A. Rikvold, and M. A. Novotny,
[8] M. Iosifescu, Finite Markov Processes and Their Appli Phys. Rev. Lett. 71, 3898 (1993).
cation (John Wiley R Sons, New York, 1980), Chap. 3. [19] R. Kotecky and E. Olivieri, J. Stat. Phys. 70, 1121 (1993).
[9] P. A. Rikvold and B. M. Gorman, in Annual Revt'ebs [20] T. Graim and D. P. Landau, Phys. Rev. B 24, 5156 (1981).
of Computational Physics, edited by D. Stauffer (World [21] M. Suzuki, in Quantum Monte Carlo Methods in Con
Scientific, Singapore, 1994), Vol. l. densed Matter Physics, edited by M. Suzuki (World Sci-
[10] E. Jordao Neves and R. H. Schonmann, Commun. Math. entific, Singapore, 1993).
Phys. 137, 209 (1991). [22] J. W. Greene and K. J. Supowit, IEEE Trans. Computer-
[11) M. A. Novotny (unpublished). Aided Design CAD-5, 221 (1986).
[12] P. A. Rikvold, H. Tomita, S. Miyashita, and S. W. Sides, [23] G. H. Gilmer, J. Cryst. Growth 35, 15 (1976).
Phys. Rev. E 49, 5080 (1994). [24] P. A. Maksym, Semicond. Sci. Technol. 3, 594 (1988).
[13] M. E. Fisher, Physics (Long Island City) 3, 255 (1967).