Chapter 2

Thermodynamics and statistical

mechanics
Molecular simulations can be used to predict the structural, dynamical, and
thermodynamic properties of a classical many-body system. The number of
properties that can be computed once we know the natural time evolution of
an N-particle system is effectively unlimited, but not all quantities that can be
computed in a simulation correspond to properties that are experimentally acces-
sible. To give a specific example: in a Molecular Dynamics simulation of liquid
water, we could compute the time evolution of the positions and velocities of all
molecules in the liquid. However, such information cannot be compared with
experiments, because no known experimental technique can provide us with
such detailed information. Rather, typical experiments sample properties that
are averaged over a large number of particles and usually also averaged over the
time of the measurement. This feature of experiments is no accident: measure-
ments should yield outcomes that are reproducible by repeating the experiment
many times, or by running it for a long time. If we wish to use computer sim-
ulation as the numerical counterpart of experiments,1 we must know what kind
of computational measurements yield reproducible outcomes.
As in experiments, the reproducible properties of a model system depend on
a small number of macroscopic control variables that characterize the macro-
scopic state in which the system was prepared: e.g., its energy, volume, and
number of particles. The theoretical framework of thermodynamics provides us
with rules that relate the variation of macroscopic observables with the change
in the “thermodynamic” control parameters such as volume or applied exter-
nal fields. Different sets of thermodynamic parameters correspond to different
experiments, e.g., we can either consider a system at fixed density or at fixed
pressure. Hence, before we start our discussion of simulation methods, we need
to introduce the language of thermodynamics. We also need the language of sta-
tistical mechanics, which provides the link between the microscopic state of a
system and its macroscopic properties.
As this is not a book about either of these subjects, our introduction to ther-
modynamics and statistical mechanics will be oversimplified and should be seen

1 Simulations can also be used to test theoretical predictions, but such predictions also tend to
focus on reproducible properties, even though they may not always correspond to experimentally
accessible quantities.

Understanding Molecular Simulation. https://doi.org/10.1016/B978-0-32-390292-2.00010-6 11

Copyright © 2023 Elsevier Inc. All rights reserved.
12 PART | I Basics

either as a refresher, or as an incentive to learn more about the subject from the
many excellent textbooks.
In what follows, we present a brief overview of classical thermodynamics
and a quick (and slightly dirty) derivation of the basic expressions of statistical
mechanics. The aim of these derivations is only to show that there is nothing
mysterious about concepts such as phase space, temperature, entropy, and many
of the other statistical mechanical objects that will appear time and again in the
remainder of this book.
If you are familiar with thermodynamics, you may wish to skip the next
section. But remember the quote attributed to Sommerfeld:
“Thermodynamics is a funny subject. The first time you go through it, you don’t
understand it at all. The second time you go through it, you think you understand
it, except for one or two points. The third time you go through it, you know you
don’t understand it, but by that time you are so used to the subject, it doesn’t bother
you anymore.”

2.1 Classical thermodynamics

Thermodynamics is difficult because it seems so abstract. However, we should
always bear in mind that thermodynamics is based on experimental observa-
tions. For instance, the First Law of thermodynamics expresses the empirical
observation that energy is conserved, even though it can be converted into vari-
ous forms. The internal energy of a system can be changed by performing work
w on the system or by transferring an amount of heat q. It is meaningless to
speak about the total amount of heat in a system, or the total amount of work.
This is not mysterious: it is just as meaningless to speak about the number of
train-travelers and the number of pedestrians in a train-station: people enter the
station as pedestrians, and exit as train-travelers or vice versa. However, if we
add the sum of the changes in the number of train-travelers and pedestrians, then
we obtain the change in the number of people in the station. And this quantity is
well-defined. Similarly, the sum of q and w is equal to the change in the internal
energy E of the system
dE = q + w. (2.1.1)
This is the First Law of thermodynamics.
The Second Law seems more abstract, but it is not. The Second Law is
based on the experimental observation that it is impossible to make an engine
that works by converting heat from a single heat bath (i.e., a large reservoir
in equilibrium) into work. This observation is equivalent to another —equally
empirical— observation, namely that heat can never flow spontaneously (i.e.,
without performing work) from a cold reservoir to a warmer reservoir. This
statement is a bit more subtle than it seems because, before we have defined
temperature, we can only distinguish hotter and colder by looking at the di-
rection of heat flow. What the Second Law says is that it is never possible to
 Thermodynamics and statistical mechanics Chapter | 2 13

sustain a spontaneous heat flow in the “wrong” direction (e.g., if heat can flow
spontaneously from system A to system B, and from system B to system C,
then heat cannot flow spontaneously from C to A). How do we get from such
a seemingly trivial statement to something as abstract as entropy? This is most
easily achieved by introducing the concept of a reversible heat engine, i.e., an
engine that is completely free of internal dissipative energy losses.
A reversible engine is, as the word suggests, an engine that can be operated
in reverse. During one cycle (a sequence of steps that is completed when the
engine is returned to its original state) this engine takes in an amount of heat q1
from a hot reservoir, converts part of it into work w, and delivers a remaining
amount of heat q2 to a cold reservoir. The reverse process is that, by performing
an amount of work w, we can take an amount of heat q2 from the cold reservoir
and deliver an amount of heat q1 to the hot reservoir. Reversible engines are an
idealization because in any real engine, there will be additional frictional losses.
However, the ideal reversible engine can be approximated arbitrarily closely by
a real engine if, at every stage, the real engine is sufficiently close to equilibrium.
As the engine is returned to its original state at the end of one cycle, its internal
energy E has not changed. Hence, the First Law tells us that

E = q1 − (w + q2 ) = 0, (2.1.2)

or
q1 = w + q2 . (2.1.3)
Now consider the “efficiency”, η, of the engine η ≡ w/q1 —i.e., the amount
of work delivered per amount of heat taken in. At first, one might think that
η depends on the precise design of our reversible engine. However, this is not
true. η is the same for all reversible engines operating between the same two
reservoirs. To demonstrate this, we show that if different engines could have
different values for η then we would contradict the Second Law in its form
“heat can never spontaneously flow from a cold to a hot reservoir”. Suppose,
therefore, that we have another reversible engine that takes in an amount of heat
q1 from the hot reservoir, delivers the same amount of work w, and then delivers
an amount of heat q2 to the cold reservoir. Let us denote the efficiency of this
engine by η . Now, we use the work generated by the engine with the highest
efficiency (say η) to drive the second engine in reverse. The amount of heat
delivered to the hot reservoir by the second engine is

q1 = w/η = q1 (η/η ), (2.1.4)

where we have used w = q1 η. As, by assumption, η < η it follows that q1 >
q1 . Hence there would be a net heat flow from the cold reservoir into the hot
reservoir. But this contradicts the Second Law of thermodynamics. Therefore
we must conclude that the efficiency of all reversible heat engines operating
between the same reservoirs is identical. The efficiency only depends on the
14 PART | I Basics

temperatures t1 and t2 of the reservoirs (the temperatures t could be measured

in any scale, e.g., in Fahrenheit or Réaumur, as long as it is single-valued. As
η(t1 , t2 ) depends only on the temperature in the reservoirs, then so does the
ratio q2 /q1 = 1 − η. Let us call this ratio R(t2 , t1 ). Now suppose that we have
a reversible engine that consists of two stages: one working between reservoir
1 and 2, and the other between 2 and 3. In addition, we have another reversible
engine that works directly between 1 and 3. As both engines must be equally
efficient, it follows that

R(t3 , t1 ) = R(t3 , t2 )R(t2 , t1 ). (2.1.5)

This can only be true in general if R(t1 , t2 ) is of the form

f (t2 )
R(t2 , t1 ) = , (2.1.6)
f (t1 )

where f (t) is an, as yet unknown function of our measured temperature. What
we do now is to introduce an “absolute” or thermodynamic temperature T given
by
T ≡ f (t). (2.1.7)
Then, it immediately follows that

q2 T2
= R(t2 , t1 ) = . (2.1.8)
q1 T1
Note that the thermodynamic temperature could just as well have been de-
fined as c × f (t). In practice, c has been fixed such that, around room temper-
ature, 1 degree in the absolute (Kelvin) scale is equal to 1 degree Celsius. But
that choice is, of course, purely historical and —as it will turn out later— a bit
unfortunate.
Why do we need all this? We need it to introduce entropy, the most mys-
terious of all thermodynamic quantities. To do so, note that Eq. (2.1.8) can be
written as
q1 q2
= , (2.1.9)
T1 T2
where q1 is the heat that flows in reversibly at the high temperature T1 , and q2
is the heat that flows out reversibly at the low temperature T2 . We see there-
fore that, during a complete cycle, the difference between q1 /T1 and q2 /T2 is
zero. Recall that, at the end of a cycle, the internal energy of the system has not
changed. Now Eq. (2.1.9) tells us that there is also another quantity, denoted by
S, which is unchanged when we restore the system to its original state. Follow-
ing Clausius, we use the name “entropy” for S.
In Thermodynamics, quantities such as S that are unchanged if we return a
system to its original state, are called state functions. We do not know what S is,
 Thermodynamics and statistical mechanics Chapter | 2 15

but we do know how to compute its change. In the above example, the change in
S was given by S = (q1 /T1 ) − (q2 /T2 ) = 0. In general, the change in entropy
of a system due to the reversible addition of an infinitesimal amount of heat
δqrev from a reservoir at temperature T is
δqrev
dS = . (2.1.10)
T
We also note that S is extensive. That means that the total entropy of two
non-interacting systems, is equal to the sum of the entropies of the individual
systems. Consider a system with a fixed number of particles2 N and a fixed vol-
ume V . If we transfer an infinitesimal amount of heat δq to this system, then the
change in the internal energy of the system, dE is equal to δq. Hence,
 
∂S 1
= . (2.1.11)
∂E V ,N T

The most famous, though not the most intuitively obvious, statement of the Sec-
ond Law of Thermodynamics is that A spontaneous change in a closed system,
i.e., a system that exchanges neither energy nor particles with its environment,
can never lead to a decrease of the entropy. Hence, in equilibrium, the entropy of
a closed system is at a maximum. The argument behind this sweeping statement
is simple: consider a system with energy E, volume V and number of particles
N that is in equilibrium. We denote the entropy of this system by S0 (E, V , N ).
In equilibrium, all spontaneous changes that can happen, have happened. Now
suppose that we want to change something in this system —for instance, we
increase the density in one half of the system and decrease it in the other half.
As the system was in equilibrium, this change does not occur spontaneously.
Hence, in order to realize this change, we must perform a certain amount of
work, w (for instance, by placing a piston in the system and moving it). We
assume that this work is performed reversibly in such a way that E, the total
energy of the system, stays constant (and also V and N ). The First Law tells us
that we can only keep E constant if, while doing the work, we allow an amount
of heat q, to flow out of the system, such that q = w. Eq. (2.1.10) tells us that
when an amount of heat q flows reversibly out of the system, the entropy S of
the system must decrease. Let us denote the entropy of this constrained state by
S1 (E, V , N ) < S0 (E, V , N ). Having completed the change in the system, we
insulate the system thermally from the rest of the world, and we remove the
constraint that kept the system in its special state (taking the example of the
piston: we make an opening in the piston). Now the system goes back spon-
taneously (and irreversibly) to equilibrium. However, no work is done, and no
2 Thermodynamics does not assume anything about atoms or molecules. In thermodynamics, we
would specify the total mass (a macroscopic quantity) of a given species in the system, rather than
the number of particles. But when we talk about Statistical Mechanics or simulation techniques, we
will always specify the amount of matter in a system by the number of molecules. By doing the
same while discussing thermodynamics, we can keep the same notation throughout.
16 PART | I Basics

heat is transferred. Hence the final energy E is equal to the original energy (and
V and N are also constant). This means that the system is now back in its orig-
inal equilibrium state and its entropy is once more equal to S0 (E, V , N ). The
entropy change during this spontaneous change is equal to S = S0 − S1 . But,
as S1 < S0 , it follows that S > 0. As this argument is quite general, we have
indeed shown that any spontaneous change in a closed system leads to an in-
crease in the entropy. Hence, in equilibrium, the entropy of a closed system is at
a maximum.
We can now combine the First Law and the Second Law to arrive at an
expression for the energy change of a thermodynamic system. We consider an
infinitesimal reversible change in the system due to heat transfer and work. The
First Law states:
dE = q + w.
For a reversible change, we can write q = T dS. In the case of w, there are many
ways in which work can be performed on a system, e.g., compression, electrical
polarization, magnetic polarization, elastic deformation, etc. Here we will focus
on one such form of work, namely work due to a volume change against an
external pressure P . In that case, the work performed on the system during an
infinitesimal volume change dV is w = −P dV , and the First Law can be written
as
dE = T dS − P dV .
However, there is another, important way in which we can change the energy
of a system, namely by transferring matter into or out of the system. For conve-
nience, we consider a system containing only one species. As before, the number
of molecules of this species is denoted by N . As we (reversibly) change the num-
ber of molecules in the system at constant V and S, the energy of the system
changes: dE = μdN . This expression defines the constant of proportionality, μ,
the “chemical potential”
 
∂E
μ≡ . (2.1.12)
∂N S,V
We note that, for consistency with the rest of the book, we have defined the
chemical potential in terms of the number of molecules. In classical thermody-
namics, the chemical potential is defined as
 
∂E
μ thermo
≡ , (2.1.13)
∂n S,V

where n denotes the number of moles. The relation between the thermodynamic
μthermo and the molecular μ is simple:

μthermo = NA μ,
 Thermodynamics and statistical mechanics Chapter | 2 17

where NA denotes Avogadro’s number. The generalization of Eq. (2.1.12) to

multi-component systems is straightforward and will be encountered later. We
can now write down the most frequently used form of the First Law of Thermo-
dynamics
dE = T dS − P dV + μdN . (2.1.14)
Often we use Eq. (2.1.14) in the form
1 P μ
dS = dE + dV − dN , (2.1.15)
T T T
which implies that:
 
∂S P
=
∂V E,N T
and
 
∂S μ
=− .
∂N E,V T
We already knew that
 
∂S 1
= .
∂E V ,N T
It is important to make a distinction between thermodynamic properties that
are extensive and those that are intensive. Intensive properties do not depend
on the size of the system under consideration. Examples are temperature, pres-
sure, and chemical potential. If we combine two identical systems in the same
thermodynamic state, then the temperature, pressure, and chemical potential of
the resulting system are the same as that of the constituent systems. In contrast,
energy, entropy, volume, and number of particles are extensive. This means that
they scale with the system size. Now assume that we construct a thermodynamic
system by combining a large number of infinitesimal systems. Then the exten-
sivity of E, S, V , and N implies that, for the resulting system, we have

E = T S − P V + μN . (2.1.16)

Now consider a small variation in E:

dE = dT S − dP V + dμN = T dS + SdT − P dV − V dP + μdN + N dμ .

If we combine this with the First Law of thermodynamics, we find:

0 = SdT − V dP + N dμ. (2.1.17)

This is an important relation because it shows that T , P , and μ are dependent

variables. Two of them suffice to specify the thermodynamic state of the system.
18 PART | I Basics

However, in addition, we always need (at least) one extensive thermodynamic

variable to specify the size of the system: T , P , and μ are intensive and therefore
they do not contain that information.
From this point on, we can derive all of thermodynamics, except one law: the
so-called Third Law of Thermodynamics. The Third Law can be formulated in a
number of ways. The shortest version states that the entropy of the equilibrium
state of a pure substance at T = 0 is equal to zero. However, the fact that the
value of the entropy at T = 0 must be zero follows from considerations outside
thermodynamics. The Third Law is not as “basic” as the First and the Second,
and, anyway, we shall soon get a more direct interpretation of its meaning.

2.1.1 Auxiliary functions

Eq. (2.1.14) expresses the First Law of thermodynamics as a relation between
the variations in E with those in S, V , and N . Sometimes, it is more convenient
to use other independent variables, e.g., the temperature instead of the entropy,
the pressure instead of the volume, or the chemical potential instead of the num-
ber of particles. “Convenient” variables are those that can be controlled in a
given experiment. There is a simple procedure to recast the First Law in terms
of these other variables.

Enthalpy
For instance, if we use S, P , and N as the independent variables we can carry
out a so-called Legendre transform, which allows us to replace the energy with
a new state function that is a function of S, P , and N . This function is called the
Enthalpy (H ), defined as H ≡ E + P V . Clearly

dH = dE + dP V = T dS − P dV + μdN + P dV + V dP = T dS + V dP + μdN,
(2.1.18)
showing that the independent variables controlling the enthalpy are S, P , and N .

Helmholtz free energy

Similarly, we can introduce a function F , called the Helmholtz free energy, de-
fined as F ≡ E − T S. As in the case of the enthalpy, it is easy to show that

dF = −SdT − P dV + μdN. (2.1.19)

Gibbs free energy

The Gibbs free energy G is defined as F + P V , and it satisfies

dG = −SdT + V dP + μdN. (2.1.20)

 Thermodynamics and statistical mechanics Chapter | 2 19

FIGURE 2.1 An isolated system consisting of two boxes 1 and 2 that each have a fixed volume, the
two subsystems can exchange heat but the subsystem 2 is much larger than system 1 and therefore
acts as a heat bath.

Grand Potential
Finally, we can introduce the Grand Potential, , which is  ≡ F − μN , satis-
fying

d = −SdT − P dV − N dμ. (2.1.21)

However, for homogeneous systems, we rarely use the symbol  for the Grand
Potential, because, if the pressure of a system is well-defined, F − μN = −P V ,
we can replace  by −P V .3

Auxiliary functions and the Second Law

For a closed N -particle system with a given internal energy E and volume V ,
equilibrium is reached when the entropy S is at a maximum. To describe ex-
periments at conditions other than constant N , V , E, we must reformulate the
Second Law of thermodynamics, because, in general, S will not be at a max-
imum in equilibrium if we hold P or T constant. Fortunately, we can use the
original Second Law of thermodynamics to derive the condition for thermody-
namic equilibrium under conditions other than constant E, V , and N .
Let us consider the system shown in Fig. 2.1. The total system is isolated
and has a fixed volume. In the system, we have a subsystem 1, which is much
smaller than subsystem 2 (we will refer to the larger system as the “heat bath”).
If we allow subsystems 1 and 2 to exchange energy, the total energy of the
combined system is still conserved. Hence, we can apply the Second Law of
thermodynamics to the combined system, i.e., the total entropy of the combined
system must be at a maximum in equilibrium. As long as there is a net energy
flux between systems 1 and 2, the total system is not yet in equilibrium and we

3 The pressure is not a convenient variable for describing the state of confined or porous systems.
For such systems, it is best to use  ≡ F − μN .
20 PART | I Basics

must have
Stot = S1 + S2 ≥ 0. (2.1.22)
As the heat bath (subsystem 2) is much larger than subsystem 1, its tem-
perature T will not change when a small amount of energy is exchanged with
subsystem 1. Using dS = δqrev /T (Eq. (2.1.10)), we can then write:
E2
S2 = . (2.1.23)
T
As the total energy is conserved, (i.e., E2 = −E1 ), we can then write:
1
S1 + S2 = (T S1 − E1 ) ≥ 0. (2.1.24)
T
This equation expresses the condition for equilibrium in terms of the properties
of subsystem 1. The only effect of the heat bath is that it imposes the temperature
T . Note that the quantity that is varied in Eq. (2.1.24) is nothing else than the
Helmholtz free energy (see Eq. (2.1.19))
F1 (N, V , T ) ≡ E1 − T S1 .
Then the Second Law, in the form given by Eq. (2.1.24), implies that, for a
system in contact with a heat bath,
1
− F1 ≥ 0. (2.1.25)
T
In other words: when an N-particle system in a volume V is in contact with a
heat bath at a (positive) temperature T , then a spontaneous change can never
increase its Helmholtz free energy:
dF ≤ 0. (2.1.26)
Similarly, we can define two systems that can not only exchange of energy,
but also change their volumes in such a way that the total volume remains con-
stant (see Fig. 2.2). As before, the combined system is isolated and its total
volume is fixed. Then we can again apply the Second Law of thermodynam-
ics to the combined system. As system 2 is again assumed to be much larger
than system 1, its temperature and pressure do not change when the energy and
volume of system 1 change. The entropy change of system 2 is therefore given
by
E2 P V2
S2 = + . (2.1.27)
T T
As the total energy is conserved (E2 = −E1 ) and the total volume remains
constant (V2 = −V1 ), we can write for the total entropy change:
1
S1 + S2 = (T S1 − E1 − P V1 ) ≥ 0, (2.1.28)
T
 Thermodynamics and statistical mechanics Chapter | 2 21

FIGURE 2.2 An isolated system consisting of two boxes 1 and 2 that can exchange heat and
change their volume in such a way that the total volume and energy remains constant. System 2 is
much larger compared to system 1 so it can act as a heat bath that exerts a constant pressure on
system 1.

or (dG)/T ≤ 0, where dG is the variation in the Gibbs free energy (Eq. (2.1.20)).
In this equation, we have again expressed the inequality in terms of the proper-
ties of system 1 only. A spontaneous change in a system at constant temperature
and pressure can never increase its Gibbs free energy.

2.1.2 Chemical potential and equilibrium

Thus far, we have considered systems in contact with a reservoir at constant
temperature and constant pressure. Let us now consider what happens if we
bring a system in contact with a “particle reservoir,” i.e., a system at constant
chemical potential. Clearly, as T , P , and μ are linearly dependent, we cannot
consider a system in contact with a reservoir at constant T , P , μ, because these
variables are not enough to fix the size of the system. Hence, when considering
a system in contact with a particle reservoir, we should fix at least one extensive
variable. The most convenient choice is to fix the volume V . So, we will consider
a system (1) of volume V in contact with a reservoir (system 2) at constant T
and μ. As before, we can use the Second Law of thermodynamics as applied
to the combined system, to derive the equilibrium condition for system 1 under
conditions where this system can exchange heat and particles with the reservoir
(system 2):

E1 μN1
Stot = S1 + S2 = S1 − + ≥0 (2.1.29)
T T
or
 (T S1 − E1 + μN1 )
Stot = ≥ 0, (2.1.30)
T
or, using Eq. (2.1.21),  ≤ 0. Hence, at constant T , V , and μ,  is at a mini-
mum in equilibrium.
22 PART | I Basics

We have now obtained the conditions for equilibrium for some of the condi-
tions of the greatest practical importance, namely:

Equilibrium at constant N, V, E : S is maximal

Equilibrium at constant N, V, T : F is minimal
Equilibrium at constant N, P, T : G is minimal (2.1.31)
Equilibrium at constant μ, V, T :  is minimal

Using the definitions of F , G, and , we can write down expressions for an

infinitesimal variation in each of these quantities. For instance:

dF = dE − dT S.

Using the First Law, we can rewrite this as

dF = T dS − P dV + μdN − T dS − SdT = −SdT − P dV + μdN.

Similarly, we can write:

1 P μ
dS = dE + dV − dN
T T T
dF = −SdT − P dV + μdN
dG = −SdT + V dP + μdN
d = −SdT − P dV − N dμ . (2.1.32)

In what follows, we will replace  by −P V , unless explicitly stated otherwise.

Eq. (2.1.32), in combination with the equilibrium conditions (2.1.31), are ex-
tremely important because they specify the conditions under which a system is
in thermodynamic (chemical or phase) equilibrium. Later, we shall make exten-
sive use of the conditions for phase equilibrium.
Next, consider a closed system containing two subsystems. The total volume
of the system V = V1 + V2 is fixed. Similarly, N1 + N2 and E1 + E2 are fixed.
These conditions imply that dV1 = −dV2 , dN1 = −dN2 and dE1 = −dE2 . The
Second Law tells us that, in equilibrium, the total entropy of the system Stot =
S1 + S2 , must be an extremum (note that Stot is not fixed). Hence, the derivatives
of Stot with respect to E1 , N1 and V1 must vanish, i.e.:
     
∂ (S1 + S2 ) ∂S1 ∂S2
= − =0
∂E1 ∂E1 ∂E2
     
∂ (S1 + S2 ) ∂S1 ∂S2
= − =0
∂V1 ∂V1 ∂V2
     
∂ (S1 + S2 ) ∂S1 ∂S2
= − =0. (2.1.33)
∂N1 ∂N1 ∂N2
 Thermodynamics and statistical mechanics Chapter | 2 23

If we combine Eq. (2.1.33) with the expression for dS, Eq. (2.1.32), we obtain
1 1
=
T1 T2
P1 P2
=
T1 T2
μ1 μ2
= . (2.1.34)
T1 T2
The first condition implies thermal equilibrium between the two systems, i.e.,
T1 = T2 ≡ T . Then the second condition simply implies P1 = P2 , and the third
is μ1 = μ2 .4 Eq. (2.1.34) is the starting point for all free-energy-based calcula-
tions to locate the point where two systems (two phases) are in equilibrium (see
Chapter 8).

2.1.3 Energy, pressure, and chemical potential

One of the aims of molecular simulations is to compute the thermodynamic
properties of a system based on our knowledge of the interactions between
constituent molecules. In subsequent chapters, we discuss how the relevant ex-
pressions can be derived from Statistical Mechanics. Here we focus on one
general feature: in all cases, we will start from the thermodynamic definitions
of the quantities to be computed. For instance, the starting point for computing
the pressure, P , is a thermodynamic relation of the type:
 
∂F
P =− . (2.1.35)
∂V N,T

This is the expression that we would use if we consider a system at fixed N , V ,

and T . However, if we would consider a system at constant N , V and E, we
would use:
 
P ∂S
= .
T ∂V N,E
Analogous expressions can be written down for other thermodynamic variables
and other conditions. For instance, for a system at constant N , V and T , the
energy is given by the following thermodynamic relation:
 
∂F
E =F +TS =F −T
∂T V ,N
 
∂F /T
= . (2.1.36)
∂1/T V ,N

4 For a multicomponent mixture, the chemical potential μα of each component α in the mixture
must be equal in the two subsystems: μα1 = μα2 .
24 PART | I Basics

We can also obtain the chemical potential, μ, from F using:

 
∂F
μ= . (2.1.37)
∂N T ,V

As most Monte Carlo simulations are carried out at constant N , V , and T , we

will use these relations extensively.

Relations between partial molar derivatives

All extensive thermodynamic variables X of a multi-component system can be
written as:

X= Ni xi ,
i

where xi is the partial derivative of the quantity X with respect to Ni , the number
of particles of species i, at constant P , T , and Nj :
 
∂X
xi ≡ .
∂Ni P ,T ,{Nj }

For instance, in the case that X = S. Then


S= Ni si ,
i

where, we have for the molar entropy of component i, s i

 
∂S
si ≡ .
∂Ni P ,T ,{Nj }

It is important to note that it makes a difference what thermodynamic variables

we keep constant. Clearly,
 
∂S
si = = μi /T .
∂Ni E,V ,{Nj }

But there is no inconsistency: when we add a particle of species i at constant

P and T , the internal energy changes by an amount ei and the volume by an
amount v i . To reduce the system to constant energy and volume, we should
therefore compute
       
∂S ∂S ∂S ∂S
= − vi − ei
∂Ni E,V ,{Nj } ∂Ni P ,T ,{Nj } ∂V E,{N } ∂E V ,{N }
P vi ei
= si − −
T T
 Thermodynamics and statistical mechanics Chapter | 2 25

 
T si − P vi − ei μi 1 ∂G
= =− =− .
T T T ∂Ni P ,T ,{Nj }

2.2 Statistical thermodynamics

In the previous section, we introduced the framework of thermodynamics. Ther-
modynamics is a phenomenological theory: it provides relations between exper-
imentally observable quantities. However, it does not predict these quantities
on the basis of a microscopic model. Statistical Mechanics provides the link be-
tween the microscopic description of a system of interacting atoms or molecules
and the prediction of thermodynamical observables such as pressure or chemical
potential. For all but the simplest systems, the statistical mechanical expressions
for the thermodynamical observables are too complex to be evaluated analyti-
cally. However, in many cases, numerical simulations will allow us to obtain
accurate estimates of the quantities of interest.

2.2.1 Basic assumption

Most of the computer simulations that we discuss are based on the assump-
tion that classical mechanics can be used to describe the motions of atoms and
molecules. This assumption leads to a great simplification in almost all calcu-
lations, and it is therefore most fortunate that it is justified in many cases of
practical interest. Surprisingly, it turns out to be easier to derive the basic laws
of statistical mechanics using the language of quantum mechanics. We will fol-
low this route of least resistance. In fact, for our derivation, we need only little
quantum mechanics. Specifically, we need the fact that a quantum mechanical
system can be found in different states. For the time being, we limit ourselves
to quantum states that are eigenvectors of the Hamiltonian H of the system (i.e.,
energy eigenstates). For any such state |i, we have that H|i = Ei |i, where
Ei is the energy of state |i. Most examples discussed in quantum mechanics
textbooks concern systems with only a few degrees of freedom (e.g., the one-
dimensional harmonic oscillator or a particle in a box). For such systems, the
degeneracy of energy levels will be small. However, for the systems that are of
interest to statistical mechanics (i.e., systems with O(1023 ) particles), the degen-
eracy of energy levels is super-astronomically large. In what follows, we denote
by  the number of eigenstates with energy E of a system of N particles in a
volume V ,  = (E, V , N ). We now express the basic assumption of statisti-
cal mechanics as follows: a system with fixed N , V , and E is equally likely to
be found in any of its (E) eigenstates. Much of statistical mechanics follows
from this simple (but nontrivial) assumption.
To see this, let us again consider a system with total energy E that consists
of two weakly interacting subsystems. In this context, weakly interacting means
that the subsystems can exchange energy but that we can write the total energy
of the system as the sum of the energies E1 and E2 of the subsystems. There are
26 PART | I Basics

many ways in which we can distribute the total energy over the two subsystems
such that E1 + E2 = E. For a given choice of E1 , the total number of degenerate
states of the system is 1 (E1 ) × 2 (E2 ). Note that the total number of states is
the product of the number of states in the individual systems. In what follows,
it is convenient to have a measure of the degeneracy of the subsystems that is
additive. A logical choice is to take the (natural) logarithm of the degeneracy.
Hence:
ln (E1 , E − E1 ) = ln 1 (E1 ) + ln 2 (E − E1 ). (2.2.1)
We assume that subsystems 1 and 2 can exchange energy. What is the most
likely distribution of the energy? We know that every energy state of the total
system is equally likely. But the number of eigenstates corresponding to a given
distribution of the energy over the subsystems depends very strongly on the
value of E1 . We wish to know the most likely value of E1 , that is, the one that
maximizes ln (E1 , E − E1 ). The condition for this maximum is that
 
∂ ln (E1 , E − E1 )
=0 (2.2.2)
∂E1 N,V ,E

or, in other words,

   
∂ ln 1 (E1 ) ∂ ln 2 (E2 )
= . (2.2.3)
∂E1 N1 ,V1 ∂E2 N2 ,V2

We introduce the shorthand notation

 
∂ ln (E, V , N )
β(E, V , N) ≡ . (2.2.4)
∂E N,V

With this definition, we can write Eq. (2.2.3) as

β(E1 , V1 , N1 ) = β(E2 , V2 , N2 ). (2.2.5)

Clearly, if initially, we put all energy in system 1 (say), there will be energy
transfer from system 1 to system 2 until Eq. (2.2.3) is satisfied. From that mo-
ment on, no net energy flows from one subsystem to the other, and we say
that the two subsystems are in (thermal) equilibrium. When this equilibrium
is reached, ln  of the total system is at a maximum. This suggests that ln  is
somehow related to the thermodynamic entropy S of the system. As we have
seen in the previous section, the Second Law of thermodynamics states that the
entropy of a system N, V , and E is at its maximum when the system has reached
thermal equilibrium.
To establish the relation between ln  and the entropy we could simply start
by assuming that the entropy is equal to ln  and then check whether predictions
based on this assumption agree with experiment. If we do so, we find that the
answer is “not quite”: for (unfortunate) historical reasons (entropy already had
 Thermodynamics and statistical mechanics Chapter | 2 27

units before statistical mechanics had been created), entropy is not simply equal
to ln ; rather we have

S(N, V , E) ≡ kB ln (N, V , E), (2.2.6)

where kB is Boltzmann’s constant, which in S.I. units has the value 1.380649
10−23 J/K. With this identification, we see that our assumption that all degen-
erate eigenstates of a quantum system are equally likely immediately implies
that, in thermal equilibrium, the entropy of a composite system is at a maxi-
mum. It would be a bit premature to refer to this statement as the Second Law
of thermodynamics, as we have not yet demonstrated that the present definition
of entropy is, indeed, equivalent to the thermodynamic definition. We simply
take an advance on this result.
The next thing to note is that thermal equilibrium between subsystems 1 and
2 implies that β1 = β2 . In everyday life, we have another way to express the
same thing: we say that two bodies brought into thermal contact are in equilib-
rium if their temperatures are the same. This suggests that β must be related to
the absolute temperature. The thermodynamic definition of temperature is given
by Eq. (2.1.11), or
 
∂S
1/T = . (2.2.7)
∂E V ,N
If we use the same definition here, we find that

β = 1/(kB T ). (2.2.8)

2.2.2 Systems at constant temperature

Now that we have a statistical mechanical definition of temperature, we can con-
sider what happens if, as in section 2.1.1, we have a small system (denoted by
system 1) in thermal equilibrium with a large heat bath (system 2) (see Fig. 2.1).
The total system is closed; that is, the total energy E = E1 + E2 is fixed. Sup-
pose that the system 1 is prepared in one specific quantum state i with energy
Ei . The bath then has an energy E1 = E − Ei and the degeneracy of the bath is
given by 2 (E − Ei ). Clearly, the degeneracy of the bath determines the prob-
ability P i to find system 1 in state i:

2 (E − Ei )
Pi =  . (2.2.9)
j 2 (E − Ej )

To compute 2 (E − Ei ), we assume that our bath, system 2, is much larger than

system 1, which allows us to expand ln 2 (E − Ei ) around Ei = 0:

∂ ln 2 (E)
ln 2 (E − Ei ) = ln 2 (E) − Ei + O(1/E), (2.2.10)
∂E
28 PART | I Basics

and, using Eqs. (2.2.6) and (2.2.7),

ln 2 (E − Ei ) = ln 2 (E) − Ei /kB T + O(1/E). (2.2.11)

If we insert this result in Eq. (2.2.9), and take the limit E → ∞, we get
exp(−Ei /kB T )
Pi =  . (2.2.12)
j exp(−Ej /kB T )

This is the well-known Boltzmann distribution for a system at temperature T .

Knowledge of the energy distribution allows us to compute the average energy
E of the system at the given temperature T :

E = Ei Pi
i

Ei exp(−Ei /kB T )
= i
j exp(−Ej /kB T )

∂ ln i exp(−Ei /kB T )
=−
∂1/kB T
∂ ln Q
=− , (2.2.13)
∂1/kB T
where, in the last line, we have defined the partition function, Q ≡ Q(N, V , T ).
If we compare Eq. (2.2.13) with the thermodynamic relation Eq. (2.1.36)
∂F /T
E= ,
∂1/T
where F is the Helmholtz free energy, we see that F is related to the partition
function Q:
 

F = −kB T ln Q = −kB T ln exp(−Ei /kB T ) . (2.2.14)
i

Strictly speaking, F is fixed only up to a constant. Or, what amounts to the

same thing, the reference point of the energy can be chosen arbitrarily. In
what follows, we can use Eq. (2.2.14) without loss of generality. The relation
between the Helmholtz free energy and the partition function is often more con-
venient to use than the relation between ln  and the entropy. As a consequence,
Eq. (2.2.14) is the workhorse of equilibrium statistical mechanics.

2.2.3 Towards classical statistical mechanics

Thus far, we have formulated statistical mechanics in purely quantum mechani-
cal terms. The entropy is related to the density-of-states of a system with energy
 Thermodynamics and statistical mechanics Chapter | 2 29

E, volume V , and number of particles N . Similarly, the Helmholtz free energy is

related to the partition function Q, a sum over all quantum states i of the Boltz-
mann factor exp(−Ei /kB T ). To be specific, let us consider the average value of
some observable A. We know the probability that a system at temperature T will
be found in an energy eigenstate with energy Ei and we can therefore compute
the thermal average of A as

exp(−Ei /kB T ) i|A|i
A = i , (2.2.15)
j exp(−Ej /kB T )

where i|A|i denotes the expectation value of the operator A in quantum state
i. This equation suggests how we should go about computing thermal averages:
first we solve the Schrödinger equation for the (many-body) system of inter-
est, and next we compute the expectation value of the operator A for all those
quantum states that have a non-negligible statistical weight. Unfortunately, this
approach is doomed for all but the simplest systems. First of all, we cannot
hope to solve the Schrödinger equation for an arbitrary many-body system. And
second, even if we could, the number of quantum states that contribute to the
25
average in Eq. (2.2.15) would be so huge (O(1010 )) that a numerical evalua-
tion of all expectation values would be inconceivable. Fortunately, Eq. (2.2.15)
can be simplified to a more workable expression in the classical limit. To this
end, we first rewrite Eq. (2.2.15) in a form that is independent of the specific
basis set. We note that exp(−Ei /kB T ) = i| exp(−H/kB T )|i, where H is the
Hamiltonian of the system. Using this relation, we can write

i| exp(−H/kB T )A|i
A = i
j j | exp(−H/kB T )|j 
Tr exp(−H/kB T )A
= , (2.2.16)
Tr exp(−H/kB T )
where Tr denotes the trace of the operator. As the value of the trace of an op-
erator does not depend on the choice of the basis set, we can compute thermal
averages using any basis set we like. Preferably, we use simple basis sets, such as
the set of eigenfunctions of the position or the momentum operator. Next, we use
the fact that the Hamiltonian H is the sum of a kinetic part K and a potential part
U. The kinetic energy operator is a quadratic function of the momenta of all par-
ticles. As a consequence, momentum eigenstates are also eigenfunctions of the
kinetic energy operator. Similarly, the potential energy operator is a function of
the particle coordinates. Matrix elements of U, therefore, are most conveniently
computed in a basis set of position eigenfunctions. However, H = K + U itself is
not diagonal in either basis set nor is exp[−β(K + U)]. However, if we could re-
place exp(−βH) by exp(−βK) exp(−βU), then we could simplify Eq. (2.2.16)
considerably. In general, we cannot make this replacement because

exp(−βK) exp(−βU) = exp{−β[K + U + O([K, U])]},

30 PART | I Basics

where [K, U] is the commutator of the kinetic and potential energy operators:
O([K, U]) stands for all terms containing commutators and higher-order com-
mutators of K and U. It is easy to verify that the commutator [K, U] is of order 
( ≡ h/(2π), where h is Planck’s constant). Hence, in the limit  → 0, we may
ignore the terms of order O([K, U]). In that case, we can write

Tr exp(−βH) ≈ Tr exp(−βU) exp(−βK). (2.2.17)

If we use the notation |r for eigenvectors of the position operator and |k for
eigenvectors of the momentum operator, we can express Eq. (2.2.17) as

Tr exp(−βH) = r|e−β U |r r|k k|e−β K |k k|r. (2.2.18)
r,k

All matrix elements can be evaluated directly:


r| exp(−βU)|r = exp −βU(rN ) ,

where U(rN ) on the right-hand side is no longer an operator but a function of

the coordinates of all N particles. Here, and in what follows, we denote this set
of coordinates by rN . Similarly,


N
k| exp(−βK)|k = exp −β pi2 /(2mi ) ,
i=1

where p i = ki , and

r|k k|r = 1/V N ,
where V is the volume of the system and N the number of particles. Finally,
we can replace the sum over states by an integration over all coordinates and
momenta. The final result is

1   
Tr exp(−βH) ≈ dN dp dr exp −β
N N
pi /(2mi ) + U r
2 N
h N!
i
≡ Qclassical , (2.2.19)

where d is the dimensionality of the system and the last line defines the classical
partition function. The factor 1/N ! corrects for the fact that any permutation of
indistinguishable particles corresponds to the same macroscopic state.5

5 The factor 1/N ! is often justified by invoking the quantum-mechanical indistinguishability of

identical particles. However, even in systems of non-identical particles that are so similar that they
cannot be separated, the same factor is necessary to ensure extensivity of the Helmholtz free energy
[50–52].
 Thermodynamics and statistical mechanics Chapter | 2 31

Similarly, we can derive the classical limit for Tr exp(−βH)A, and finally,
we can write the classical expression for the thermal average of the observable
A as
   2  N   N N 
dpN drN exp −β i pi /(2mi ) + U r A p ,r
A =       . (2.2.20)
dpN drN exp −β j pj /(2mj ) + U r
2 N

Eqs. (2.2.19) and (2.2.20) constitute the starting points for a wide range of simu-
lations of classical many-body systems. Eqs. (2.2.19) and (2.2.20) are expressed
in terms of high-dimensional integrals over the dN momenta and dN coordi-
nates of all N particles, where d denotes the dimensionality of the system. The
2dN-dimensional space spanned by all momenta and coordinates is called phase
space.

2.3 Ensembles
In statistical mechanics as in thermodynamics, the state of the system is deter-
mined by a number of control parameters, some of them extensive (e.g., N , the
number of particles), some of them intensive (e.g., the pressure P or the tem-
perature T ). For historical reasons we denote the collection of all realizations
of a system, which are compatible with a set of control parameters by the name
“ensemble”. There are different ensembles for different sets of control parame-
ters. The historical names of these ensembles (“micro-canonical,” “canonical,”
“grand-canonical,” etc.) are not particularly illuminating. Below, we will list
these names when we describe the most frequently used ensembles. However,
in what follows, we will often denote ensembles by the control variables that
are kept constant, e.g., the “constant-N V E ensemble” or the “constant-μV T
ensemble”. In the following sections, we assume for convenience that the sys-
tem consists of particles with no internal degrees of freedom (i.e., no rotations,
vibrations, or electronic excitations). That assumption simplifies the notation,
but for molecular systems, we will of course have to take internal degrees of
freedom into account.

2.3.1 Micro-canonical (constant-NVE) ensemble

In the micro-canonical ensemble the energy, volume, and the number of parti-
cles of every species are kept constant.6 In a classical system, the total energy
is given by the Hamiltonian, H, which is the sum of the kinetic and potential

6 In simulations we can fix different thermodynamic parameters X, Y, Z, · · · : N , V , E is just one

example. For the sake of compactness, we will often refer to an ensemble with X, Y , Z constant as
the XY Z-ensemble. We refer to the corresponding simulations as XY Z-MC or XY Z-MD.
32 PART | I Basics

energy:

N
p2i  
H= + U rN , (2.3.1)
2m
i=1

in which we have assumed that the potential energy does not depend on the
momenta p. The classical partition function in the micro-canonical ensemble is
the phase-space integral over a hyper-surface where the value of the Hamiltonian
is equal to the imposed value of the energy E.  
The constraint that the system has to be on the hyper-surface H pN , rN =
E can be imposed via a δ-function, and hence for a three-dimensional system
(d = 3):
1    
E,V ,N ≡ 3N dpN drN δ H pN , rN − E . (2.3.2)
h N!

2.3.2 Canonical (constant-NVT) ensemble

The ensemble of states at constant N , V and T is called the “canonical ensem-
ble.” As described in the previous section, the classical partition function, Q, for
a system of atoms at constant N , V and T is given by:

1   
QN,V ,T ≡ dpN drN exp −βH pN , rN . (2.3.3)
h3N N !

As the potential energy does not depend on the momenta of the system, the
integration over the momenta can be done analytically7


N   3N  
p2i p2 2πm 3N/2
dp exp −β
N
= dp exp −β = .
2m 2m β
i=1
(2.3.4)
If we define the thermal Broglie wavelength as

h2
= , (2.3.5)
2πmkB T

we can write the canonical partition function as:

1    1
Q (N, V , T ) = 3N N !
drN exp −βU rN ≡ 3N N !
Z (N, V , T ) ,
(2.3.6)

7 For molecular systems, in particular for systems of flexible molecules where bond lengths are
fixed by holonomic constraints (see section 14.1), the integration over momenta may result in a
Jacobian that depends on the coordinates of the nuclei in the molecule.
 Thermodynamics and statistical mechanics Chapter | 2 33

which defines the configurational integral, Z ≡ Z(N, V , T ):

  
Z (N, V , T ) = drN exp −βU rN . (2.3.7)

Unlike the integral over momenta, the configurational integral can almost never
be computed analytically. The canonical partition function Q (N, V , T ) is re-
lated to the Helmholtz free energy, F , through

βF = − ln Q (N, V , T ) . (2.3.8)

Having defined the configurational

 integral Z (N, V , T ), we can write the en-
semble average of a quantity A rN that depends on the coordinates only as

1     
A = drN A rN exp −βU rN . (2.3.9)
Z (N, V , T )

The probability (N ) of finding our system in a particular configuration rN , is

given by
  1     
dr δ rN − r exp −βU r
N N N
N rN =
Z (N, V , T )
  
∝ exp −βU rN . (2.3.10)

2.3.3 Isobaric-isothermal (constant-NPT) ensemble

The canonical ensemble describes a system at constant temperature and volume.
In experiments, it is more common to fix the pressure P than the volume V . As
with the constant-N V T ensemble, we can derive the probability distribution
function for the constant-N P T ensemble by considering a closed system that
consists of the system of interest (system 1), that is in contact with a reservoir
(system 2), which acts as both a thermostat and barostat (see Fig. 2.2). The two
sub-systems can exchange energy and change their volume in such a way that
the total volume is constant.
For simplicity, we start with the quantum expression for the total entropy of
the system, Eq. (2.2.6):

S = S1 + S2 = kB ln 1 (E1 , V1 , N1 ) + kB ln 2 (E2 , V2 , N2 ) . (2.3.11)

As system 2 is much larger than system 1, we can make an expansion of 

around V and E:
 
∂ ln  (E, V , N2 )
ln  (E2 , V2 , N2 ) = ln  (E, V , N2 ) + (E − E1 )
∂E N,V
 
∂ ln  (E, V , N2 )
+ (V − V1 ) + · · ·
∂V N,E
34 PART | I Basics

E − E1 P (V − V1 )
= ln  (E, V , N2 ) + + + ··· ,
kB T kB T
(2.3.12)
where we have used Eq. (2.1.15), which relates the derivative of the entropy
with respect to energy and volume to 1/T and P /T , respectively. We can then
write the probability to find system 1 with an energy E1 and volume V1 as:
 (E − E1 , V − V1 , N2 )
P (E1 , V1 , N1 ) =   



j dV  E − Ej , V − V , N2
 
E1 P V1
∝ exp − − . (2.3.13)
kB T kB T
Taking the classical limit, we obtain an expression for the N , P , T -partition
function, Q ≡ Q (N, P , T ), which is an integral over particle coordinates and
over the volume V :
βP   
Q (N, P , T ) ≡ 3N dV exp (−βP V ) drN exp −βU rN ,
N!
(2.3.14)
where the factor βP has been included to make Q (N, P , T ) dimensionless.
From Eq. (2.3.14) we obtain the probability to find our system in a particular
configuration rN and volume V :
    
N rN ∝ exp −βP V − βU rN . (2.3.15)

Q (N, P , T ) is related to the Gibbs free energy, G, via

βG = − ln Q (N, P , T ) . (2.3.16)
The above relation follows from the fact that, in the thermodynamic limit, the
integral in Eq. (2.3.14) is completely dominated by the maximum value of the
integrand ∼ exp {−β[P V ∗ + F (N, V ∗ , T )]}, where V ∗ is the volume for which
the integrand is maximal. This maximum term method is used to establish the
relation between thermodynamic variables and statistical mechanical partition
functions for other ensembles.8

2.3.4 Grand-canonical (constant-μVT) ensemble

Thus far, we have considered ensembles in which the total number of particles
remains constant. It is often convenient to consider open systems, in which the
8 The maximum-term or saddle-point approximation relies on the observation that we can ap-
proximate the (one-dimensional) integral I of a function eR(x) , which is sharply peaked at x ∗ , by
replacing R(x) close to x ∗ by R(x ∗ ) − (c/2)(x − x ∗ )2 , where c equals (minus) the second derivative
 ∗ ∗ 2 ∗ √
of R(x) at x ∗ . The resulting Gaussian integral dx eR(x )−(c/2)(x−x ) yields I ≈ eR(x ) 2π/c.
In statistical√mechanics, ln I is related to the appropriate thermodynamic potential, and the contri-
bution of ln 2π/c is negligible in the thermodynamic limit.
 Thermodynamics and statistical mechanics Chapter | 2 35

FIGURE 2.3 An isolated system consisting of two boxes 1 and 2 that can exchange heat and
exchange particles in such a way that the total energy and number of particles remain constant.
System 2 is much larger compared to system 1, so it can act as a heat bath and buffer on system 1.

number of particles is allowed to change. We consider again a system containing

two subsystems (Fig. 2.3). The volume of system 1 is fixed, but it is allowed to
exchange energy and particles with the reservoir 2. As before, the entire system
is closed, and the entropy of the entire system is given by Eq. (2.3.11). As system
2 is much larger than system 1, we can expand ln  around E and N:
 
∂ ln  (E, V , N2 )
ln 2 (E2 , V2 , N2 ) = ln  (E, V2 , N) + (E − E1 )
∂E N,V
 
∂ ln  (E, V2 , N)
+ (N − N1 ) + · · ·
∂N E,V
E − E1 μ (N − N1 )
= ln  (E, V2 , N) + − + ···
kB T kB T
(2.3.17)

where we have used Eq. (2.1.15) to relate the derivative of the entropy with
respect to the number of particles to the chemical potential. It then follows that
the probability to find system 1 with an energy E1 and number of particles N1
is:

 (E − E1 , V2 , N − N1 )
P (E − E1 , V2 , N − N1 ) =  N  
j M=0  E − Ej , V2 , N − M
 
E1 μN1
∝ exp − + . (2.3.18)
kB T kB T

The classical partition function now involves a summation of over all particles
in system 1. As the reservoir is much larger than system 1, we can replace the
upper limit of this summation by ∞:
36 PART | I Basics

∞
 exp (βμN)   
(μ, V , T ) ≡ 3N N !
drN exp −βU rN
N =0
∞
= exp (βμN) e−βF (N,V ,T ) , (2.3.19)
N =0

where we have defined the grand-canonical partition function, ≡ (μ, V , T ).

From Eq. (2.3.19) we obtain the probability to find N particles in system 1, in
configuration rN :
    
N rN ∝ exp βμN − βU rN . (2.3.20)

From Eqs. (2.3.19) and (2.3.8) it follows, using the maximum term method, that

− kB T ln = F − Nμ =  , (2.3.21)

where  is the grand potential defined in section 2.1.2. For homogeneous sys-
tems, we can replace  by −P V .

2.4 Ergodicity
Thus far, we have discussed the average behavior of many-body systems in a
purely static sense: we introduced only the assumption that every quantum state
of a many-body system with energy E is equally likely to be occupied. Such
an average over all possible quantum states of a system is called an ensemble
average. However, this is not how we usually think about the average behavior
of a system. In most experiments, we perform a series of measurements during
a certain time interval and then determine the average of these measurements.
In fact, the idea behind Molecular Dynamics simulations is precisely that we
can study the average behavior of a many-particle system simply by comput-
ing the natural time evolution of that system numerically and then average the
quantity of interest over a sufficiently long time. To take a specific example, let
us consider a fluid consisting of atoms. Suppose that we wish to compute the
average density of the fluid at a distance r from a given atom i, ρi (r). Clearly,
the instantaneous density depends on the coordinates rj of all particles j in
the system. As time progresses, the atomic coordinates will change (according
to Newton’s equations of motion), and hence the density around atom i will
change. Provided that we have specified the initial coordinates and momenta of
all atoms (rN (0), pN (0)), we know, at least in principle, the time evolution of
ρi (r; rN (0), pN (0), t).
In a standard Molecular Dynamics simulation, we measure the time-
averaged density ρi (r) of a system of N atoms, in a volume V , at a constant
total energy E:
1 t
ρi (r) = lim dt  ρi (r; t  ). (2.4.1)
t→∞ t 0
 Thermodynamics and statistical mechanics Chapter | 2 37

Note that, in writing down this equation, we have implicitly assumed that, for
t sufficiently long, the time average does not depend on the initial conditions.
This is, in fact, a subtle assumption that is not true in general (see e.g., [53]).
However, we shall disregard subtleties and simply assume that, once we have
specified N , V , and E, time averages do not depend on the initial coordinates
and momenta. If that is so, then we would not change our result for ρi (r) if we
average over many different initial conditions; that is, we consider the hypothet-
ical situation where we run a large number of Molecular Dynamics simulations
at the same values for N , V , and E, but with different initial coordinates and
momenta,
 
 1 t   N
lim dt ρi r; r (0), pN (0), t 
t→∞ t 0
initial conditions
ρi (r) = . (2.4.2)
number of initial conditions
We now consider the limiting case where we average over all initial conditions
compatible with the imposed values of N , V , and E. In that case, we can replace
the sum over initial conditions with an integral:
  
f rN (0), pN (0)   N 
N N N
initial conditions E dr dp f r (0), p (0)
→ , (2.4.3)
number of initial conditions (N, V , E)

where f denotes an arbitrary

 function of the initial conditions rN (0), pN (0),
while (N, V , E) = E dr dpN , where we have ignored a constant factor.
N

Note that the second line in Eq. (2.4.3) is nothing else than the micro-canonical
(constant-N V E) average of f . In what follows, we denote an ensemble average
by · · ·  to distinguish it from a time average, denoted by a bar. If we switch the
order of the time averaging and the averaging over initial conditions, we find

1 t   
ρi (r) = lim dt  ρi r; rN (0), pN (0), t  . (2.4.4)
t→∞ t 0 NV E

However, the ensemble average in this equation does not depend on the time
t  . This is so because there is a one-to-one correspondence between the ini-
tial phase-space coordinates of a system and those that specify the state of
the system at a later time t  (see e.g., [53,54]). Hence, averaging over all ini-
tial phase-space coordinates is equivalent to averaging over the time-evolved
phase-space coordinates. For this reason, we can leave out the time averaging in
Eq. (2.4.4), and we find
ρi (r) = ρi (r)N V E . (2.4.5)
This equation states that, if we wish to compute the average of a function of the
coordinates and momenta of a many-particle system, we can either compute that
quantity by time averaging (the “MD” approach) or by ensemble averaging (the
38 PART | I Basics

“MC” approach). It should be stressed that the preceding paragraphs are meant
only to make Eq. (2.4.5) plausible, not as a proof. In fact, that would have been
quite impossible because Eq. (2.4.5) is not true in general. However, in what
follows, we shall simply assume that the “ergodic hypothesis”, as Eq. (2.4.5) is
usually referred to, applies to the systems that we study in computer simulations.
The reader should be aware that there are many examples of systems that are not
ergodic in practice, such as glasses and metastable phases, or even in principle,
such as nearly harmonic solids.

2.5 Linear response theory

Until now, we focused on ensemble averages (Eq. (2.2.20)) or time averages
(Eq. (2.4.1)) of quantities that are constant in time (once fluctuations have been
averaged out). One of the great advantages of the Molecular Dynamics method
is that it also allows us to predict the time-dependent response of a system to an
external perturbation. Examples of such responses are the heat current due to a
temperature gradient, or the electrical current induced by an electric field (we
shall see more examples in Chapter 5).
It would seem that computing such currents would require a non-equilibrium
simulation, where we impose the external perturbation of interest. This is indeed
possible, but the disadvantage is that for every different perturbation, we would
have to carry out a separate simulation. Fortunately, for systems that are only
weakly perturbed (such that the response is linear in the applied perturbation),
we can predict the response to an applied perturbation by studying the decay of
fluctuations in the corresponding current in equilibrium.
Onsager [55,56] was the first to suggest that a response (e.g., a current) in-
duced in a system by a weak external perturbation, decays in the same way
as a spontaneous fluctuation of that current in equilibrium. Onsager formulated
his “regression” hypothesis in the language of non-equilibrium thermodynam-
ics; in fact, Onsager’s 1931 papers [55,56], although partly inspired by earlier
work, created the field (see [57]). The theory of non-equilibrium thermodynam-
ics provides a phenomenological description of the relation between applied
perturbations and resulting fluxes/currents. In particular, it defines the linear
transport coefficients that relate a small perturbation to the resulting fluxes.
However, non-equilibrium thermodynamics is not an atomistic theory and hence
it is not always straightforward to make a link between the macroscopic forces
and fluxes and the atomistic description of the same quantities. Moreover, for
simulations, we need expressions for the transport coefficients that are com-
putable in terms of molecular coordinates and momenta. Such expressions are
provided by linear response theory.
Below we give a simple introduction to classical linear response theory, to
illustrate the mechanical basis of Onsager’s regression hypothesis. For a more
detailed discussion, the reader is referred to textbooks on advanced statistical
mechanics, such as [53]. A simple introduction (similar to the one presented
 Thermodynamics and statistical mechanics Chapter | 2 39

here) is given in the book by Chandler [58], while an extensive discussion of

linear response theory in the context of the theory of liquids is given in [59].

2.5.1 Static response

Before discussing transport, we consider the static response of a system to a
weak applied field. The field could be an electric field, for instance, and the re-
sponse might be the electric current or, for a nonconducting material, the electric
polarization. Suppose that we are interested in the response to a property that can
be expressed as the ensemble average of a dynamical variable A. In the presence
of an external perturbation, the average of A changes from its equilibrium value
A0 to A0 + A. Next, we must specify the perturbation. We assume that
the perturbation also can be written as an explicit function of the coordinates
(and, possibly, momenta) of the particles in the system. The effect of the pertur-
bation is to change the Hamiltonian H0 of the system, to H0 − λB(pN , rN ). For
instance, in the case of an electric field along the x direction, the change in H
would be H = −Ex Mx (rN ), where Mx is the x component of the total dipole
moment of the system. The electric field Ex corresponds to the parameter λ. We
can immediately write down the general expression for A:

d A exp[−β(H0 − λB)]
A0 + A =  , (2.5.1)
d exp[−β(H0 − λB)]
 
where we have used the symbol  to denote pN , rN , the phase-space coordi-
nates of the system. Let us now compute the part of A that varies linearly
with λ. To this end, we compute
 
∂ A
. (2.5.2)
∂λ λ=0

Straightforward differentiation shows that

 
∂ A
= β { AB0 − A0 B0 } . (2.5.3)
∂λ λ=0

Taking again the example of the electric polarization, we can compute the
change in dipole moment of a system due to an applied field Ex :
    
∂Mx
Mx  = Ex = βEx Mx2 − Mx 2 . (2.5.4)
∂Ex Ex =0

Suppose that we wish to compute the electric susceptibility of an ideal gas of

nonpolarizable dipolar molecules with dipole moment μ. In that case,
   N  
j
Mx2 − Mx 2 = μix μx
i,j =1
40 PART | I Basics

 
= N (μix )2
N μ2
= ,
3
and hence,
Mx μ2 ρ
Px ≡ = Ex . (2.5.5)
V 3kB T
Of course, this example is special because it can be evaluated exactly. But, in
general, we can compute the expression (2.5.3) for the susceptibility, only nu-
merically. It should also be noted that, actually, the computation of the dielectric
susceptibility is quite a bit more subtle than suggested in the preceding exam-
ple. The subtleties are discussed in the book by Allen and Tildesley [21] and the
contribution of McDonald in [44]).

Hamiltonian thermodynamic integration

The above discussion of static linear response is but a special case of the ef-
fect of a change in the Hamiltonian of a system on its free energy (see sec-
tion 8.4.2) —an approach pioneered by Kirkwood [60], partly together with
Monroe-Boggs [61].9
The essence of Hamiltonian integration can be expressed in a few lines.
Consider a Hamiltonian H(λ), such that H(λ = 0) = H0 and H(λ = 1) = H1 .
Typically, H0 corresponds to a reference state for which we know the free en-
ergy. We need not assume that H(λ) is a linear function of λ. In what follows,
we use the notation
 
∂H(λ)
H (λ) ≡ .
∂λ
For the case discussed above H(λ) = H0 − λB, and hence H (λ) = −B.
In general, we have
 
∂F (λ)
= H (λ) ,
∂λ
and hence
1
F (λ = 1) = dλ H (λ)λ . (2.5.6)
0
In Eq. (2.5.6), the subscript in · · · λ means that the Boltzmann average is
computed using a Boltzmann weight corresponding to the Hamiltonian H(λ).
Eq. (2.5.6) is the starting point for most “Hamiltonian” integration schemes in
simulations that will be discussed in Section 8.4.2. Such schemes aim to com-
pute the unknown free energy of a system at λ = 1 from knowledge of the free
9 The same Elisabeth Monroe who, together with Kirkwood, laid the foundations for the theory of
entropic freezing.
 Thermodynamics and statistical mechanics Chapter | 2 41

energy of a reference state (λ = 0) using the average values of H (λ) evaluated

during the simulations.

2.5.2 Dynamic response

Next, we consider a simple time-dependent perturbation. We begin by preparing
the system in the presence of a very weak, constant perturbation (λB). The static
response of A to this perturbation is given by Eq. (2.5.3). At time t = 0, we
switch off the external perturbation. The response A will now decay to 0. We
can write an expression for the average of A at time t:

d exp[−β(H0 − λB)]A(t)
A(t) =  , (2.5.7)
d exp[−β(H0 − λB)]

where A(t) is the value of A at time t if the system started at point  in phase
space and then evolved according to the natural time evolution of the unper-
turbed system. For convenience, we have assumed that the average of A in the
unperturbed system vanishes. In the limit λ → 0, we can write

d exp[−βH0 ]BA(t)
A(t) = βλ 
d exp[−βH0 ]
= βλ B(0)A(t) . (2.5.8)

The quantity B(0)A(t) is called a time correlation function (if B = A, it is

called an autocorrelation function). The time correlation function B(0)A(t) is
the time average of the product of the value of A at time τ and B at time t + τ :

1 t0    
B(0)A(t) ≡ lim dτ A rN (τ ), pN (τ ) B rN (t + τ ), pN (t + τ ) ,
t0 →∞ t0 0
(2.5.9)
where {rN (x), pN (x)} denotes the phase-space coordinates at time x. Note that
the time evolution of {rN (x), pN (x)} is determined by the unperturbed Hamil-
tonian H0 .
To give a specific example, consider a gas of dipolar molecules again in
the presence of a weak electric field Ex . The perturbation is equal to −Ex Mx .
At time t = 0, we switch off the electric field. When the field was still on, the
system had a net dipole moment. When the field is switched off, this dipole
moment decays:
Mx (t) = Ex β Mx (0)Mx (t) . (2.5.10)
In words, the decay of the macroscopic dipole moment of the system is de-
termined by the dipole autocorrelation function, which describes the decay of
spontaneous fluctuations of the dipole moment in equilibrium. This relation be-
tween the decay of the response to an external perturbation and the decay of
fluctuations in equilibrium is an example of Onsager’s regression hypothesis.
42 PART | I Basics

It might seem that the preceding example of a constant perturbation that

is suddenly switched off is of little practical use, because we are interested in
the effect of an arbitrary time-dependent perturbation. Fortunately, in the linear
regime that we are considering, the relation given by Eq. (2.5.8) is enough to
derive the general response.
To see this, let us consider a time-dependent external field f (t) that couples
to a mechanical property B; that is,

H(t) = H0 − f (t)B. (2.5.11)

To linear order in f (t), the most general form of the response of a mechanical
property A to this perturbation is
∞
A(t) = dt  χAB (t, t  )f (t  ), (2.5.12)
−∞

where χAB , the “after-effect function”, describes the linear response. We know
several things about the response of the system that allow us to simplify
Eq. (2.5.12). First of all, the response must be causal; that is, there can be no
response before the perturbation is applied. As a consequence,

χAB (t, t  ) = 0 for t < t  . (2.5.13)

Secondly, the response at time t to a perturbation at time t  depends only on the

time difference t − t  . Hence,
t
A(t) = dt  χAB (t − t  )f (t  ). (2.5.14)
−∞

Once we know χ, we can compute the linear response of the system to an arbi-
trary time-dependent perturbing field f (t). To find an expression for χAB , let us
consider the situation described in Eq. (2.5.8), namely, an external perturbation
that has a constant value λ until t = 0 and 0 from then on. From Eq. (2.5.14), it
follows that the response to such a perturbation is
0
A(t) = λ dt  χAB (t − t  )
−∞
∞
=λ dτ χAB (τ ). (2.5.15)
t

If we compare this expression with the result of Eq. (2.5.8), we see immediately
that
∞
λ dτ χAB (τ ) = βλ B(0)A(t) (2.5.16)
t
 Thermodynamics and statistical mechanics Chapter | 2 43

or

−β B(0)Ȧ(t) for t >0
χAB (t) = (2.5.17)
0 for t ≤ 0.

To give a specific example, consider the mobility of a molecule in an external

field Fx . The Hamiltonian in the presence of this field is

H = H0 − Fx x. (2.5.18)

The phenomenological expression for the steady-state velocity of a molecule in

an external field is
vx (t) = mFx , (2.5.19)
where m is the mobility of the molecule under consideration. We can now de-
rive a microscopic expression for the mobility in terms of a time-correlation
function. From Eqs. (2.5.14) through (2.5.17), we have
t
vx (t) = Fx dt  χvx x (t − t  )
−∞
∞
= Fx dτ χvx x (τ )
0
∞
= −βFx dτ x(0)v̇x (τ )
0
∞
= +βFx dτ vx (0)vx (τ ) . (2.5.20)
0

In the last line of Eq. (2.5.20), we used the stationarity property of time-
correlation functions:
d 
A(t)B(t + t  ) = 0. (2.5.21)
dt
Carrying out the differentiation, we find that
 
Ȧ(t)B(t + t  ) = − A(t)Ḃ(t + t  ) . (2.5.22)

Combining Eqs. (2.5.19) and (2.5.20), we find that

∞
m=β dt vx (0)vx (t) . (2.5.23)
0

Eq. (2.5.23) relating a transport coefficient to an integral of a time correlation

function, is an example of a so-called Green-Kubo relation [62]. In Chapter 5 we
shall see that the mobility m is related to the self-diffusion coefficient D through
the Einstein relation m = βD. In Section F.1 of the Appendix, we discuss how,
44 PART | I Basics

from the knowledge of certain time-correlation functions, we can compute the

rate of dissipation due to an applied periodic perturbation. Such information is
very useful when modelling, for instance, the absorption of radiation, be it that
the expressions derived in the Appendix only apply to the classical case where
ω  kB T .

Power spectra
Time-correlation functions are often computed (and also measured in spectro-
scopic experiments), by Fourier transforming from the frequency domain using
the Wiener-Khinchin (WK) theorem (see e.g., [53]).
To derive the WK theorem, we first define the Fourier transform of the ob-
servable of interest over time interval T
T
â(ω) ≡ dt A(t)eiωt . (2.5.24)
0

Note that we define the Fourier transform over a finite time interval {0 − T},
because the simulation has a finite length. We now define GA (ω), the power
spectrum of A, as

1  
GA (ω) ≡ lim |â(ω)|2 (2.5.25)
T→∞ 2πT
1 T T  
= lim dt dt  A(t)A(t  ) eiωt e−iωt
T→∞ 2πT 0 0
1 T T−t   
= lim dt  dt − t  A(0)A(t − t  ) eiω(t−t ) ,
T→∞ 2πT 0 −t 

where we have used the fact that an equilibrium time correlation function only
depends on the time difference t − t  . When T is much longer than the time
it takes the correlation function to decay, we can now write (in the limit that
T → ∞):
1 ∞
GA (ω) = dτ A(0)A(τ ) eiωτ , (2.5.26)
2π −∞

where we have defined τ ≡ t − t  . Equation (2.5.26) shows that GA (ω) is the

Fourier transform of A(0)A(τ ), and conversely that
∞
A(0)A(τ ) = dω GA (ω)e−iωτ . (2.5.27)
−∞

Equation (2.5.27) is often used to obtain correlation functions from spectro-

scopic data (see e.g., section F.1). For the calculation of correlation functions,
it is important to note that correlating n points, requires n(n − 1)/2 multiplica-
tions, whereas a (fast) Fourier transform [38] only requires n ln n operations.
 Thermodynamics and statistical mechanics Chapter | 2 45

Power spectra can often be determined by measuring the dissipation in a

system subject to a periodic perturbation (see Appendix F.1).

2.6 Questions and exercises

Question 1 (Number of configurations).
1. Consider a system A consisting of subsystems A1 and A2 , for which 1 =
1020 and 2 = 1022 . What is the number of configurations available to the
combined system? Also, compute the entropies S, S1 , and S2 .
2. By what factor does the number of available configurations increase when
10 m3 of air at 1.0 atm and 300 K is allowed to expand by 0.001% at constant
temperature? Here and in what follows, we assume that air behaves like an
ideal gas.
3. By what factor does the number of available configurations increase when
150 kJ is added to a system containing 2.0 mol of particles at constant vol-
ume and T = 300 K?
4. A sample consisting of five molecules has a total energy 5. Each molecule
is able to occupy states of energy j , with j = 0, 1, 2, · · · , ∞. Draw up a
table with columns by the energy of the states and write beneath them all
configurations that are consistent with the total energy. Identify the type of
configuration that is most probable.

Question 2 (Thermodynamic variables in the canonical ensemble). Starting

with an expression for the Helmholtz free energy (F ) as a function of N , V , T

− ln [Q (N, V , T )]
F= ,
β
one can derive all thermodynamic properties. Show this by deriving equations
for U , p, and S.

Question 3 (Ideal gas (Part 1)). The canonical partition function of an ideal
gas consisting of monoatomic particles is equal to

1 VN
Q (N, V , T ) = d exp [−βH ] = ,
h3N N ! λ3N N !
√
in which λ = h/ 2πm/β and d = dq1 · · · dqN dp1 · · · dpN .
Derive expressions for the following thermodynamic properties:
• F (N, V , T ) (hint: ln (N !) ≈ N ln (N ) − N )
• p (N, V , T ) (which leads to the ideal gas law)
• μ (N, V , T ) (which leads to μ = μ0 + RT ln ρ)
• U (N, V , T ) and S (N, V , T )
• Cv (heat capacity at constant volume)
• Cp (heat capacity at constant pressure)
46 PART | I Basics

Question 4 (Van der Waals equation of state). The van der Waals equation of
state describes the behavior of non-ideal gasses.
RT a
P= − 2,
v−b v
where R is the gas constant. Show that the constants a and b can be related to
the critical point:

27 R 2 Tc2
a=
64 Pc
1 RTc
b= .
8 Pc
In this book, we will use the Lennard-Jones fluid very often. The critical point
of the Lennard Jones fluid is Tc = 1.32, ρc = 0.32 (molecules per unit volume),
and Pc = 0.131 [63]. These constants are expressed in reduced units (see sec-
tion 3.3.2.5).
Plot the equation of state (pressure as a function of the molar volume for
T = 2.00 and T = 1.00.
Question 5 (Fugacity). In most Chemical Engineering textbooks, the chemical
potential is replaced by the fugacity, f .
f
βμ − βμ0 = ln ,
f0

where f 0 is the fugacity of the reference state.

• Compute the fugacity of an ideal gas
• How would you choose the reference state f 0 for a real gas
The fugacity coefficient φ is introduced to quantify deviations from the ideal gas
behavior:
f
φ= ,
P
where P is the pressure.10
Compute φ as a function of the molar volume for T = 2.0 and T = 1.0 for
the van der Waals equation of state (see Question 4).
Question 6 (Reversible work). The First Law of thermodynamics expresses the
conservation of the energy E of a system

dE = q + w ,
10 Note that for an ideal gas, P = ρk T . We can therefore also write  = f/(ρ k T ). In the rest
B id B
of the book, we use this relation to define the fugacity through  = f  /ρid . Note that f  = f/kB T .
In what follows, we drop the prime.
 Thermodynamics and statistical mechanics Chapter | 2 47

where q and w denote the (infinitesimal) energy changes of the system due to
heat flow and work, respectively. In the case of reversible changes, we can use
the First and Second Laws together, to express how the different state functions
(E, S, F , G, · · · ) change in a reversible transformation. The best-known expres-
sion is
dE = T dS − P dV + μdN ,
or, in terms of the Helmholtz free energy F :

dF = −SdT − pdV + μdN.

This expression for dF applies to the case where the reversible work is due to
a change of the volume against an external pressure P . However, we can also
consider other forms of work, for instance, due to changing the polarization of
a system at a constant applied electric field, or to changing the surface area A
at constant surface tension γ . Here, we consider the latter case:

dF = −SdT + γ dA − P dV + μdN.

1. We will assume that both V and A are linear in N . Making use of the fact
that the free energy of a system is extensive, show that

F = μN − P V + γ A

We can interpret this expression as a definition of the surface free energy

Fs ≡ γ A.
2. Derive an expression for (∂Fs /∂A)N,V ,T . This expression may look strange,
because we change A at constant N .
3. Under what conditions is

(∂Fs /∂A)N,V ,T = γ ?

Would this condition be satisfied for:

• a liquid-vapor interface?
• a solid-liquid interface?
Question 7 (Ising model). Consider a system of N spins arranged on a simple
lattice (1d: a linear chain, 2d: a square lattice, 3d: a simple cubic lattice, etc.).
In the presence of a magnetic field, H , the energy of the system is


N 
U =− H si − J si sj ,
i=1 i>j

where J is called the coupling constant (J > 0) and si = ±1. The second sum-
mation is a summation over all pairs (d × N for a periodic system, d is the
dimensionality of the system). This system is called the Ising model.
48 PART | I Basics

1. Show that for positive J , and H = 0, the lowest energy of the Ising model is
equal to
U0 = −dN J .
2. Show that the free energy per spin of a 1d Ising model with zero field is equal
to
F (β, N) ln (2 cosh (βJ ))
=−
N β
when N → ∞. The function cosh (x) is defined as
exp [−x] + exp [x]
cosh (x) = . (2.6.1)
2
3. Derive equations for the energy and heat capacity of this system.
Question 8 (The photon gas). An electromagnetic field in thermal equilibrium
can be described as a photon gas. From the quantum theory of the electromag-
netic field, it is found that the total energy of the system (U ) can be written as
the sum of photon energies:


N 
N
U= nj ω j  = n j j
j =1 j =1

in which j is the characteristic energy of a photon with frequency ω, j ,

nj = 0, 1, 2, · · · , ∞ is the so-called occupancy number of mode j , and N is
the number of field modes (here we take N to be finite).
1. Show that the canonical partition function of the system can be written as
!
N
1
Q=  . (2.6.2)
j =1
1 − exp −βj

Hint: you will have to use the following identity for | x |< 1:

i=∞
1
xi = . (2.6.3)
1−x
i=0

For the product of partition functions of two independent systems A and B,

we can write
QA × QB = QAB (2.6.4)
when A ∩ B =  and A ∪ B = AB. 
2. Show that the average occupancy number of state j , nj , is equal to
 ∂ ln Q 1
nj =  =   . (2.6.5)
∂ −βj exp βj − 1

3. Describe the behavior of nj when T → ∞ and when T → 0.
 Thermodynamics and statistical mechanics Chapter | 2 49

Question 9 (Ideal gas (Part 2)). An ideal gas is placed in a constant gravita-
tional field. The potential energy of N gas molecules at height z is Mgz, where
M = mN is the total mass of N molecules, and g the gravitational acceleration.
The temperature in the system is uniform, and the system is infinitely large. We
assume that the system is locally in equilibrium, so we are allowed to use a local
partition function.
1. Show that the grand-canonical partition function of a system in volume V at
height z is equal to
∞
 exp [βμN]
Q (μ, V , T , z) = d exp [−β (H0 + Mgz)] (2.6.6)
h3N N !
N =0

in which H0 is the Hamiltonian of the system at z = 0.

2. Explain that a change in z is equivalent to a change in chemical potential,
μ. Use this to show that the pressure of the gas at height z is equal to

p (z) = p (z = 0) × exp [−βmgz] . (2.6.7)

(Hint: you will need the formula for the chemical potential of an ideal gas.)
Exercise 1 (Distribution of particles). Consider an ideal gas of N particles in a
volume V at constant energy E. Let us divide the volume into p identical compart-
ments. Every compartment contains ni molecules such that

i=p

N= ni . (2.6.8)
i=1

An interesting quantity is the distribution of molecules over the p compartments.

Because the energy is constant, every possible eigenstate of the system will be
equally likely. This means that in principle it is possible that one of the compart-
ments is empty.
1. On the book’s website you can find a program that calculates the distribution of
molecules among the p compartments. Run the program for different numbers
of compartments (p) and total number of gas molecules (N ). The output of the
program is the probability of finding x particles in a particular compartment as
a function of x.
2. What is the probability that one of the compartments is empty?
3. Consider the case p = 2 and N even. The probability of finding N/2 + n1
molecules in compartment 1 and N/2 − n1 molecules in compartment 2 is
given by

N!
P (n1 ) = . (2.6.9)
(N/2 − n1 )!(N/2 + n1 )!2N

Compare your numerical results with the analytical expression for different val-
ues of N . Show that this distribution is a Gaussian for small n1 /N . Hint: For
50 PART | I Basics

x > 10, it might be useful to use Stirling’s approximation:

1 1
x! ≈ (2π ) 2 x x+ 2 exp [−x] . (2.6.10)

Exercise 2 (Boltzmann distribution). Consider a system of N energy levels with

energies 0, , 2, · · · , (N − 1)  and  > 0.
1. Calculate, using the given program, the occupancy of each level for different
values of the temperature. What happens at high temperatures?
2. Change the program in such a way that the degeneracy of energy level i equals
i + 1. What do you see?
3. Modify the program in such a way that the occupation of the energy levels, as
well as the partition function (q), is calculated for a heteronuclear linear rotor
with moment of inertia I . Compare your result with the approximate result

2I
q= (2.6.11)
β 2

for different temperatures. Note that the energy levels of a linear rotor are

2
U = J (J + 1) (2.6.12)
2I
with J = 0, 1, 2, · · · , ∞. The degeneracy of level J equals 2J + 1.

Exercise 3 (Coupled harmonic oscillators). Consider a system of N harmonic os-

cillators with a total energy U . A single harmonic oscillator has energy levels
0, , 2, · · · , ∞ ( > 0). All harmonic oscillators in the system can exchange energy.
1. Invent a computational scheme to update the system at constant total energy
(U ). Compare your scheme with the scheme that is incorporated in the com-
puter code that you can find on the book’s website.
2. Make a plot of the energy distribution of the first oscillator as a function of
the number of oscillators for a constant value of U/N . Which distribution is
recovered when N becomes large? What is the function of the other N − 1
harmonic oscillators? Explain.
3. Compare this distribution with the canonical distribution of a single oscillator
at the same average energy (use the option NVT).
4. How does this exercise relate to the derivation of the Boltzmann distribution for
a system at temperature T ?

Exercise 4 (Random walk on a 1d lattice). Consider the random walk of a single

particle on a line. The particle performs jumps of fixed length. Assuming that the
probability for forward or backward jumps is equal, the mean-squared displace-
ment of a particle after N jumps is equal to N . The probability that, after N jumps,
the net distance covered by the particle equals n is given by
 
1 2 n2
ln [P (n, N )] ≈ ln − .
2 πN 2N

1. Derive this equation using Stirling’s approximation for ln x!.

 Thermodynamics and statistical mechanics Chapter | 2 51

2. Compare your numerical result for the root mean-squared displacement with
the theoretical prediction (the computed function P (n, N ). What is the diffu-
sivity of this system?
3. Modify the program in such a way that the probability to jump in the forward
direction equals 0.8. What happens?

Exercise 5 (Random walk on a 2d lattice). Consider the random walk of N particles

on a M × M lattice. Two particles cannot occupy the same lattice site. On this
lattice, periodic boundaries are used. This means that when a particle leaves the
lattices it returns on the opposite side of the lattice; i.e., the coordinates are given
modulo M.
1. What is the fraction of occupied sites (θ) of the lattice as a function of M and
N?
2. Make a plot of the diffusivity D as a function of θ for M = 32. For low values of
θ , the diffusivity can be approximated by

D ≈ D0 (1 − θ) .

Why is this equation reasonable at low densities? Why does it break down at
higher densities?
3. Modify the program in such a way that the probability of jumping in one direc-
tion is larger than the probability of jumping in the other direction. Explain the
results.
4. Modify the program in such a way that periodic boundary conditions are used in
one direction and reflecting boundary conditions in the other. What happens?