Thermodynamics and Statistical Physics

1
B. Zheng
Zhejiang Institute of Modern Physics, Zhejiang University,
Hangzhou 310027, P.R. China

1
e-mail: [email protected]; tel: 0086-571-87952753; Fax: 0086-571-87952754
Chapter 1

Non-equilibrium statistical
mechanics

Non-equilibrium physical phenomena

• Dynamic relaxation process

The relaxation process from a non-equilibrium state
to the equilibrium state, with or without external
fields. For example, the system is at a high tem-
perature state at t = 0, and then suddenly released
to a low temperature state at t > 0. In this dy-
namic process, the physical observables obviously
vary with time t.
• Dynamic transportation process
The system is at the ‘stationary’ state, and most
of the physical observables do not vary with time
t. However, there exist ‘currents’, such as the par-
ticle, energy, and charge, or domain wall, vortex,
and skyrmion etc. These currents imply that some
quantities such as the positions of the particles are

1
 time dependent. More importantly, one generally
needs dynamic equations to describe the system.
Here the so-called ‘stationary’ state is the state
when t goes to infinity. Naturally, the relaxation
processes from non-stationary states to the station-
ary state are also non-equilibrium physical physical
phenomena.
• Driven dynamic systems
The system is at the ‘stationary’ state, but contin-
uously driven by external forces, which are macro-
scopic. Or, the system can not be simply described
by the Hamiltonian, or the Hamiltonian is unknown.
These systems are somewhat similar to the equilib-
rium ones, but can not be described by the ensem-
ble theory.
Dynamic responses and dynamic fluctuations around
equilibrium or stationary states are also considered
as nonequilibrium phenomena.

Briefly, one needs equations of motion to describe the

non-equilibrium physical phenomena. We are not able
to deal with arbitrary dynamic processes, but only those
with relatively simple macroscopic conditions such as
the external forces, boundary conditions and initial con-
ditions etc.

2
1.1 Boltzmann equation
Quasi-independent identical particles form a dynamic
system, and the number of particles is conserved. The
microscopic state of a single particle is described by
(⃗r, p⃗), and the space spanned by (⃗r, p⃗) is called the µ
space.

The equilibrium state

A macroscopic state of the system is described by the
density distribution function for the number of particles
in the µ space. In the canonical ensemble, it takes the
form
f (⃗r, p⃗) = exp(−βε)
where ε = ε(⃗r, p⃗) is the energy of a single particle. Please
note that f (⃗r, p⃗) is only the function of ε, not generally
of (⃗r, p⃗). This is a characteristic of the equilibrium state.

The non-equilibrium state

The density distribution function for the number of
particles is time dependent f (⃗r, p⃗, t)

The key problem: how to derive f (⃗r, p⃗, t)?

Obviously, if t and ⃗r are at the microscopic scale,

the difficulty to derive f (⃗r, p⃗, t) is similar to solve the
microscopic equations of motion, or it is not meaning-
ful. Therefore, t and ⃗r should be at the mesoscopic or
macroscopic scale. Our strategy is

3
 • First write down the equations of motion
• Then solve the equations

Question: why?

1.1.1 Without external forces and collisions

Since the particles are assumed to be quasi-independent,
there are no general interactions between the particles.
But external forces and collisions may exist! In fact,
the case without external forces and collisions can be
studied in mechanics. We may simply write down the
conservation equation for the number of particles
∂f (⃗r, p⃗, t) ⃗
+ ∇ · [⃗v f (⃗r, p⃗, t)] = 0
∂t
where p⃗ = m⃗v .

How to justify it?

We integrate both sides of the equation over ⃗r in a
small volume ∆V , as shown in Fig. 1.1

∫ ∫
∂f (⃗r, p⃗, t)
d⃗r + ∇
⃗ · [⃗v f (⃗r, p⃗, t)]d⃗r = 0
∆V ∂t ∆V

∫ ∫
∂
⇒ f (⃗r, p⃗, t)dV = − [⃗v f (⃗r, p⃗, t)] · dS
⃗
∂t ∆V ∆S

• Left hand side: the increasing rate of the number

of the particles in ∆V

4
 dS

V dS

Figure 1.1:

p''

p p'

p'''

Figure 1.2:

5
 • Right hand side: the number of the particles flowing
into ∆V in a unit time.
⃗
As shown in Fig. 1.1, the ‘positive’ direction of dS
is usually defined as the normal direction pointing
outward. Therefore, the projection of ⃗v on the di-
rection of −dS
⃗ is dl/dt, and l is the distance that the
particles move into ∆V along the negative normal
direction. In other words,
dl · dS
−f⃗v · dS
⃗=f
dt
dV
=f
dt
is the number of the particles flowing into ∆V through
dS⃗ in a unit time. The integration over the surface
∆S of ∆V is the total number of the particles flow-
ing into ∆V

Since the number of particles is conserved, the above

conservation equation for the number of particles holds.

An alternative way:
The variation of the number of the particles at (⃗r, p⃗)
in a small time dt is

d̄f (⃗r, p⃗, t) = f (⃗r, p⃗, t + dt) − f (⃗r, p⃗, t)

Without external forces and collisions, the momentum

of a particle is a constant at least at a local time. The
particles at site ⃗r at time t + dt come from site ⃗r − ⃗v dt at
time t, since the spatial displacement of the particles in

6
dt is ⃗v dt. Thus
∂f
= [f (⃗r, p⃗, t + dt) − f (⃗r, p⃗, t)] /dt
∂t
= [f (⃗r − ⃗v dt, p⃗, t) − f (⃗r, p⃗, t)] /dt
∂f
= −⃗v · = −∇ ⃗ · (⃗v f )
∂⃗r
Here we keep in mind that the velocity, i.e., the mo-
mentum p⃗, is a constant.

1.1.2 With external forces but without collisions

The variation of the number of the particles at ⃗r and p⃗
in a small time dt is
d̄f (⃗r, p⃗, t) = f (⃗r, p⃗, t + dt) − f (⃗r, p⃗, t)
= f (⃗r − ⃗v dt, p⃗ − p⃗˙dt, t) − f (⃗r, p⃗, t)

∂f ∂f ∂f
∴ = −⃗v · − p⃗˙ ·
∂t ∂⃗r ∂⃗p
∂f ∂f
= −⃗v · − F⃗ ·
∂⃗r ∂⃗p
Here the external force F⃗ is assumed to be only (⃗r, p⃗, t)-
dependent, not particle-dependent.
Question: how to derive the equation from the con-
servation of the number of particles?

1.1.3 With external forces and collisions

( )
∂f ∂f ∂f ∂f
+ ⃗v · + F⃗ · =
∂t ∂⃗r ∂⃗p ∂t c

7
where (∂f /∂t)c is the variation of the number of particles
induced by the collisions of the particles.

This is the so-called Boltzmann equation. In princi-

ple, one may solve all the dynamic behaviors of a dy-
namic system composed of quasi-independent identical
particles.

In fact, collisions are also a kind of forces. The dif-

ference between the external forces and the collisions is
actually not ‘external’ or ‘internal’. The external forces
can be from either the environment or other particles
inside the system. The collisions can be either between
the particles or between the particles and the environ-
ment. Therefore, we should carefully distinguish the
external forces and collisions.
• Collisions occur in microscopic time and spatial scales.
In the Boltzmann equation, t and ⃗r are least meso-
scopic, and collisions are considered to take place
at a certain point (⃗r, t).
• External forces act on larger time and spatial scales.

Assumptions:
• There are only binary collisions.
• Boundary conditions are not important.
• External forces affect only the motion of individual
particles, not the collisions.

8
 • f (⃗r, p⃗, t) at different ⃗r and p⃗ do not correlate each
other. In fact, there should be certain correlations,
if one traces the microscopic motion of the particles.
But they are neglected.
Question: give some examples.
• The temporal and spatial scales in the equation are
much larger than those of the particles.

Let us consider the binary collision as shown in Fig. 1.2.

• Initial state p⃗, p⃗′
• Final state p⃗′′ , p⃗′′′
• Energy conservation
ε + ε′ = ε′′ + ε′′′

• Momentum conservation
p⃗ + p⃗′ = p⃗′′ + p⃗′′′

The reverse process

• Initial state p⃗′′ , p⃗′′′

• Final state p⃗, p⃗′
• The same energy and momentum conservation

At site ⃗r and time t, the particle number with p⃗, p⃗′

produced in collisions by those with p⃗′′ , p⃗′′′ is
f (⃗r, p⃗′′ , t)f (⃗r, p⃗′′′ , t)R

9
where R is the rate of the binary collision. When the
above form is written down, it is already assumed that
f (⃗r, p⃗′′ , t) and f (⃗r, p⃗′′′ , t) are not correlated each other.
The detailed form of R is generally unknown, but it
should obey the energy conservation and momentum
conservation.

R ∝ δ(⃗p + p⃗′ − p⃗′′ − p⃗′′′ ) · δ(ε + ε′ − ε′′ − ε′′′ )

Therefore, at (⃗r, p⃗) and time t, the increasing rate of the

number of particles is
∫
f (⃗r, p⃗′′ , t)f (⃗r, p⃗′′′ , t)Rd⃗p′ d⃗p′′ d⃗p′′′

On the other hand, the decreasing rate is

∫
f (⃗r, p⃗, t)f (⃗r, p⃗′ , t)Rd⃗p′ d⃗p′′ d⃗p′′′

Therefore,
( ) ∫
∂f
= (f ′′ f ′′′ − f f ′ )Rd⃗p′ d⃗p′′ d⃗p′′′
∂t c

f = f (⃗r, p⃗, t) f ′ = f (⃗r, p⃗′ , t)

f = f (⃗r, p⃗′′ , t)
′′
f ′′′ = f (⃗r, p⃗′′′ , t)

Note that the time t here is at the mesoscopic scale, and

all details in collisions are ignored.

Exercise: assume ε = p⃗2 /2m + U (⃗r), perform the inte-

gration over p⃗′′ and p⃗′′′ in (∂f /∂t)c .

10
1.2 Simple examples
1.2.1 Equilibrium state
The equilibrium state requires

f (⃗r, p⃗, t) = f (⃗r, p⃗)

and one additionally assumes

( )
∂f
=0
∂t c
Note that from the form of the binary collisions dis-
cussed in the previous section, the latter seems to be not
a simple result of the former. Anyway, one could search
for possible solutions “with such a more restricted con-
dition”.
Therefore,
p⃗ ∂f ∂f
· + F⃗ · =0
m ∂⃗r ∂⃗p
Let us assume

f (⃗r, p⃗) = f (ε)

2
p⃗ ∂U
ε= + U (⃗r), F⃗ = −
2m ∂⃗r

11
Then compute
∂f ∂f ∂ε p⃗ ∂f
= · =
∂⃗p ∂ε ∂⃗p m ∂ε
∂f ∂f ∂U ∂f
= · = −F⃗
∂⃗r ∂ε ∂⃗r ∂ε

p⃗ ( ⃗ ) ∂f p⃗ ∂f
∴ · −F + F⃗ · =0
m ∂ε m ∂ε
In other words, f (ε) is indeed the possible solution for
the equilibrium state.
Now we need to verify whether (∂f /∂t)c = 0 holds.
Obviously, an arbitrary form of f (ε) can not guarantee
(∂f /∂t)c = 0. An ansatz is the Boltzmann distribution,
f (ε) ∼ exp(−βε). From the uniqueness of the equilibrium
state, this should be the solution.

Question: why? (since f ′′ f ′′′ − f f ′ = 0)

1.2.2 Without external forces and collisions

Assuming ( )
∂f
= 0, F⃗ = 0
∂t c
therefore
∂f ∂f
+ ⃗v · =0
∂t ∂⃗r
The solution is simply written down
f (⃗r, p⃗, t) = Φ(⃗r − ⃗v t, p⃗)
where Φ(⃗r, p⃗) is the initial distribution.

12
 Example: at t = 0, a gas with temperature T con-
denses at the origin, i.e.,

Φ(⃗r, p⃗) = N (2πmT )−3/2 e−⃗p

2
/2mT
δ(⃗r),

compute the distribution function for the number of

particlea at time t.

Question: why is it in such a form?

Note that the origin ⃗r = 0 is a mesoscopic or macro-
scopic point, and at the microscopic scale, the particles
still move.
Solution:
∫
n(⃗r, t) = d⃗pf (⃗r, p⃗, t)
∫
= d3 pΦ(⃗r − ⃗v t, p⃗)
∫
−3/2
= N (2πmT ) d3 p exp(−⃗p2 /2mT )δ(⃗r − ⃗v t)

Since p⃗ = m⃗v , and

( ) ( ) ( )
p⃗ m 3 3 m⃗r
δ ⃗r − t =
3
δ p⃗ −
m t t
we have
( m )3
−3/2
n(⃗r, t) = N (2πmT ) exp(−m⃗r2 /2T t2 )
t
As time evolves, the particles diffuse from the origin
to ⃗r.

13
1.2.3 With external forces and without collisions

∂f ⃗ + F⃗ · ∂f = 0
+ ⃗v · ∇f
∂t ∂⃗p
For example, for a two-dimensional dilute gas of elec-
trons, the Coulomb force and the electric potential are

F⃗ (⃗r, t) = e∇ϕ
⃗

∫
e
ϕ(⃗r, t) = − d2⃗r′ n′ (⃗r′ , t)
|⃗r − ⃗r |
′

N
n′ (⃗r, t) = n(⃗r, t) − 2
L
Here N/L2 is the average density of electrons, n(⃗r, t) is
the density distribution of electrons, and n′ (⃗r, t) is the
net one. n(⃗r, t) is related to f (⃗r, p⃗, t) by
∫
n(⃗r, t) = d2 p⃗ f (⃗r, p⃗, t)

Note that
• The spatial scale of the system is much larger than
that of the electron, therefore, the electric forces
between the volume elements are considered to be
‘external force’.
• If the electrons are uniformly distributed, all forces
are canceled out.
• The Coulomb force is a long-range force, and it
should not be considered as ‘collisions’.

14
 How to solve the equation? The difficulty is that
′
n (⃗r, t) is related to f (⃗r, p⃗, t), and the equation looks to
be nonlinear! We need approximation to reduce the
nonlinearity.

Let us introduce a trial form of the solution

⃗
n′ (⃗r, t) = n′0 e−iωt+ik·⃗r , n′0 is a constant
∫
e ′ −iωt+i⃗k·⃗r′
ϕ(⃗r, t) = − d2⃗r′ n e
|⃗r − ⃗r′ | 0
2πen′0 −iωt+i⃗k·⃗r
=− e
k
where ω and ⃗k are the parameters which feature the
solution.
Exercise: compute
∫
1 ⃗ 2π
d2⃗r eik·⃗r =
r k
Hint:
∫ ∫ ∞ ∫ π ∫ ∞ ∫ 2π
1 ikr cos ϕ
e rdrdϕ = dr dϕeikr cos ϕ
+ dr dϕ · · ·
r 0 ∫ ∞ ∫ π
0 0 π

= ··· + dr dϕe−ikr cos ϕ

∫ ∞ 0∫ π 0
= dr dϕeikr cos ϕ
−∞
∫ π 0
2π 2π
= dϕδ(cos ϕ) =
k 0 k
Therefore,
2 ′
⃗ = −i 2πe n0 ⃗ke−iωt+i⃗k·⃗r
F⃗ = e∇ϕ
k
15
 Let us assume n′0 is small, and take the first order
approximation
⃗
f (⃗r, p⃗, t) = f0 (⃗p) + f ′ (⃗p)e−iωt+ik·⃗r , f ′ (⃗p) ∝ n′0
f0 (⃗p) ≡ n(2πmT )−1 e−⃗p /2mT
2

f ′ (⃗p) ≪ f0 (⃗p) (∵ n′0 small)

Here f0 (⃗p) is the distribution function in equilibrium,
and the constant n is the density of electrons. The
non-equilibrium effect is induced by n′ (⃗r, t), which is de-
⃗
scribed by the term f ′ (⃗p)e−iωt+ik·⃗r .
This is an ansatz, which is relatively uniform in space
and time. Naturally, the equilibrium state is more uni-
form in space and time. But we are looking for the non-
equilibrium solution, and this Ansatz is the one not far
from equilibrium.

Inserting the ansatz into the Boltzmann equation, we

obtain
⃗ ′ 2πe2 n′0 ⃗
−i(ω − k · ⃗v )f (⃗p) − i k · ⃗v f0 /T = 0
k
where we have used the result
∂f0 (⃗p)
= −⃗v f0 (⃗p)/T
∂⃗p
and neglecting the higher order term ∝ f ′ (⃗p)n′0 .
′ 2πe2 n′0 ⃗k · ⃗v f0 (⃗p)/T
∴ f (⃗p) = ·
k ω − ⃗k · ⃗v
Note that f ′ is a function of p⃗, but n′0 is only a constant.
On the other hand
∫ ∫
′ ′ 2πe2 n′0 ⃗ v f0 (⃗p)/T
2 k ·⃗
2
n0 = d p⃗f (⃗p) = · d p⃗
k ω − ⃗k · ⃗v
16
That is, the condition for the solution is
∫ ⃗ v f0 /T
2πe2 2 k ·⃗
1= d p⃗
k ω − ⃗k · ⃗v
Now there is no equilibrium state, and the electrons are
oscillating. We assume k/ω is small, i.e., the distribution
is more uniform in space than in time,
∫ ⃗ v )2
2πe2 2 (k · ⃗
1= d p⃗ f0 (⃗p)/T + · · ·
k ω2
From the symmetry,

(⃗k · ⃗v )2 ∼ kx2 vx2 + ky2 vy2

∫ ∫
d p⃗vx f0 (⃗p) = d2 p⃗vy2 f0 (⃗p)
2 2

therefore ∫
d2 p⃗(⃗k · ⃗v )2 f0 (⃗p) ∼ k2

Finally we obtain
2πe2 n k
1≈ · 2
m ω
2πe2 n
i.e., ω2 = k
m
This is the so-called dispersion relation. For the classical
free particle, it is
ω = k2

17
1.3 Langevin equation and Fokker-Planck equa-
tion of a single degree of freedom
1.3.1 Langevin equation
The action or Hamiltonian of a particle with a single
degree of freedom is denoted as S(x), and x is a real
variable.
The Langevin equation is written as

dx(t) ∂S(x)
=− + η(t) (1.3.1)
dt ∂x
where η(t) is the Gaussian white noise
⟨η(t)⟩η = 0, ⟨η(t)η(t′ )⟩η = 2δ(t − t′ ) (1.3.2)
In the Langevin equation, the time t is usually meso-
scopic. For a fixed t,
P (η) ∼ e−ση
2
(1.3.3)
Naturally, the solution x(t) is also the functional of the
noise η(t).
Without the random noises η(t), the equilibrium state
is given by ẋ = −∂S/∂x = 0, i.e., the state with the min-
imum energy. With the random noise η(t), the particle
is driven away from the state with the minimum energy,
and equilibrates at an equilibrium state which is gen-
erally described a probability distribution P (x) around
the state with the minimum energy.

An example: the Brownian motion - a pollen moving

in water
d⃗v (t)
m = −γ⃗v (t) + η(t), γ > 0 (1.3.4)
dt
18
The solution in one dimension
∫
1 t ′ ′ −γ(t−t′ )/m
⃗v (t) = dt η(t )e + v(0)e−γt/m
m 0
Therefore
∫ t
1 ′ ′′
⟨v 2 (t)⟩η = v 2 (0)e−2γt/m + 2 dt′ dt′′ ⟨η(t′ )η(t′′ )⟩η e−γ(2t−t −t )/m
m 0
∫ t
−2γt/m 2 ′
2
= v (0)e + 2 dt′ dt′′ e−2γ(t−t )/m
m 0
For γ = 0
2t
⟨v 2 (t)⟩η = v 2 (0) +
m2
This actually is the well-known random walk, but in the
velocity space.
For γ ̸= 0
1
⟨v 2 (t)⟩η = v 2 (0)e−2γt/m + (1 − e−2γt/m )
γm
In the equilibrium state
1
⟨v 2 (t)⟩η =
γm

How may Eq. (1.3.4) reduce to Eq. (1.3.1)? As-

suming there may be still an interaction term D(⃗v (t)) in
addition to −γ⃗v (t), it is difficult to solve the equation
of motion. Thus one tries to decompose md⃗v (t)/dt into
two parts, the fast mode and slow mode, which may
be absorbed into η and D(⃗v (t)) respectively. Then the
equation of motion is reduced to Eq. (1.3.1).

19
 Due to the random noise, the Langevin equation is
more complicated than one imagines from its form. We
must carefully consider the ‘singularity’ induced by the
random noise. In other words, one should carefully de-
fine the calculus of η.
For simplicity, we discretize the time t. Such a dis-
cretization is convenient in numerical simulations and
also intuitive in theoretical calculations. A standard
discretization of δ(t − t′ ) may be
2
⟨η(t)η(t′ )⟩η = δtt′
∆t
and it requires
∆t
σ=
4
in the probability distribution in Eq. (1.3.3). The Langevin
equation is now written as
∂S(x(t))
∆x(t) = x(t + ∆t) − x(t) = − ∆t + ∆t η(t)
∂x(t)
√ √
Since η is the order of 1/ ∆t, we rescale ∆t/2 η → η,
and the Langevin equation then transforms to
∂S(x(t)) √
∆x(t) = − ∆t + 2∆t η(t) (1.3.5)
∂x(t)
with
⟨η(t)η(t′ )⟩η = δtt′
An important feature√ of the equation is that the term
of η is the order of ∆t rather than ∆t. The Langevin
equation in this form can be straightforwardly solved in
numerical simulations.

20
1.3.2 Fokker-Planck equation
The solution of Eq. (1.3.5), xη (t), is the functional of η,
and xη (t) is also a random variable. For example, given
an initial value x0 , the value of x at time t is still not
fixed, and it depends on the noise η. In other words, x
at time t obeys a certain probability distribution P (x; t).
A physical observable can be calculated by the aver-
age either over the random noise η, or over the proba-
bility distribution P (x; t)
∫
⟨O(xη (t))⟩η = dxP (x; t)O(x) (1.3.6)

Question: what is the meaning of ⟨O(xη (t))⟩η ?

Answer: one should average over the noise η(t′ ) for
all t′ < t.

∆⟨O(xη )⟩η = ⟨∆O(xη (t))⟩η

∂O(xη (t)) 1 ∂ 2 O(xη (t))
= ⟨ ∆xη (t) + (∆xη (t))2 + ...⟩η
∂xη (t) 2 ∂ 2 xη (t)
∂O(xη (t)) ∂S(xη (t)) √
= ⟨ [− ∆t + 2∆t η(t)]
∂xη (t) ∂xη (t)
1 ∂ 2 O(xη (t)) ∂S(xη (t)) √
+ 2
[− ∆t + 2∆t η(t)]2 + ...⟩η
2 ∂ xη (t) ∂xη (t)
Since xη (t) depends only on η(t′ ) with t′ < t, not on η(t),
∂O(xη (t)) ∂O(xη (t)) ∂S(xη (t))
⟨ ∆xη (t)⟩η = −⟨ ∆t⟩η
∂xη (t) ∂xη (t) ∂xη (t)
Note that this property essentially replies on the calcu-
lus, and for example, one can not simply change O(xη (t))

21
to O(xη (t + ∆t)) in the Langevin equation. Neglecting
higher order terms,
⟨...(∆xη (t))2 ⟩η = ⟨...2∆t⟩η
Therefore
∆⟨O(xη (t))⟩η ∂O ∂S ∂ 2 O
= ⟨− + 2 ⟩η
∆t ∫ ∂x ∂x ∂ x
( )
∂O ∂S ∂ 2 O
= dxP (x; t) − + 2
∂x ∂x ∂ x
∫ ( 2 )
∂ ∂ ∂S
= dxO(x) + P (x; t)
∂ 2 x ∂x ∂x
When the integration by part is performed, we assume
already P (±∞; t) = 0.
On the other hand,
∫
∆⟨O(xη (t))⟩η ∂P (x; t)
= dxO(x)
∆t ∂t
Thus we obtain the Fokker-Planck equation
∂P (x; t)
= −HF P P (x; t)
∂t
with ( )
∂ ∂ ∂S
HF P = − +
∂x ∂x ∂x
Note that the operator ∂/∂x acts on all objects on its
right side. In the equilibrium state, ∂P (x; t)/∂t = 0,
HF P P (x; ∞) = 0
A possible solution is
P (x; ∞) ∼ e−S(x)

22
 Question: under what conditions the above solution
is the real solution for the equilibrium state?
Both the Fokker-Planck equation and the Boltzmann
equation are formally similar to the Schrödinger equa-
tion, and may be solved with the ideology in quantum
mechanics. For example, one may expand the solution
with the eigenfunctions of the Hamiltonian.

23