PHY114 - Thermal Physics and Radiation

Lecture 02
Summer 2024
Sayeda Tashnuba Jahan

Probability Distributions
Discrete Probability Distribution
Discrete random variables can only take a finite number of values. Examples include
the number obtained when throwing a die (1, 2, 3, 4, 5, or 6 ), the number of children
in each family (0, 1, 2, . . .), and the number of people killed per year in the UK in
bizarre gardening accidents (0, 1, 2, . . .). Let x be a discrete random variable which
takes values xi with probability Pi . We require that the sum of the probabilities of
every possible outcome adds up to one. This may be written
X
Pi = 1.
i

We define the mean (or average or expected value) of x to be

X
⟨x⟩ = xi P i .
i

The idea is that you weight by its probability each value taken by the random vari-
able x.

It is also possible to define the mean squared value of x using

X
x2 = x2i Pi .
i

In fact, any function of x can be averaged, using (by analogy)

X
⟨f (x)⟩ = f (xi ) Pi .
i

Now let us actually evaluate the mean of x for a particular discrete distribution.

1 1 1
Let x take values 0,1, and 2 with probabilities , ,
2 4
and 4
respectively. Calcu-
late ⟨x⟩ and ⟨x2 ⟩.

Solution:

1 1 1
P
First check that Pi = 1. Since 2
+ 4
+ 4
= 1, this is fine. Now we can cal-
culate the averages as follows:
X
⟨x⟩ = x i Pi
i
1 1 1
=0· +1· +2·
2 4 4
3
= .
4
Again, we find that the mean ⟨x⟩ is not actually one of the possible values of x. We
can now calculate the value of ⟨x2 ⟩ as follows:
X
x2 = x2i Pi
i
1 1 1
=0· +1· +4·
2 4 4
5
=
4

Continuous Probability Distribution

Let x now be a continuous random variable which has a probability P (x)dx of having
a value between x and x + dx. Continuous random variables can take a range of
possible values. Examples include the height of children in a class, the length of time
spent in a waiting room, and the amount a person’s blood pressure increases when
reading their mobile-phone bill. These quantities are not restricted to any finite set
of values, but can take a continuous set of values.
As before, we require that the total probability of all possible outcomes is one.
Because we are dealing with continuous distributions, the sums become integrals,
and we have Z
P (x)dx = 1

The mean is defined as Z

⟨x⟩ = xP (x)dx.

Similarly, the mean square value is defined as

Z
2
x = x2 P (x)dx,

and the mean of any function of x, f (x), can be defined as

Z
⟨f (x)⟩ = f (x)P (x)dx.

Linear Transformations
Sometimes one has a random variable, and one wants to make a second random
variable by performing a linear transformation on the first one. If y is a random
variable, which is related to the random variable x by the equation

y = ax + b,

where a and b are constants, then the average value of y is given by

⟨y⟩ = ⟨ax + b⟩ = a⟨x⟩ + b.

The proof of this result is straightforward and is left as an exercise.

Variance
We now know how to calculate the average of a set of values, but what about the
spread in the values? The first idea one might have to quantify the spread of values
in a distribution is to consider the deviation from the mean for a particular value of
x. This is defined by
x − ⟨x⟩.
This quantity tells you by how much a particular value is above or below the mean
value. We can work out the average of the deviation (averaging over all values of x
) as follows:
⟨x − ⟨x⟩⟩ = ⟨x⟩ − ⟨x⟩ = 0,
which follows from the equation for linear transformation. Thus the average devi-
ation is not going to be a very helpful indicator! Of course, the problem is that
the deviation is sometimes positive and sometimes negative, and the positive and
negative deviations cancel out. A more useful quantity would be the modulus of the
deviation,
|x − ⟨x⟩|,
which is always positive, but this will suffer from the disadvantage that modulus
signs in algebra can be both confusing and tedious. Therefore, another approach
is to use a different quantity which is always positive, the square of the deviation,
(x − ⟨x⟩)2 . This quantity is what we need: always positive and easy to manipulate
algebraically. Hence, its average is given a special name, the variance. Consequently
the variance of x, written as σx2 , is defined as the mean squared deviation:

σx2 = (x − ⟨x⟩)2 .

We will call σx the standard deviation, and it is defined as the square root of the
variance: p
σx = ⟨(x − ⟨x⟩)2 ⟩.
The standard deviation represents the "root mean square" (known as the "rms")
scatter or spread in the data. The following identity is extremely useful:

σx2 = (x − ⟨x⟩)2
= x2 − 2x⟨x⟩ + ⟨x⟩2
= x2 − 2⟨x⟩⟨x⟩ + ⟨x⟩2
= x2 − ⟨x⟩2 .

Independent Variables
If u and v are independent random variables the probability that u is in the range
from u to u + du and v is in the range from v to v + dv is given by the product

Pu (u)duPv (v)dv.
Hence, the average value of the product of u and v is
ZZ
⟨uv⟩ = uvPu (u)Pv (v)du dv
Z Z
= uPu (u)du vPv (v)dv

= ⟨u⟩⟨v⟩,

because the integrals separate for independent random variables. This implies that
the average value of the product of u and v is equal to the product of their average
values.

Suppose that there are n independent random variables, Xi , each with the same
2
mean ⟨X⟩ and variance σX . Let Y be the sum of the random variables, so that
Y = X1 + X2 + · · · + Xn . Find the mean and variance of Y .

Solution:

The mean of Y is simply

⟨Y ⟩ = ⟨X1 ⟩ + ⟨X2 ⟩ + · · · + ⟨Xn ⟩ ,

but since all the Xi have the same mean ⟨X⟩ this can be written

⟨Y ⟩ = n⟨X⟩.

Hence the mean of Y is n times the mean of the Xi . To find the variance of Y , we
can use the formula
σY2 = Y 2 − ⟨Y ⟩2 .
Hence
Y 2 = X12 + · · · + XN2 + X1 X2 + X2 X1 + X1 X3 + · · ·
= X12 + · · · + XN2 + ⟨X1 X2 ⟩ + ⟨X2 X1 ⟩ + ⟨X1 X3 ⟩ + · · ·

There are n terms like ⟨X12 ⟩ on the right-hand side, and n(n − 1) terms like ⟨X1 X2 ⟩.
The former terms take the value ⟨X 2 ⟩ and the latter terms (because they are the
product of two independent random variables) take the value ⟨X⟩⟨X⟩ = ⟨X⟩2 .
Hence, using eqn 3.35,

Y 2 = n X 2 + n(n − 1)⟨X⟩2 ,

so that
σY2 = Y 2 − ⟨Y ⟩2
= n X 2 − n⟨X⟩2
2
= nσX .
An application is in the theory of random walks. Imagine a drunken person stagger-
ing out of a pub and attempting to walk along a narrow street (which confines him
or her to motion in one dimension). Let’s pretend that with each inebriated step, the
drunken person is equally likely to travel one step forwards or one step backwards.
The effects of intoxication are such that each step is uncorrelated with the previous
one. Thus the average distance travelled in a single step is ⟨X⟩ = 0. P After n such
steps, we would have an expected total distance travelled of ⟨Y ⟩ = ⟨Xi ⟩ = 0.
However, in this case the root mean squared distance is more revealing.√ In this case
⟨Y 2 ⟩ = n ⟨X 2 ⟩, so that the rms length of a random walk of n steps is n times the
length of a single step. This result will be useful in considering Brownian motion.

Random Walk and Binomial Distribution

The atomic approach to macroscopic properties came about from the study of the
kinetic theory of gases. Therefore, to understand the kinetic theory of gases, we
need to lay the foundation towards that theory. As mentioned earlier, it is rudimen-
tary that whenever systems are considered statistically (using probabilities), it is
necessary to set an assembly (ensemble) of many similarly prepared systems. Note
also that the probability depends very much on the nature of the ensemble which is
contemplated in defining this probability.

In the following discussion of basic probability concepts it will be useful to keep in

mind a specific simple, but important, illustrative example-the so-called "random-
walk problem." In its simplest idealized form, the problem can be formulated in the
following traditional way: A drunk starts out from a lamppost located on a street.
Each step he takes is of equal length l. The man is, however, so drunk that the
direction of each step-whether it is to the right or to the left-is completely indepen-
dent of the preceding step. All one can say is that each time the man takes a step,
the probability of its being to the right is p, while the probability of its being to the
left is q = 1 − p. (In the simplest case p = q, but in general p ̸= q. For example,
the street might be inclined with respect to the horizontal, so that a step downhill
to the right is more likely than one uphill to the left.)

Figure 1: The drunkard’s random walk in one dimension.

Choose the x axis to lie along the street so that x = 0 is the position of the point
of origin, the lamppost. Clearly, since each step is of length l, the location of the
man along the x axis must be of the form x = ml, where m is an integer (positive,
negative, or zero). The question of interest is then the following: After the man has
taken N steps, what is the probability of his being located at the position x = ml ?

This statistical formulation of the problem again implies that one considers a very
large number K of similar men starting from similar lampposts. (Alternatively, if
the situation does not change in time, e.g., if the man does not gradually sober up,
one could also repeat the same experiment ℜ times with the same man.) At each
step, one finds that a fraction p of the men moves to the right. One then asks what
fraction of the men will be located at the position x = ml after N steps.

One can readily generalize this one-dimensional problem to more dimensions whether
it, be two (a drunk starting out from a lamppost in the middle of a parking lot),
three, or more. One again asks for the probability that after N steps the man is
located at a given distance from the origin (although the distance is no longer of the
form ml, with m integral).

Figure 2: Example of a random walk in two dimensions.

The random walk problem illustrates some very fundamental results of proba-
bility theory. The techniques used in the study of this problem are powerful and
basic, and recur again and again throughout statistical physics. It is therefore very
instructive to gain a good understanding of this problem.