1 Distribution function

1.1 Monotone functions

We begin with a discussion of distribution functions as a traditional way of
introducing probability measures. It serves as a convenient bridge from
elementary analysis to probability theory, upon which the beginner may
pause to review his mathematical background and test his mental agility.
Some of the methods as well as results in this chapter are also useful in the
theory of stochastic processes.
In this book we shall follow the fashionable usage of the words "positive",
"negative", "increasing", "decreasing" in their loose interpretation. For
example, "x is positive" means "x > 0 " ; the qualifier "strictly" will be added
when "x > 0 " is meant. By a "function" we mean in this chapter a real
finite-valued one unless otherwise specified.
Let t h e n / b e an increasing function defined on the real line (—oo, +oo).
Thus for any two real numbers xx and x2,

(1) Xi < X2 =>f(xi) ^fixz)·

2 I DISTRIBUTION FUNCTION

We begin by reviewing some properties of such a function. The notation

"f f x" means " i < x, / - > * " ; "t ψ x " means " / > x, t->x".

(i) For each x9 both unilateral limits

(2) lim/(i) = / ( * - ) and lim/(0 = / ( * + )

exist and are finite. Furthermore the limits at infinity

exist; the former may be —oo, the latter may be +00.

This follows from monotonicity; indeed

(ii) For each JC,/IS continuous at x if and only if

To see this, observe that the continuity of a monotone function/at x is

equivalent to the assertion that

By (i), the limits above exist a s / ( * - ) a n d / ( x + ) and

(3)

from which (ii) follows.

In general, we say that the function/has a, jump at x iff the two limits in
(2) both exist but are unequal. The value of / a t x itself, viz. f(x), may be
arbitrary, but for an increasing/the relation (3) must hold. As a consequence
of (i) and (ii), we have the next result.

(iii) The only possible kind of discontinuity of an increasing function is

a jump. [The reader should ask himself what other kinds of discontinuity
there are for a function in general.]
If there is a jump at x, we call x a point of jump o f / a n d the number
f(x + ) — fix — ) the size of the jump or simply "the jump" at x.
It is worthwhile to observe that points of jump may have a finite point of
accumulation and that such a point of accumulation need not be a point of
jump itself. Thus, the set of points of jump is not necessarily a closed set.
 1.1 MONOTONE FUNCTIONS | 3

Example 1. Let x0 be an arbitrary real number, and define a function/as follows :

f(x) = 0 for x < x0 — 1 ;
= 1 n for xxo0 - -< x < xo - ^-qrj» «=1,2,...;
n n
= 1 for x > x0.
The point x0 is a point of accumulation of the points of jump <x0 , n > 1>,
but fis continuous at x0.

Before we discuss the next example, let us introduce a notation that will
be used throughout the book. For any real number /, we set
0 for x < t,
(4)
w δ«(χ)
*w ^1 for x > t
We shall call the function 8t the point mass at t.
Example 2. Let {an, n > 1} be any given enumeration of the set of all rational
numbers, and let {bn, n > 1} be a set of positive (> 0) numbers such that
00

2 bn < oo. For instance, we may take bn = 2~ n . Consider now

n= l

(5) /(*) = 2 Ma„(*).

n= l

Since 0 < δαη(Λ:) < 1 for every n and JC, the series in (5) is absolutely and uniformly
convergent. Since each δαη is increasing, it follows that if *i < x2,

/(*2> - / ( * i ) = I bn[8an(x2) - 8an(Xl)] > 0.

n=l

Hence/is increasing. Thanks to the uniform convergence (why?) we may deduce

that for each x,

(6) /(*+ ) " / ( * - ) = Σ ^η[δαη(Χ + ) - M*")]·

π=1

But for each «, the number in the square brackets above is 0 or 1 according as
x Φ an or x = an. Hence if x is different from all the an's, each term on the
right side of (6) vanishes ; on the other hand if x = ak, say, then exactly one term,
that corresponding to n = k, does not vanish and yields the value bk for the whole
series. This proves that the function / has jumps at all the rational points and
nowhere else.
This example shows that the set of points of jump of an increasing
function may be everywhere dense; in fact the set of rational numbers in the
example may be replaced by an arbitrary countable set without any change of
the argument. We now show that the condition ofcountability is indispensable.
By "countable" we mean always "finite (possibly empty) or countably infinite".
4 I DISTRIBUTION FUNCTION

(iv) The set of discontinuities o f / i s countable.

We shall prove this by a topological argument of some general applic-

ability. In Exercise 3 after this section another proof based on an equally
useful counting argument will be indicated. For each point of jump x consider
the open interval Ix = (f(x — ), / ( * + ))· If *' is another point of jump and
x < x\ say, then there is a point x such that x < x < x'. Hence by
monotonicity we have
/(*+)</(*)</(*'-).
It follows that the two intervals Ix and Ix, are disjoint, though they may abut
on each other if/(x + ) =f(x' — ). Thus we may associate with the set of
points of jump in the domain o f / a certain collection of pairwise disjoint open
intervals in the range of/ Now any such collection is necessarily a countable
one, since each interval contains a rational number, so that the collection of
intervals is in one-to-one correspondence with a certain subset of the rational
numbers and the latter is countable. Therefore the set of discontinuities is
also countable, since it is in one-to-one correspondence with the set of
intervals associated with it.

(v) Let/χ and/ 2 be two increasing functions and D SL set that is (every-
where) dense in (-00, +00). Suppose that

ΥΧΕΖ):Λ(Χ)=/2(Χ).

Then / i and / 2 have the same points of jump of the same size, and they
coincide except possibly at some of these points of jump.

To see this, let x be an arbitrary point and let tn e D, t'ne D, tn \ x,

t'n I x. Such sequences exist since D is dense. It follows from (i) that

(6)

In particular

The first assertion in (v) follows from this equation and (ii). Furthermore if
/x is continuous at x, then so i s / 2 by what has just been proved, and we have

proving the second assertion.

How c a n / i a n d / 2 differ at all? This can happen only when/^*) and
f2{x) assume different values in the interval ( Λ ( χ - ) , / i ( * + )) = (Λί* —),
Λ(* + ))· It w i l l t u r n o u t in Chapter 2 (see in particular Exercise 21 of Sec. 2.2)
 1.1 MONOTONE FUNCTIONS | 5

that the precise value of / at a point of jump is quite unessential for our
purposes and may be modified, subject to (3), to suit our convenience. More
precisely, given the function / , we can define a new function / in several
different ways, such as

/(*)=/(*-), /(*)=/(*+), Kx)=Ax~)+2f{x+\

and use one of these instead of the original one. The third modification is
found to be convenient in Fourier analysis, but either one of the first two is
more suitable for probability theory. We have a free choice between them
and we shall choose the second, namely, right continuity.

(vi) If we put

t h e n / i s increasing and right continuous everywhere.

Let us recall that an arbitrary function g is said to be right continuous

at x iff lim g(t) exists and the limit, to be denoted by g(x + ) , is equal to g(x).
tix
To prove the assertion (vi) we must show that
VJC: lim/(f+) = / ( * + ) .
tix
This is indeed true for any / s u c h t h a t / ( i + ) exists for every t. For then:
given any e > 0, there exists 8(e) > 0 such that
Vse(x,x+8): \f(s) -f(x + )\ < e.
Let t e(x, x + 8) and let s j t in the above, then we obtain
| / ( / + ) - f(x+)\ <c,
which proves t h a t / i s right continuous. It is easy to see that it is increasing
if/ is so.

Let D be dense in (—oo, +oo), and suppose t h a t / i s a function with the

domain D. We may speak of the monotonicity, continuity, uniform con-
tinuity, and so on of / on its domain of definition if in the usual definitions
we restrict ourselves to the points of D. Even if / is defined in a larger
domain, we may still speak of these properties "on Z>" by considering the
"restriction o f / t o D " .

(vii) Let / b e increasing on D, and define/on (—oo, +oo) as follows:

Vx : / ( * ) = inf fit)-
x<teD

T h e n / i s increasing and right continuous everywhere.

6 I DISTRIBUTION FUNCTION

This is a generalization of (vi). / is clearly increasing. To prove right

continuity let an arbitrary x0 and e > 0 be given. There exists t0 e D, t0 > x0,
such that
f(t0) - €<f(x0)<f(t0).
Hence if t G D, X0 < t < t0, we have

o <Λ0 -/fro) <f(to) -A**) < c

This implies by the definition off that for xQ < x < t0 we have
0<f(x)-f(x0)<e.
Since e is arbitrary, it follows that / i s right continuous at x0, as was to be
shown.

EXERCISES

1. Prove that for the fin Example 2 we have

/ ( - o o ) = 0, /(+oo) = I bn.
n= l

2. Construct an increasing function on (—oo, +oo) with a jump of size

one at each integer, and constant between jumps. Such a function cannot
00

be represented as J bnSn(x) with bn = 1 for each n, but a slight modification

n= l
will do. Spell this out.
**3. Suppose t h a t / i s increasing and that there exist real numbers A and
B such that Vx : A < f(x) < B. Show that for each e > 0, the number of
jumps of size exceeding e is at most (B - A)/e. Hence prove (iv), first for
bounded/and then in general.
4. L e t / b e an arbitrary function on (—oo, +oo) and L be the set of x
where/is right continuous but not left continuous. Prove that L is a count-
able set. [HINT: Consider L n M n , where Mn = {x | 0(f; x) > l/n} and
0(f; x) is the oscillation o f / a t x.]
*5. L e t / a n d / b e as in (vii). Show that the continuity off on D does not
imply that o f / o n (-oo, +oo), but uniform continuity does imply uniform
continuity.
6. Given any extended-valued/on (-oo, +oo), there exists a countable
set D with the following property. For each t, there exist tne D, /n -> t such
t h a t / ( 0 = lim f(tn). This assertion remains true if " / n - w " is replaced
71-» 00

by "/ n j / " o r " / n f /". [This is the crux of "separability" for stochastic
processes. Consider the graph (t,f(t)) and introduce a metric]

* * indicates specially selected exercises (as mentioned in the Preface).

 1.2 DISTRIBUTION FUNCTIONS | 7

Ί.2 Distribution functions

Suppose now that / is bounded as well as increasing and not constant. We
have then
VJC : -oo < / ( - o o ) < / ( * ) </(+oo) < +00.
Consider the "normalized" function:

(1)

which is bounded and increasing with

(2) / ( - o o ) = 0, /(+oo)=l.
Owing to the simple nature of the linear transformation in (1) from / t o / a n d
vice versa, we may without loss of generality assume the normalizing con-
ditions (2) in dealing with a bounded increasing function. To avoid needless
complications, we shall also assume t h a t / i s right continuous as discussed in
Sec. 1.1. These conditions will now be formalized.

DEFINITION OF A DISTRIBUTION FUNCTION. A real-valued function F with

domain (-00, +00) that is increasing and right continuous with F(—co) = 0,
F(+oo) = 1 is called a distribution function, to be abbreviated hereafter as
"d.f." A d.f. that is a point mass as defined in (4) of Sec. 1.1 is said to be
"degenerate", otherwise "nondegenerate".
Of course all the properties given in Sec. 1.1 hold for a d.f. Indeed the
added assumption of boundedness does not appreciably simplify the proofs
there. In particular, let {a;} be the countable set of points of jump of Fand bj
the size at jump at aj9 then
F(a,) - F{aj-) = b,
since F(a} + ) = F{at). Consider the function
Fd{x) = 2 *A,(*)
j

which represents the sum of all the jumps of F in the half-line ( — 00, x\. It
is clearly increasing, right continuous, with

(3) Fd(-oo) = 0, F d (+co) = Σ b, < 1.

j

Hence Fd is a bounded increasing function. It should constitute the "jumping

part" of F, and if it is subtracted out from F, the remainder should be
positive, contain no more jumps, and so be continuous. These plausible
statements will now be proved—they are easy enough but not really trivial.
8 I DISTRIBUTION FUNCTION

Theorem 1.2.1. Let

Fc(x) = F(x)-Fd(x);
then Fc is positive, increasing, and continuous.
PROOF. Let x < x\ then we have
(4) Fd(xf)-Fd(x) = Σ b,= Σ [F(a3) - F(a,-)]
x<a.j<x' x<aj<x'
< F(x') - F(x).
It follows that both Fd and Fc are increasing, and if we put x = — oo in the
above, we see that Fd < F and so Fc is indeed positive. Next, Fd is right
continuous since each δα) is and the series defining Fd converges uniformly in
x; the same argument yields (cf. Example 2 of Sec. 1.1)

Fd(x) - F&-) = | 0 otherw.se

Now this evaluation holds also if Fd is replaced by F according to the defini-

tion of üj and bj9 hence we obtain for each x :
Fc(x) - Fc(x-) = F(x) - F(x~) - [Fd(x) - Fd(x-)] = 0.
This shows that Fc is left continuous ; since it is also right continuous, being
the difference of two such functions, it is continuous.

Theorem 1.2.2. Let F be a d.f. Suppose that there exist a continuous

function Gc and a function Gd of the form

[where {a]) is a countable set of real numbers and 2 WJ\ < oo], such that

then

where Fc and Fd are defined as before.

PROOF. If Fd Φ Gd, then either the sets {aj) and {a}} are not identical,
or we may relabel the a) so that a) = a5 for all j but b] Φ b0 for some j . In
either case we have for at least one j , and a = a3 or a) :

Since Fc — Gc = Gd — Fd, this implies that

contradicting the fact that Fc — Gc is a continuous function. Hence Fd = Gd

and consequently Fc = Gc.
 1.2 DISTRIBUTION FUNCTIONS | 9

DÉFINITION. A d.f. F that can be represented in the form

where {α}) is a countable set of real numbers, bj > 0 for every j and 2 è, = 1,
is called a discrete d.f. A d.f. that is continuous everywhere is called a
continuous d.f.

Suppose F c ^ 0, F d ^ 0 in Theorem 1.2.1, then we may set a = Fd(oo)

so that 0 < a < 1,

Fx = - Fd, F2 = î Fc,
a 1— a
and write
(5) F=aF1 + (1 - a)F2.
Now F x is a discrete d.f., F2 is a continuous d.f., and F is exhibited as a
convex combination of them. If F c == 0, then F is discrete and we set a = 1,
Fi = F, F2 = 0; if Fd = 0, then F is continuous and we set a = 0, Fx = 0,
F 2 Ξ F; in either extreme case (5) remains valid. We may now summarize
the two theorems above as follows.

Theorem 1.2.3. Every d.f. can be written as the convex combination of a

discrete and a continuous one. Such a decomposition is unique.

EXERCISES

1. Let F be a d.f. Then for each x,

lim [F(x + €) - F(x - e)] = 0

unless x is a point of jump of F, in which case the limit is equal to the size
of the jump.
*2. Let F be a d.f. with points of jump {a}}. Prove that the sum

Σ [Ffo) - F(aj-)]
X-€<dj <X

converges to zero as e | 0, for every x. What if the summation above is

extended to x — c < a3 < x instead ? Give another proof of the continuity
of Fc in Theorem 1.2.1 by using this problem.
3. A plausible verbal definition of a discrete d.f. may be given thus: "It
is a d.f. that has jumps and is constant between jumps." [Such a function is
sometimes called a "step function", though the meaning of this term does
not seem to be well established.] What is wrong with this? But suppose
that the set of points of jump is "discrete" in the Euclidean topology, then
the definition is valid (apart from our convention of right continuity).
10 I DISTRIBUTION FUNCTION

4. For a general increasing function F there is a similar decomposition

F = Fc + Fd9 where both F c and Fd are increasing, i^ is continuous, and Fd
is "purely jumping", [HINT: Let a be a point of continuity, put Fd(a) = F(a),
add jumps in (a9 oo) and subtract jumps in (—00, a) to define Fd. Cf. Exercise
2 in Sec. 1.1.]
5. Theorem 1.2.2 can be generalized to any bounded increasing function.
More generally, let/be the difference of two bounded increasing functions on
(—00, +00); such a function is said to be of bounded variation there. Define
its purely discontinuous and continuous parts and prove the corresponding
decomposition theorem.
*6. A point x is said to belong to the support of the d.f. F iff for every
e > 0 we have F(x + e) - F(x - e) > 0. The set of all such x is called the
support of F. Show that each point of jump belongs to the support, and that
each isolated point of the support is a point of jump. Give an example of a
discrete d.f. whose support is the whole line.
7. Prove that the support of any d.f. is a closed set, and the support of
any continuous d.f. is a perfect set.

Ί.3 Absolutely continuous and singular distributions

Further analysis of d.f.'s requires the theory of Lebesgue measure. Through-

out the book this measure will be denoted by m; "almost everywhere" on the
real line without qualification will refer to it and be abbreviated to "a.e."; an
integral written in the form j · · dt is a Lebesgue integral; a function / i s
said to be "integrable" in (0, b) iff

is defined and finite [this entails, of course, t h a t / b e Lebesgue measurable].

The class of such functions will be denoted by L\a, b), andLx(—00, 00) is ab-
breviated to ZÂ The complement of a subset S of an understood "space"
such as (—00, +00) will be denoted by Sc.

DEFINITION. A function F is called absolutely continuous [in (-00, 00)

and with respect to the Lebesgue measure] iff there exists a function/in L1
such that we have for every x < x' :

0) F(x')-F(x) = jXx'f(t)dt.
It follows from a well-known proposition (see, e.g., Natanson [3]*) that

* Numbers in brackets refer to the General Bibliography.

 1.3 ABSOLUTELY CONTINUOUS AND SINGULAR DISTRIBUTIONS | 11

such a function F has a derivative equal t o / a . e . In particular, if F i s a d.f.,

then

(2) f> Oa.e. and Γ f(t)dt = 1.

J-oo
Conversely, given a n y / i n L1 satisfying the conditions in (2), the function F
defined by

(3) Vx:F(*)= Γ f{t)dt

J-oo
is easily seen to be a d.f. that is absolutely continuous.

DEFINITION. A function F is called singular iff it is not identically zero

and F' (exists and) equals zero a.e.

The next theorem summarizes some basic facts of real function theory ;
see, e.g., Natanson [3].

Theorem 1.3.1. Let F be bounded increasing with F(-co) = 0, and let F'
denote its derivative wherever existing. Then the following assertions are true.

(a) If S denotes the set of all x for which F\x) exists with 0 < F\x) < oo,
then m(Sc) = 0.
(b) This F' belongs to L1, and we have for every x < x':

(4) [*' F\t)dt< F{x') - F(x).

Jx

(c) If we put

(5) Vx : Fac(x) = f* F\t) dt, Fs(x) = F(x) - Fac(x),

J — CO

then F'ac = F' a.e. so that F's = F' — F'ac = 0 a.e. and consequently Fs is
singular if it is not identically zero.

DEFINITION. Any positive function / that is equal to F' a.e. is called a

density of F. Fac is called the absolutely continuous part, Fs the singular part of
F. Note that the previous Fd is part of Fs as defined here.

It is clear that Fac is increasing and Fac < F. From (4) it follows that
if x < x'
Fix') - Fix) = F(x') - F(x) - j*f(t)dt > 0.
12 I DISTRIBUTION FUNCTION

Hence Fs is also increasing and Fs < F. We are now in a position to announce

the following result, which is a refinement of Theorem 1.2.3.

Theorem 1.3.2. Every d.f. F can be written as the convex combination of a

discrete, a singular continuous, and an absolutely continuous d.f. Such a
decomposition is unique.

EXERCISES

1. A d.f. Fis singular if and only if F = Fs;it is absolutely continuous if

and only if F = Fac.
2. Prove Theorem 1.3.2.
* 3 . If the support of a d.f. (see Exercise 6 of Sec. 1.2) is of measure zero,
then F is singular. The converse is false.
*4. Suppose that F is a d.f. and (3) holds with a continuous / . Then
F' = f > 0 everywhere.
5. Under the conditions in the preceding exercise, the support of F is
the closure of the set {t \f(t) > 0}; the complement of the support is the
interior of the set {t \ f(t) = 0}.
6. Prove that a discrete distribution is singular. [Cf. Exercise 13 of
Sec. 2.2.]
7. Prove that a singular function as defined here is (Lebesgue) measur-
able but need not be of bounded variation even locally, [HINT: Such a
function is continuous except on a set of Lebesgue measure zero ; use the
completeness of the Lebesgue measure.]

The remainder of this section is devoted to the construction of a singular

continuous distribution. For this purpose let us recall the construction of
the Cantor (ternary) set (see, e.g., Natanson [3]). From the closed interval
[0, 1], the "middle third" open interval (^, f) is removed; from each of the
two remaining disjoint closed intervals the middle third, (£, f) and Q, f),
respectively, are removed and so on. After n steps, we have removed
1 + 2 + · · · + 2 71 - 1 = 2n - 1
disjoint open intervals and are left with 2 n disjoint closed intervals each of
length l/3 n . Let these removed ones, in order of position from left to right,
be denoted by JUtk, 1 < k < 2n — 1, and their union by Un. We have
m N 1 2 4 2n~1 Λ (2\n

As n f oo, Un increases to an open set U; the complement C of U with

respect to [0, 1] is a perfect set, called the Cantor set. It is of measure zero
 1.3 ABSOLUTELY CONTINUOUS AND SINGULAR DISTRIBUTIONS | 13

since
m(C) = 1 - m(U) = 1 - 1 = 0 .
Now for each n and k, n > 1, 1 < k < 2n - 1, we put

Cn,k r\n ?

and define a function F on U as follows :

(7) F(x) = cntk forxeJntk.
This definition is consistent since two intervals, Jnk and Jn'tk>, are either
disjoint or identical, and in the latter case so are cUfk = cn>x. The last
assertion becomes obvious if we proceed step by step and observe that
=
Jn + l,2k — Jn,k> c
n+l,2k c
n,k * Ο Γ 1 < k < 2n — 1 .

The value of F is constant on each Jntk and is strictly greater on any other
Jn>tk* situated to the right ofJnk. Thus Fis increasing and clearly we have
l i m i t e ) = 0, l i m i t e ) = 1.

Let us complete the definition of F by setting

FO) = 0 for x < 0, F(x) = 1 for x > 1.
F is now defined on the domain D = (— oo, 0) u U u (1, oo) and increasing
there. Since each Jn>k is at a distance > l/3 n from any distinct Jn.%w and the
total variation of F over each of the 2 n disjoint intervals that remain after
removing Jntk9 1 < k < 2" - 1, is 1/2", it follows that

0<x'-x<yn=>0< F(x') - F(x) < 1 .

Hence F is uniformly continuous on D. By Exercise 5 of Sec. 1.1, there

exists a continuous increasing F on (—oo, +oo) that coincides with F on D.
This F is a continuous d.f. that is constant on each Jnk. It follows that
P' = 0 on U and so also on (—oo, +oo) — C. Thus F is singular. Alterna-
tively, it is clear that none of the points in D is in the support of F, hence the
latter is contained in C and of measure 0, so that P is singular by Exercise 3
above. [In Exercise 13 of Sec. 2.2, it will become obvious that the measure
corresponding to F is singular because there is no mass in U.]

EXERCISES

The F in these exercises is the P defined above.

8. Prove that the support of F is exactly C.
14 I DISTRIBUTION FUNCTION

*9. It is well known that any point x in C has a ternary expansion without
the digit 1 :

Prove that for this x we have

10. For each x e [0, 1], we have

11. Calculate

[HINT: This can be done directly or by using Exercise 10; for a third method
see Exercise 9 of Sec. 5.3.]
12. Extend the function F on [0, 1] trivially to (-00,00). Let {rn}
be an enumeration of the rationals and

Show that G is a d.f. that is strictly increasing for all x and singular. Thus we
have a singular d.f. with support (—00, 00).
*13. Consider F on [0, 1]. Modify its inverse F'1 suitably to make it
single-valued in [0, 1]. Show that F'1 so modified is a discrete d.f. and find
its points of jump and their sizes.
14. Given any closed set C in (—00, +00), there exists a d.f. whose
support is exactly C. [HINT: Such a problem becomes easier when the
corresponding measure is considered; see Sec. 2.2 below.]
*15. The Cantor d.f. F is a good building block of "pathological"
examples. For example, let H be the inverse of the homeomorphic map of
[0, 1] onto itself: x->?[F(x) + x]; and E a subset of [0, 1] which is not
Lebesgue measurable. Show that

where H(E) is the image of E, \B is the indicator function of B, and o denotes

the composition of functions. Hence deduce: (1) a Lebesgue measurable
function of a strictly increasing and continuous function need not be Lebesgue
measurable; (2) there exists a Lebesgue measurable function that is not Borel
measurable.