Francesc Dilmé

Reduced Notes
Mathematics for Economists, University of Bonn

1 Sets, Functions, and Maxima

1.1 Sets Thm.1.1 Let {Ai |i ∈ I } be a collection. Then:
1. B∪(∩i∈I Ai ) = ∩i∈I (B∪Ai ) for all B ⊂ X.
We fix a collection of elements, or universe,
2. B∩(∪i∈I Ai ) = ∪i∈I (B∩Ai ) for all B ⊂ X.
X. We use x ∈ X to indicate that x is an ele-
ment of X. 3. (∪i∈I Ai )c = ∩i∈I Aci .
4. (∩i∈I Ai )c = ∪i∈I Aci .
A set A (in this universe) is a collection of
its elements (we do not properly define a Def. The Cartesian product of A ⊂ X and
set). We use A ⊂ X to indicate that A is a B ⊂ X is A × B ≡ {(x, x0 )|x ∈ A and x0 ∈ B}.
set in X. The power set is the “set of sets of
X”, and it is denoted 2X . In particular, 2X X n is the set of n-tuples (x1 , ..., xn ), where
contains X and the empty set ∅ as elements. xi ∈ X for all i. Many times we abuse no-
Note that A ⊂ X is equivalent to A ∈ 2X . tation and identify X 3 with, for example,
Note that, for each x ∈ X and A ⊂ X, either (X ×X)×X or X ×(X ×X). Note that R1 = R,
x ∈ A or x < A, but never both. Rn = Rn−1 × R for n > 1.
Def. B ⊂ X is a subset of A ⊂ X if, for all
x ∈ B, we have x ∈ A. Real numbers

B ⊂ A indicates that B is a subset of A. N = {1, 2, 3, ...}, Z = {..., −2, −1, 0, 1, 2, ...}.

R+ = [0,+∞), R++ = (0,+∞), R = R∪{−∞,+∞}.
Def. The complement of A ⊂ X is Ac ≡ {x ∈
X|x < A} ⊂ X. Thm (Archimedean Property).1.2 For any
x ∈ R++ there exists n ∈ N such that 1/n < x < n.
A collection of sets indexed by some set I ,
{Ai |i ∈ I }, assigns a set to Ai ⊂ X each ele-
ment i ∈ I . 1.2 Relations
Def. The union and intersection of {Ai |i ∈ I } Let X and Y be two universes.
are, respectively,
∪i∈I Ai ≡ {x|∃i ∈ I s.t. x ∈ Ai } and Def. A relation R on two sets X and Y is a
∩i∈I Ai ≡ {x|x ∈ Ai ∀i ∈ I } . subset of X × Y . Usually, xRy denotes that
(x, y) ∈ R (“x is related to y”). Also ¬(xRy)
Note that B ⊂ ∩i∈I Ai if and only if B ⊂ Ai or ¬xRy denotes that (x, y) < R.
for all i ∈ I .
Def. The image of R is {y ∈ Y |∃x ∈
Def. {Ai |i ∈ I } is a partition of X if Ai ∩Ai 0 = X s.t. xRy} ⊂ Y . The domain of R is {x ∈
∅ for all i , i 0 and ∪i∈I Ai = X. X|∃y ∈ Y s.t. xRy} ⊂ X.
Given a relation R, its associated correspon- We use f : X → Y to indicate that f is a
dence maps each x ∈ X to r(x) ≡ {y|xRy}. function from X to Y . A function can also
Note that xRy ⇔ y ∈ r(x). be denoted “x 7→ f (x) for all x ∈ X”. The
• For A ⊂ X, r(A) ≡ {y|∃x ∈ A s.t. xRy} ⊂ Y . set of functions from X to Y is denoted Y X .
• For B ⊂ Y , r −1 (B) ≡ {x|∃y ∈ B s.t. xRy} ⊂ X. Note that f : X → Y is equivalent to f ∈ Y X .
For each f : X → Y , X and Y are called,
Relations on X respectively, the domain and the range of f .

Def. A relation R on X is a subset of X × X. Def. Let f : X → Y . The image of A ⊂ X un-

der f is f (A) ≡ {y ∈ Y | ∃x ∈ A s.t. f (x) = y} ⊂
Def. A relation R on X is said to be
Y , which is also denoted {f (x)|x ∈ A}. The
• reflexive if xRx for all x ∈ X. image of f is f (X). The anti-image of B ⊂ Y
• complete if xRy or yRx for all x, y ∈ X. is f −1 (B) ≡ {x|f (x) ∈ B} ⊂ X.
• transitive if xRy and yRz implies xRz, for
all x, y, z ∈ X. Def. The graph of f : X → Y is {(x,y)|f (x) =
• symmetric if xRy ⇒ yRx for all x, y ∈ X. y} ⊂ X ×Y .
• antisymmetric if xRy and yRx implies x =
y for all x, y ∈ X.
• asymmetric if xRy implies ¬yRx for all Types of functions and operations
x, y ∈ X.
Def. f : X → Y is one-to-one if for all x, x0 ∈
Def. A preorder  is a reflexive and transi- X if x , x0 then f (x) , f (x0 ) (it is also called
tive relation. injective or an injection).

Def. The asymmetric part  of a preorder 

Def. f : X → Y is onto if f (X) = Y (it is also
is a relation on X such that, for all x,y ∈ X,
called surjective).
xy if and only if xy and ¬yx. The sym-
metric part ∼ of a preorder  is a relation
Def. f : X → Y is bijective if it is one-to-one
on X such that, for all x,y ∈ X, x∼y if and
and onto (it is also called a bijection).
only if xy and yx.

For a given preorder , x ∈ X is maximal When f is bijective there exists a unique

if x0  x implies x0 ∼ x for all x0 ∈ X, and is function f −1 : Y → X, called the inverse of
x ∈ X is the largest if x  x0 for all x0 ∈ X. Min- f , that satisfies f −1 (f (x)) = x for all x ∈ X.
imal and lowest are defined analogously,
Thm.1.3 If there are an injection from X to Y
Def. A partial order (or order) is an anti-
and an injection from Y to X, then there is a
symmetric preorder. A total (or complete)
bijection from X and Y .
order is a complete partial order.

Def. The composition of f : X → Y and

1.3 Functions g : Y → Z is (g ◦ f ) : X → Z defined by
(g ◦ f )(x) ≡ g(f (x)) ∀x ∈ X.
Fix some universes X, Y and Z.

Def. A function f from X to Y is a map that Def. The restriction of f : X → Y to A ⊂ X is

assigns to each x ∈ X an element f (x) ∈ Y . f |A : A → Y defined by f |A (x) ≡ f (x) ∀x ∈ A.
Functions to R occurs if, for example, A is not bounded
above or if A is of the form [a, b).
Many operations in R can be defined in RX .
Minimum is defined analogously.
For example, for all f , g ∈ RX we define
• (−f )(x) = −f (x) for all x ∈ X. Thm.1.5 Let A ⊂ N be non-empty. Then, A has
• (αf +g+β)(x) = αf (x)+g(x)+β for all x ∈ X, a minimum.
α, β ∈ R.
Def. x ∈ R is a supremum of A ⊂ R if x is
• (f g)(x) = f (x) g(x) for all x ∈ X.
an upper bound of A and there is no upper
Def. f : R → R is increasing if f (x) ≥ f (x0 ) bound of A lower than x.
whenever x ≥ x0 . f : R → R is strictly in-
Thm.1.6 Let A ⊂ R be non-empty and bounded
creasing if f (x) > f (x0 ) whenever x > x0 .
above. Then, x = sup(A) if and only if x is an
Def. The indicator function associated to upper bound of A and, for all ε > 0, there is
A ⊂ X is IA : X → {0, 1} so that IA (x) = 1 x0 ∈ A such that x0 > x−ε.
if and only if x ∈ A.
Infimum is defined analogously. If A has a
Def. Fix some f : X → R. Then: supremum, then it is unique, and we use
1. Its epigraph is epi f ≡ {(x, y)|y ≥ f (x)}. sup(A) ∈ R to denote it. We use the con-
2. Its hypograph is hyp f ≡ {(x, y)|y ≤ f (x)}. vention that sup(∅) = −∞ and sup(A) = +∞
when A is not bounded above. Under this
Note that for any f , g : X → Y , we have convention, we have:
that the epigraph of max{f , g} is the inter-
section of the epigraphs of f and g. Thm.1.7 Every set in R has a supremum in R.

Thm.1.4 Let f , g : X → R. Then: If a set is non-empty and bounded above

1. epi max{f , g} = epi f ∩ epi g. then it has a finite supremum.
2. epi min{f , g} = epi f ∪ epi g.
Maxima and suprema of functions
1.4 Maxima and Suprema We can apply the concepts of maximum
and supremum to functions by applying
Def. x ∈ R is an upper bound of A ⊂ R if x ≥ x0
them to their images.
for all x0 ∈ A.
Def. f : X → R is bounded (above/below) if
The definition of lower bound is analogous.
f (X) is bounded (above/below).
Def. A ⊂ R is bounded above if it has an up-
Def. The supremum of f : X → R on A ⊂ X
per bound, bounded below if it has a lower
is supx∈A f (x) ≡ sup(f (A)).
bound, and bounded if it is bounded above
and below. In the previous definition, if f (A) has a
Def. x ∈ R is a maximum of A ⊂ R if x ∈ A maximum, then such maximum is denoted
and x is an upper bound of A. maxx∈A f (x) (or simply max(f (A))).

Def. The argument of the maximum of f :

If A has a maximum, then it is unique, and
X → R on A ⊂ X is
we use max(A) ∈ R to denote it. It may
be that A does not have maximum. This arg max f (x) ≡ f −1 ({sup(f (A)}))∩A ⊂ X .
x∈A
The argument of the maximum is non- Note that we can identify A{1,...,N } with AN
empty if and only if max(f (A)) exists. for any N ∈ N (note that f 7→ (f (1), ..., f (N ))
is bijective). Similarly, {0, 1}X can be identi-
Thm.1.8 Let f , g : X → R be bounded, and let
fied with 2X (note that f 7→ {x ∈ X|f (x) = 1}
f ≡ supx∈X f (x) and g ≡ supx∈X g(x). Then:
is bijective).
1. supx∈X (f (x)+g(x)) ≤ f +g.
2. If f (x) < g(x) for all x ∈ X, then f ≤ g. Def. A ⊂ X is countable if there is a bijection
3. supx∈X (a f (x)+b) = a f +b for all a ∈ R+ and between A and N, in which case we write
b∈R . |A| = ℵ0 . A is discrete if it is either finite or
countable.
1.5 Cardinality
Note that Z and Q are countable.
Def. A ⊂ X is finite if either is equal to ∅ or
there is a bijection between A and {1, ..., N } Thm.1.10 R is not discrete.
for some N ∈ N. In this case, the cardinality
of A, denoted |A|, is 0 and N , respectively. Def. A ⊂ X has the cardinatlity of the conti-
nuum if there is a bijection between A and
Note that N is not finite. R. It is denoted |A| = ℵ1 or |A| = c.
Thm.1.9 Assume that A and B are finite. Then,
|2A | = 2|A| , |A ∪ B| = |A| + |B| − |A ∩ B|, |A × B| = Rn , for any n ∈ N, and 2N have the cardi-
|A| |B| and |BA | = |B||A| . nality of the continuum.

2 Distances, Vectors, and Sequences

In this section, we fix some universe X. • A ⊂ X is closed if Ac is open.

By definition, ∅ and X are both open and

2.1 Metric Spaces closed sets. There may be sets which are
Def. A function d : X 2 → R is a distance on neither open nor closed, such as [a, b) ⊂ R.
X if, for all x, x0 , x00 ∈ X, Thm.2.1 Let {Ai |i ∈ I } and {Bn |n = 1, ..., N } be
1. d(x, x0 ) ≥ 0. collections of open sets, for some I . Then
2. d(x, x0 ) = 0 if and only if x = x0 . 1. ∪i∈I Ai is open and ∩i∈I Aci is closed.
3. d(x, x0 ) = d(x0 , x) . 2. ∩N N c
n=1 Bn is open and ∪n=1 Bn is closed.
4. d(x, x0 ) + d(x0 , x00 ) ≥ d(x, x00 ).
Def. 1. A ⊂ X is bounded if there is some
A pair hX, di, where d is a distance on X, M > 0 and x ∈ X such that A ⊂ BM (x).
form a metric space. The rest of this section 2. A ⊂ X is compact if is closed and
takes some metric space hX, di as given. bounded.

Def. The ball of radius ε > 0 centered at Def. Fix A ⊂ X. Then its
x ∈ X is Bε (x) ≡ {x0 |d(x0 , x) < ε}. 1. interior is int(A) ≡ {x ∈ A|∃ε > 0 Bε (x) ⊂ A}.
Def. • A ⊂ X is open if for all x ∈ A there is 2. closure is Ā ≡ { x ∈ X|∀ε > 0 Bε (x)∩A , ∅ }.
some ε > 0 such that Bε (x) ⊂ A. 3. boundary is ∂A = Ā\int(A).
Thm.2.2 1. The closure of A ⊂ X is the inter- Euclidean space
section of all closed sets containing A.
2. The interior A ⊂ X is the union of all open Def. The n-dimensional Euclidean space is
sets contained in A. the set Rn with, for all x, x0 ∈ Rn :
1. The inner (dot/scalar) product between x
Def. A ⊂ X is dense in X if for all x ∈ X and and x0 is x · y ≡ x1 x01 + ... + xn x0n ∈ R.
ε > 0 we have A ∩ Bε (x) , ∅. 2. The scalar multiplication between λ ∈ R
and x is λ x ≡ (λ x1 , ...., λ xn ) ∈ Rn .
2.2 Vector Spaces 3. The Euclidean (2-)norm ||x|| ≡ (x · x)1/2 .

Def. A real vector space is a set V with two Thm (Cauchy-Schwarz inequality).2.6
operations + : V ×V → V and · : R×V → V |u · v| ≤ ||u|| ||v|| ∀u,v ∈ Rn .
such that, for all u, v, w ∈ V and a, b ∈ R,1
1. (V , +) is a commutative group with, Convexity of sets
(a) Identity: there is a zero vector 0 ∈ V
Def. x00 ∈ V is a convex combination be-
such that 0+v = v +0 = v,
tween x, x0 ∈ V if there is some λ ∈ [0, 1]
(b) Additive inversion: the additive in-
such that x00 = λ x+(1−λ) x0 .
verse of v is −v, so v−v ≡ v+(−v) = 0,
(c) Commutativity: u +v = v +u, Def. A ⊂ V is convex if λ x+(1−λ) x0 ∈ A for
(d) Associativity: u +(v +w) = (u +v)+w. all x, x0 ∈ A, and λ ∈ (0, 1).
2. Associativity of the scalars: a (bv) = (ab) v.
Def. A ⊂ Rn is strictly convex if λ x+(1−λ) x0 ∈
3. Identity of scalar multiplication: 1 v = v.
int(A) for all x, x0 ∈ A, x , x0, and λ ∈ (0, 1).
4. Distributivity of scalar multiplication
with respect to addition: a (u+v) = a u+a v. Thm.2.7 Let {Ai |i ∈ I } be a collection of convex
5. Distributivity of multiplication with re- sets. Then, ∩i∈I Ai is a convex set.
spect to scalar addition: (a+b) v = a v +b v.
Def. The convex hull of A ⊂ V is the inter-
Def. || · || : V → R is a norm if: section of all convex sets containing A.
1. ||v|| ≥ 0 for all v ∈ V .
2. ||v|| = 0 ⇔ v = 0 for all v ∈ V . The convex hull of A contains the set of all
3. ||u + v|| ≤ ||u|| + ||v|| for all u, v ∈ V . convex combinations of elements in A, but
4. ||α v|| = |α| ||v|| for all v ∈ V and α ∈ R. can contain other points.

Thm.2.3 (u, v) 7→ ||u − v|| for all u, v ∈ V is

a distance on V , and it is called the distance 2.3 Sequences
generated by || · ||. In this section we fix a metric space hX, di.
Def. For each p ∈ [1, +∞], the p-norm in
Def. A sequence (xn )n in X is an element of
Rn is defined ||(x1 , ..., xn )||p ≡ ( ni=1 |xi |p )1/p
P
X N ; i.e., a function from N to X. It assigns
(with ||(x1 , ..., xn )||∞ ≡ max{|xi | |i =, ..., n}).
to each n ∈ N the value xn ∈ X.
Thm.2.4 || · ||p a norm on Rn for all p ∈ [1,+∞].
Def. A sequence (xn )n converges to x ∈ X if
Thm.2.5 All norms in Rn generate the same for all ε > 0 there is some N ∈ N such that
open sets. d(xn , x) < ε for all n ≥ N .
1 “+” is called the (vector) addition and “·” the scalar multiplication.
Thm.2.8 A sequence (xn )n converges to x if and Cauchy sequences
only if (d(x, xn ))n converges to 0.
Def. A sequence (xn )n is called a Cauchy
Thm.2.9 If a sequence (xn )n converges to x then sequence if for every ε > 0, ∃Nε such that
it does not converge to any x0 , x. d(xn , xm ) < ε for all n, m > Nε .

We use xn → x and limn→∞ xn = x to denote All convergent sequences are Cauchy se-
that the sequence (xn )n converges to x, and quences. A metric space is complete if all
we call x the limit of (xn )n . A sequence is Cauchy sequences are convergent. A Ba-
convergent if converges to some limit, and nach space is a complete normed vector
divergent otherwise. space.
Thm.2.14 Rn is complete for all n ∈ N.
Def. (xn0 )n is a subsequence of (xn )n if there
exists a strictly increasing sequence (kn )n Instead, Q is not complete: not all Cauchy
in N such that xn0 = xkn for all n ∈ N. sequences in Q are convergent in Q.

If a sequence is convergent, all its subse-

Sequences in R
quences are converge to the same limit.
Def. A sequence (xn )n ∈ RN tends to +∞
Cluster points of sequences (resp −∞) if for all M > 0 there is some
N ∈ N such that xn > M (resp. xn < −M) for
Def. x ∈ X is a cluster (or accumulation) all n ≥ N .
point of (xn )n if for all ε > 0 and N ∈ N there
is some N 0 > N such that d(xN 0 , x) < ε. If (xn )n ∈ RN tends to +∞ then we write
xn → ∞ and limn→∞ xn = +∞.
Thm.2.10 x is a cluster point of (xn )n if and Thm.2.15 Any sequence in RN contains a
only if x is the limit of a subsequence of (xn )n . monotone subsequence. Also, a monotone se-
quence tends to some point in R.
The following (very) important result es-
tablishes the existence of cluster points in Def. The superior limit of (xn )n ∈ RN is
bounded sequences in Rn . lim sup({xm |m ≥ n}) ∈ R .
n→∞
Thm (Bolzano-Weierstrass). 2.11 Any Thm.2.16 The superior limit of (xn )n ∈ RN al-
bounded sequence in Rn has one cluster point. ways exists (in R). If (xn )n ∈ RN is bounded
above, then its superior limit is finite.
Limit points of sets Thm.2.17 The superior and inferior limits of
(xn )n ∈ RN coincide if and only if limn→∞ xn
Def. x ∈ X is a limit point of A ⊂ X if it is
exists (in R).
the limit of a sequence in A\{x}.
Def. The sequence of partial sums of (xn )n is
Thm.2.12 A ⊂ Rn is closed if and only if con- (Sn )n , with Sn ≡ ni=1 xi .
P
tains all its limit points.
If (Sn )n is convergent, the series associated
to (xn )n is ∞
P
Def. x ∈ A is an isolated point if Bε (x)∩A = n=1 xn ≡ limn→∞ Sn . An impor-
{x} for some ε > 0. tant example of convergent series is
n
Thm.2.13 x ∈ A if and only if x is a limit or iso-
X
i 1−λn+1 1
lim λ = lim = ,
n→∞ n→∞ 1−λ 1−λ
lated point of A (it is never both). Also, x ∈ A i=0
if and only if a sequence in A converges to x. where λ ∈ (−1, 1).
Def. ∞ 2. (αxn+xn0 )n converges to αx+x0 for any α ∈ R.
P
n=1 xn is absolutely convergent if the
sequence
P of partial sums of absolute values 3. (xn xn0 )n converges to x x0 .
n
i=1 |xi | n converges. 4. If xn0 , 0 for all n and y , 0, then (xn /xn0 )n
converges to x/x0 .
If ∞
P
n=1 xn is absolutely convergent then it 0
5. If (xn )n is absolutely convergent  and (xn )n
is also convergent. P
n 0
is bounded, then i=1 xn xn n is abso-
Thm.2.18 Let (xn )n , (xn0 )n ∈ RN converging to lutely convergent.
x and x0 , respectively. Then: 6. If (xn00 )n is such that xn ≥ xn00 ≥ xn0 for all n
1. If xn ≥ xn0 for all n, then x ≥ x0 . and x = x0 , then xn00 → x.

3 Continuity, Differentiability, and Convexity

In this section we fix two metric spaces Real functions
hX, di and hY , d 0 i.
Def. f : X → R is upper semi-continuous at
x0 if ∀ε > 0 ∃δ > 0 such that f (x) < f (x0 ) + ε
3.1 Continuity for all x ∈ Bδ (x0 ).
Def. f : X → Y is continuous at x0 ∈ X if Equivalently, f : X → R is upper semi-con-
for any sequence (xn )n converging to x0 we tinuous at x0 if lim supn→∞ f (xn ) ≤ f (x0 ) for
have that (f (xn ))n converges to f (x0 ). any (xn )n converging to x0 .
Lower semi-continuity is analogous (with
There is an equivalent definition of conti- lim inf and ≥). Note that if f is upper
nuity using balls: semi-continuous, then −f is lower semi-
Thm.3.1 f : X → Y is continuous at x0 if and continuous.
only if for all ε > 0 there is some δ > 0 such Thm.3.3 f : X → R is continuous if and only
that f (Bδ (x0 )) ⊂ Bε (f (x0 )). if it is both upper and lower semi-continuous.
Thm.3.4 For all f : X → R, it is equivalent
A function is continuous if it is continuous
on all its domain. 1. f is upper semi-continuous,
2. the hypograph of f is closed, and
Thm.3.2 f : X → Y is continuous if and only 3. {x|f (x) ≥ α} ⊂ X is closed for every α ∈ R.
if the inverse of any open set is an open set.
Thm.3.5 Let {fi : X → R}i∈I be a set of upper
We use C 0 (X, Y ) to denote the set of contin- semi-continuous functions. Assume that, for
uous functions from X to Y . all x ∈ X, f (x) ≡ infi∈I fi (x) > −∞. Then f is
upper semi-continuous.
Def. y0 ∈ Y is the limit of f : X → Y at x0 ∈ X
if (f (xn ))n converges to y0 for all sequences
3.2 Functions from R to R
(xn )n converging to x0 with x , x0 for all n.
Left and right limits
If the limit of f at x0 exists, then it is
unique, and we denote it by limx→x0 f (x). Def. f : R → R has a left limit at x0 if
Note that f is continuous at x0 if and only limn→∞ f (xn ) exists for all strictly increas-
if limx→x0 f (x) = f (x0 ). ing sequences (xn )n converging to x0 .
If the left limit of f at x0 exists, it is unique, Results
and it is denoted f (x0− ) or limx%x0 f (x). The
right limit is defined analogously. Thm (inverse function).3.7 Let A ⊂ R be
open and f ∈ C 1 (A, R). Assume f 0 (x0 ) , 0 for
Def. f : R → R is left-continuous at x0 ∈ R
some x0 ∈ A. Then, ε > 0 exists such that
if limx%x0 f (x) = f (x0 ).
1. f : Bε (x0 ) → f (Bε (x0 )) is bijective;2
Right-continuity is defined analogously. If 2. using f −1 : f (Bε (x0 )) → Bε (x0 ) to denote
both the left and right limits of f at x0 ex- the inverse of f : Bε (x0 ) → f (Bε (x0 )), we
ist and coincide, then limx→x0 f (x) denotes have that f −1 ∈ C 1 (f (Bε (x0 )), Bε (x0 )) and
their value. its derivative at f (x0 ) is given by
1
(f −1 )0 (f (x0 )) = 0 .
Differentiability f (x0 )

Def. f : R → R is differentiable at x0 ∈ R if Thm (L’Hopital rule). 3.8 Let f , g ∈

limx→x0 ((f (x)−f (x0 ))/(x−x0 )) exists. C 1 ((b, a), R), where a ∈ R ∪ {+∞} with
When f is differentiable at x0 ∈ R, then 1. limx%a |f (x)| = limx%a |g(x)| ∈ {0, +∞},
limx→x0 ((f (x) − f (x0 ))/(x − x0 )) is denoted f 0 (x)
2. limx%a g 0 (x) exists (in R) and
d
f 0 (x0 ), f (1) (x0 ), or dx f (x)x=x . 3. g 0 (x) , 0 for all x ∈ (b, a).
0

Def. f : R → R is differentiable if it is differ- f (x) f 0 (x)

Then limx%a g(x) = limx%a g 0 (x) .
entiable at x for all x ∈ R.
If f is differentiable, the function f 0 is Thm (Taylor).3.9 Let A ⊂ R, f ∈ C k (A, R) and
called its derivative. We use C 1 (R, R) to de- x0 ∈ int(A).  Then, there exists Rk (x) with
R (x)
note the set of functions from R to R which lim supx→x0  k k  = 0 such that3
(x−x0 )
are differentiable and have a continuous f 00 (x0 ) 2
f (x) =f (x0 )+f 0 (x0 ) (x−x0 )+ 2! (x−x0 )
derivative.
f (k) (x0 )
Thm.3.6 Let f , g ∈ C 1 (R, R). Then +· · ·+ k! (x − x0 )k + Rk (x) .
d(α f (x)+g(x))
1. dx = α f 0 (x) + g 0 (x) for all α ∈ R.  R (x) 
d(f (x) g(x)) If f ∈ C k+1 (A, R) then lim sup (x−xk )k+1  < ∞.
2. dx = f 0 (x) g(x) + f (x) g 0 (x). x→x0 0
d(f (x)/g(x)) f 0 (x) g(x)−f (x) g 0 (x)
3. dx = g(x)2
if g(x) , 0.
dg(f (x))
4. dx = g 0 (f (x)) f 0 (x). 3.3 Functions from Rn to R
For each m ∈ N, we define C m (R, R) recur- In this section we use x = (x1 , ..., xn ) ∈ Rn .
sively as the set of functions from R to
R which are differentiable and such that Def. A path in Rn is a map from R to Rn .
f 0 ∈ C m−1 (R, R). The m-th derivative of a
function in C m (R, R) is denoted f (m) . Note A path g : R → Rn is usually denoted g(t) =
that C m (R, R) ⊂ C m−1 (R, R) for all m. (g 1 (t), ..., g n (t)), where g i : R → R for all
C ∞ (R, R) denotes the set of functions which i = 1, ..., n. C 1 (R, Rn ) is the set of paths g
belong to C m (R, R) for all m ∈ N. such that g i ∈ C 1 (R, R) for all i = 1, ..., n.
2 With some abuse of notation, f : Bε (x0 ) → f (Bε (x0 )) denotes the function from Bε (x0 ) to f (Bε (x0 )) which
assigns f (x) ∈ f (Bε (x0 )) to each x ∈ Bε (x0 ).
3 Recall that if n ∈ N then n! ≡ 1 × 2 × 3 × ... × n, and 0! = 1.
Def. The derivative of f : Rn → R along the Thm.3.12 f : V → R is (strictly) convex if and
path g : R → Rn at t0 is (f ◦g)0 (t0 ), if it exists. only if its epigraph is (strictly) convex.

The partial derivative of f with respect to Thm.3.13 Let f : (a, b) → R be convex, for
∂f (x )
xi at x0 ∈ Rn , denoted ∂xi0 or fi (x0 ), is the some a, b ∈ R. Then f is continuous.
derivative along t 7→ (x01, ..., x0i−1, t, x0i+1, ..., x0n )
at t = x0i (if it exists). Thm.3.14 Let f ∈ C 1 (Rn , R). Then, f is con-
Def. The (first) derivative (or Jacobian) of cave if and only if
f : Rn → R at x0 ∈ Rn is f (x2 )−f (x1 ) ≤ Df (x1 ) · (x2 −x1 ) ∀x1 , x2 ∈ Rn .
Df (x0 ) ≡ (f1 (x0 ), ..., fn (x0 )) .
∂f (x0 ) Thm.3.15 Let f ∈ C 2 (Rn , R). Then f is concave
(Only defined when fi (x0 ) ≡ exists
∂xi if and only if Hf (x) is negative semidefinite
for all i = 1, ..., n.)
for all x ∈ Rn . If Hf (x) is negative definite
C 1 (Rn , R) is the set of functions f : Rn , R for all x ∈ Rn then f is strictly concave.
such that x 7→ Df (x) exists and is contin-
uous. Def. f : V → R is (strictly) quasiconvex
if for all x, x0 ∈ V and λ ∈ [0, 1] we have
Thm (chain rule 2).3.10 Let f ∈ C 1 (Rn , R) and f (λ x+(1−λ) x0 ) ≤ (<) max{f (x), f (x0 )}.
g ∈ C 1 (R, Rn ). Then, for all t ∈ R,
 
d d 1 d n
dt f (g(t)) = Df (g(t)) · dt g (t), ..., dt g (t) . Note that if f is quasiconvex then it is con-
vex. A quasiconcave function is defined
Thm (implicit function).3.11 Let A ⊂ R2 be
analogously.
open and f ∈ C 1 (A, R). Assume (x01 , x02 ) ∈ A is
such that f2 (x01 , x02 ) , 0. Then,
Thm.3.16 f : V → R is quasiconcave if and
1. there exists ε > 0 and x̃2 : (x01 − ε, x01 + ε) →
only if {x|f (x) ≥ α} ⊂ V is convex for all α ∈ R.
R such that f (x1 , x̃2 (x1 )) = f (x01 , x02 ) for all
x1 ∈ (x01 − ε, x01 + ε), and
2. x̃2 (·) is differentiable and its derivative is a
3.4 Correspondences
continuous function given by
d 2 1 f1 (x1 , x̃2 (x1 )) Fix some universes X and Y .
1 x̃ (x ) = − .
dx f2 (x1 , x̃2 (x1 ))
Def. A correspondence Γ from X to Y is a
Convexity and concavity of functions map assigning, to each x ∈ X, a set Γ (x) ⊂ Y .

Let V be a vector space (e.g., Rn ).

We use Γ : X ⇒ Y to denote that Γ is a
Def. f : V → R is concave if for all ∈V x, x0 correspondence from X to Y . Note that
0
and λ ∈ [0, 1] we have f (λ x + (1 − λ) x ) ≥ Γ : X ⇒ Y is equivalent to Γ : X → 2Y .
λf (x)+(1−λ)f (x0 ).
Def. The graph of Γ : X ⇒ Y is the set
Def. f : V → R is strictly concave if for all
{(x, y)|y ∈ Γ (x)} ⊂ X × Y .
x, x0 ∈ V such that x , x0 and λ ∈ (0, 1) we
have f (λ x+(1−λ) x0 ) > λf (x)+(1−λ)f (x0 ).
Def. Γ : X ⇒ Y is open-valued (closed,
Def. f : V → R is (strictly) convex if −f is bounded, convex...) if, for all x ∈ X, Γ (x) is
(strictly) concave. open (closed, bounded, convex...).
Continuity for correspondences Def. Γ : R ⇒ R is increasing if Γ (x) ⊂ Γ (x0 )
whenever x ≤ x0 .
Def. Γ : X ⇒ Y is upper hemicontinuous
(UHC) at x0 ∈ X if for every open A ⊃ Γ (x0 ) Def. Γ : X ⇒ X is convex if the graph of Γ
there is an open B 3 x0 such that Γ (B) ⊂ A. is convex (on X ×X).
Def. Γ : X ⇒ Y is lower hemicontinuous
(LHC) at x0 ∈ X if for every open A ⊂ Y Thm.3.17 1. Let Γ : X ⇒ Y be compact-valued.
with Γ (x0 ) ∩ A , ∅ there exists an open Then, Γ is upper hemicontinuous at x0 ∈ X
B 3 x0 such that Γ (x0 ) ∩ A , ∅ for all x0 ∈ B. if and only if ∀(xn )n with xn → x0 and (yn ∈
Γ (xn ))n , there is a convergent subsequence
Def. Γ : X ⇒ Y is continuous at x0 ∈ X if it is
(ykn )n such that limn→∞ ykn ∈ Γ (x0 ).
upper and lower hemicontinuous at x0 ∈ X.
2. Γ : X ⇒ Y is lower hemicontinuous at x0
If there is f : X → Y such that Γ (x) = {f (x)} if and only if ∀(xn )n with xn → x0 and
for all x ∈ X, then f is continuous if and ∀y ∈ Γ (x0 ) there is (yn ∈ Γ (xn ))n such that
only if Γ is continuous. yn → y.

4 Existence and Uniqueness

In this section, we fix a metric space hX, di. Def. x ∈ X is a fixed point of Γ : X ⇒ X if
x ∈ Γ (x).
4.1 Existence Results
Thm (Kakutani’s fixed point).4.4 Let A ⊂
Thm (intermediate value).4.1 Let f ∈ Rn be non-empty, compact and convex. Let
C 0 ([a, b], R). Then, for any y ∈ (f (a), f (b)), Γ : A ⇒ A have closed graph and be such that
there is some c ∈ [a, b] such that f (c) = y. Γ (x) is non-empty and convex for all x ∈ A.
Then Γ has a fixed point.
Thm (mean value).4.2 Let f ∈ C 0 ([a, b], R)∩
C 1 ((a, b), R). Then there exists some c ∈ (a, b)
f (b)−f (a)
such that f 0 (c) = b−a . Separating hyperplane theorem

Def. H ⊂ Rn is a hyperplane if H = { x | p·x =

Fixed point theorems
α } for some p ∈ Rn \{0} and α ∈ R.
An important (and extensive) class of re-
sults are the so-called “fixed point theo- Since p·(x−x0 ) = 0 ∀x, x0 ∈ H, p is called the
rems”. Here we provide some examples. normal of H. Note that x ∈ H then we can
write H = { x0 | (x0 −x) · p = 0 }.
Def. x ∈ X is a fixed point of f : X → X if
f (x) = x. Def. A, B ⊂ Rn are separated by a hyper-
Thm (Brouwer’s fixed point).4.3 Let A ⊂ Rn plane if there exist p ∈ Rn and α ∈ R such
be non-empty, compact and convex, and let that p·x ≤ α ≤ p·x0 for all x ∈ A, x0 ∈ B.
f ∈ C 0 (A, A). Then, f has a fixed point.
Thm (separating hyperplane).4.5 Let A, B ⊂
The previous result can be extended to cor- Rn be nonempty, convex, and satisfy A∩B = ∅.
respondences: Then, they are separated by a hyperplane.
4.2 Uniqueness Results that d(f (x), f (x0 )) ≤ λ d(x, x0 ) ∀x, x0 ∈ X.

Thm.4.6 Let f : R → R be strictly increasing. Thm (contraction mapping).4.8 Let f be a

Then, |{x ∈ R|f (x) = 0}| ≤ 1. contraction. Then, if X is complete, f has a
unique fixed point x∗ ∈ X.
Thm.4.7 Let f : R → R be concave and g : R →
R strictly convex. Then, |{x|f (x) = g(x)}| ≤ 2. If A ⊂ X is closed then it is itself a com-
plete metric space. Hence, if f (A) ⊂ A the
(unique) fixed point satisfies x∗ ∈ A.
Contraction mapping theorem
If x0 ∈ X then d(f n (x0 ), x∗ ) ≤ λn d(x0 , x∗ ), so
Def. A function f : X → X is a contraction we can approach x∗ recursively applying f
(mapping) if there is some λ ∈ [0, 1) such to any point x0 .

5 Optimization
In this section, we focus on optimization in Results for differentiable functions
Rn . We also fix some A ⊂ Rn and f : A → R.
Def. x∗ ∈ int(A) is a critical point of f ∈
Def. x∗ ∈ A is a (global) maximizer of f if
C 1 (A, R) if Df (x∗ ) = 0.
f (x) ≤ f (x∗ ) for all x ∈ A.

Note that x∗ is a maximizer of f is equiva- Df (x∗ ) = 0 is called the first order condition
lent to f (x∗ ) = max(f (A)), and equivalent to (FOC) for a maximum.
x∗ ∈ argmaxx∈A f (x). (Note that max(f (A))
exists if and only if the argument of the Thm.5.5 Let f ∈ C 1 (A, R). Then, if x ∈ A
maximum of f is non-empty.) is a maximizer of f , either x ∈ int(A) and
Def. x∗ ∈ A is a local maximizer of f if ∃ε > 0 Df (x) = 0, or x ∈ ∂A.
such that f (x) ≤ f (x∗ ) for all x ∈ Bε (x∗ )∩A.
Def. x∗ ∈ int(A) is a regular maximizer of
f ∈ C 2 (A, R) if Df (x∗ ) = 0 and Hf (x∗ ) is neg-
5.1 Some results
ative definite.
A very important result for the existence of
maximizers is the following: Hf (x∗ ) being negative definite is called the
Thm (extreme value).5.1 If A is non- second order condition (SOC) for a maxi-
empty and compact, and f is upper semi- mum.
continuous, then arg maxx∈A f (x) is non-
empty and compact. Thm.5.6 Let f ∈ C 2 (A, R). If x∗ is a regular
Thm.5.2 If A is convex and f is quasiconcave, maximizer of f then x∗ is a local maximizer
then arg maxx∈A f (x) is convex. of f . Conversely, if x∗ is a local maximizer of
f , then Hf (x∗ ) is negative semidefinite.
Thm.5.3 If A is convex, f is strictly concave.
Then f has at most one maximizer. Thm.5.7 Assume f ∈ C 2 (Rn , R) and that Hf is
Thm.5.4 If f is strictly convex and max(f (A)) negative definite. If x∗ ∈ Rn satisfies Df (x∗ ) =
exists, then arg maxx∈A f (x) ⊂ ∂A. 0, then x∗ is a global maximizer of f .
5.2 Constrained Optimization Thm.5.11 If C ⊂ Rn is convex and f : Rn → R
is quasiconcave and bounded, then V (θ) ≡
For the Lagrange problem, we fix a func- supx∈C f (x, θ) is quasiconcave.
tion h : Rn → RJ (also interpreted as a set
J
of functions {hj : Rn → R}j=1 . We want to
Thm (unconstrained envelope).5.12 Let f ∈
maximize f : Rn → R on {x|h(x) = 0} ⊂ Rn .
C 2 (Rn × Rm , R). Let x∗ ∈ Rn be a regular
Thm (Lagrange).5.8 Let f ∈ C 1 (Rn, R) and h ∈ maximizer of f at θ0 ∈ Rm . Then V (θ) ≡
C 1 (Rn, RJ ). Let x∗ ∈ argmaxx {f (x)|h(x) = 0}. If maxx f (x, θ) is differentiable and
rank(Dh(x∗ )) ≤ n there exist unique Lagrange
DV (θ0 ) = Dθ f (x∗ , θ0 ) ,
multipliers λ∗ ∈ RJ such that  
|
where Dθ ≡ ∂θ∂ 1 , ..., ∂θ∂m .
Df (x∗ ) − λ∗ Dh(x∗ ) = 0 .
Thm (Kuhn-Tucker).5.9 Let f ∈ C 1 (Rn, R) and Thm (constrained envelope).5.13 Let f ∈
g ∈ C 1 (Rn, RI ). Let x∗ ∈ argmaxx {f (x)|g(x) ≤ C 2 (Rn ×Rm , R) and h ∈ C 2 (Rn , Rm ). Let x∗ ∈
0} such that rank(Dg(x∗ )) = b. If b ≤ n there Rn be a regular maximizer of f at θ0 , and
exist nonnegative Lagrange multipliers λ∗ ∈ let λ∗ be the corresponding Lagrange multi-
RI+ such that pliers. Then V (θ) ≡ maxx∈{x0 |h(x0 ,θ)=0} f (x, θ)
|
Df (x∗ ) − λ∗ Dg(x∗ ) = 0 is differentiable and
and the complementary slackness conditions DV (θ0 ) = Dθ L(x∗ , λ∗ , θ0 ) .
are satisfied: λi∗ g i (x∗ ) = 0 for all i = 1, ..., I.
Thm (Berge’s or maximum).5.14 Let f : X ×
5.3 Comparative Statics Θ → R be continuous, and C : Θ ⇒ X be com-
pact-valued and non-empty. For each θ ∈ Θ,
In a model we have two types of objects: let
• Exogenous: Variables or functions given V (θ) ≡ max f (x, θ) ,
in the assumptions of the model. x∈C(θ)
∗
• Endogenous: Variables or functions un- C (θ) ≡ arg max {f (x, θ)} .
x∈C(θ)
known before the model is solved. If C is continuous at some θ0 ∈ Θ, then V
Thm.5.10 If f : Rn × Rm → R is quasicon- is continuous at θ0 and C ∗ is non-empty,
cave and bounded and C ⊂ Rm is convex, then compact-valued, and upper hemicontinuous
V (θ) ≡ supx∈C f (x, θ) is quasiconcave. at θ0 .

6 Dynamic Optimization
We now study the problem of maximizing For each initial term x1 ∈ Rn , the recur-
some objective function over time. rence can be used to recursively obtain a
sequence (xt )∞
t=1 where, for all t ∈ N,
xt+1 = gt+1 (x1 , ..., xt ) ≡ gt+1 ((xt0 )tt0 =1 ) .
6.1 Recurrences
Def. A recurrence {gt+1 |t ∈ N} has order
Def. A (n-dimensional) recurrence (rela- d if for each t ≥ d there is a function
tion) is a set of functions {gt+1 |t ∈ N}, where ĝt+1 : Rn×d → Rn such that gt+1 (x1 , ..., xt ) =
gt+1 : (Rn )t+1 → Rn for all t. ĝt+1 (xt−d+1 , ..., xt ).
In general, an order-d recurrence can be 6.2 The Dynamic Optimization
transformed into an order-1 recurrence of Problem
higher dimensionality. For example:
 1 
 xt+1 
 1 
 xt   xt2

 Def. A setting is hX, Γ , F, βi, where:
 2 
 xt+1 
 2  
 xt   .. 
1. X ⊂ Rn is the set of states.
 = g̃   ≡  . 
 .
 ..   ...  

t+1  d  2. Γ : X ⇒ X is the constraint correspondence.
 .     xt 
3. F : X × X → R the per-period payoff func-
  
d  d  1 d 
xt+1 xt gt+d (xt , ..., xt )
tion. We assume it is bounded.
Then, (xt1 , ..., xtd ) = (xt , ..., xt+d ) for all t.
4. β ∈ (0, 1) is the discount factor.
Def. A recurrence is homogeneous of order
d if there is some g : (Rn )d → Rn such that With some abuse of notation, XΓN (x1 ) de-
ĝt+1 = g for all t ≥ d. notes the set of feasible sequences (i.e.,
(xt0 )∞ 0
t=1 such that x1 = x1 and xt+1 ∈ Γ (xt ) for
Typically, instead of specifying {g2 , ..., gd−1 }, all t). The payoff of a feasible sequence
homogeneous recurrences are defined by g (xt )∞t=1 is
and d initial conditions, x1 , x2 , ..., xd . V ((xt )∞
P∞ t−1
t=1 ) ≡ t=1 β F(xt , xt+1 ) .

Order-1 homogeneous recurrences Def. The Sequence Problem (SP) for the set-
ting hX, Γ , F, βi consists on finding, ∀x1 ,
From now we fix some order-1 homoge- V ∗ (x1 ) ≡ sup (xt )∞ V ((xt )∞
N
t=1 ∈XΓ (x1 ) t=1 ) ,
neous recurrence characterized by g.
and on finding, for all x1∗ , some (xt∗ )∞ t=1 ∈
N ∗ ∗ ∞ ∗ ∗
Def. x∗ ∈ Rn is steady state of a homoge- XΓ (x1 ) such that V ((xt )t=1 ) =V (x1 ).
neous recurrence if x∗ = g(x∗ ).

Thm.6.1 If (xt )t converges to x∗ and g is con- 6.3 Recursive Analysis

tinuous then x∗ is a steady state.
Thm (optimality).6.4 Any solution of (SP)
Def. A steady state x∗ is locally stable if ε > 0 has property that, whatever the initial state
exists such that xt → x∗ for all x1 ∈ Bε (x∗ ). and decisions are, the remaining decisions
constitute an optimal policy with regard to
Note that the set of locally stable steady the state resulting from the first decisions.
states does not have any limit point. That is:
(xt∗ )∞ ∞
t=1 ∈ argmax V ((xt )t=1 ) ⇒
Def. A steady state x∗ is globally stable if (xt )∞ N
t=1 ∈XΓ (x1 )
xt → x∗ for all x1 . (xt∗ )∞ argmax V ((xt )∞
t=T ∈ t=1 ) ∀T > 0 .
(xt )∞ N ∗
t=1 ∈XΓ (xT )
Thm.6.2 If g : A → A is a contraction and A is
complete, then g has a unique globally-stable Thm (optimality).6.5 (xt∗ )∞t=1 is a solution of
steady state. ∗
(SP) for x1 if and only if the Bellman equa-
tion holds for all t ∈ N:
Thm.6.3 Assume g ∈ C 1 (R, R). Then x∗ is a
locally stable steady state only if |g 0 (x∗ )| ≤ 1. V ∗ (xt∗ ) = max ∗ (F(xt∗ , xt+1 )+β V ∗ (xt+1 )) .
xt+1 ∈Γ (xt )
Furthermore, if |g 0 (x)| ≤ λ for all x and some Moreover, for all t ∈ N,
λ < 1, and a steady state x∗ exists, and is glob- ∗
xt+1 ∈ argmax (F(xt∗ , xt+1 )+β V ∗ (xt+1 )) .
ally stable. xt+1 ∈Γ (xt )
Thm (sufficient conditions).6.6 Assume X Def. An alternative setting is hX, C, h, F a , βi,
is convex, Γ nonempty, compact valued and where:
continuous, F bounded, continuous, strictly 1. X ⊂ Rn is the set of states.
increasing in the first variable, strictly con- 2. C ⊂ Rm is the set of actions.
cave and continuously differentiable in the in- 3. h : X × C → X is the law of motion.
terior of X × X. Assume (xt∗ )∞
t=1 is such that 4. F a : X ×C → R the per-period payoff func-
∗ ∗
xt+1 ∈ int Γ (xt ), tion. We assume it is bounded.
1. the Euler equations hold: 5. β ∈ (0, 1) is the discount factor.
0 = F2 (xt∗ ,xt+1
∗ ∗
)+βF1 (xt+1 ∗
,xt+2 ) ∀t ≥ 1 ,
For each x1 and sequence (ct )∞ t=1 , we can
2. the transversality condition holds: t
recursively define (xt+1 (x1 , (ct0 )t0 =1 ))t by re-
lim β t−1 F1 (xt∗ ,xt+1
∗
) xt∗ = 0 . quiring xt+1 (x1,(ct0 )tt0 =1 ) = h(xt (x1,(ct0 )t−1
t→∞ t 0 =1 ),ct ).
Then, (xt∗ )∞ a solution of (SP) for x1∗ . The payoff of a sequence (ct )∞ is
t=1 is P∞ t−1 a
t=1
a ∞
V (x1,(ct)t=1 ) ≡ t=1 β F (xt (x1,(ct0)t−1 t 0 =1 ),ct) .
Def. A strategy is a recurrence {gt+1 |t ∈ N}
such that gt+1 (x1 , ..., xt ) ∈ Γ (xt ) for all t. Def. The Alternative Sequence Prob-
lem (SPa ) for the alternative setting
Using g to denote a strategy {gt+1 |t ∈ N}, we hX, C, h, F a , βi consists on finding,
g
can define a feasible sequence (xt (x1 ))∞ t=1 . V a∗ (x1 ) ≡ sup (c1 )∞
t=1 ∈C
a ∞
N V (x1 , (ct )t=1 ) ,
We can then also define a payoff
t−1 F(xg (x ), xg (x )) .
and on finding, for all x1∗ , some (ct∗ )∞ t=1 ∈ C
N
V g (x1 ) ≡ ∞
P
t=1 β t 1 t+1 1 ∗ ∗ ∞ ∗
such that V a (x1 , (ct )t=1 ) =V a∗ (x1 ).
Def. g is a solution of (SP) if V g = V ∗ . Thm.6.10 Let hX, C, F a , h, βi be an alt. setting.
Def. A Markov strategy is a map g : X → X Define the setting hX, Γ , F, βi, where Γ (x) ≡
such that g(x) ∈ Γ (x) for all x. h(x, C) and F(x, x0 ) ≡ sup{F a (x, c)|h(x, c) = x0 },
for all x and ∀x0 ∈ Γ (x). Then, V a∗ = V ∗ .
A Markov strategy is an order-1 strategy.

Thm.6.7 g ∗ is a Markov solution if and only if 6.4 Existence and Uniqueness

V ∗ (x) = F(x, g ∗ (x)) + β V ∗ (g ∗ (x)) ∀x ∈ X .
Let CB (X) (B(X), resp.) the set of contin-
Thm.6.8 If (SP) has a solution then it has a uous and bounded (bounded, resp.) func-
Markov solution. (SP) has a unique solution tions from X to R. Let ||·|| denote the sup
only if it has a unique Markov solution. norm on B(X), i.e. ||f || ≡ supx∈X |f (x)|.

Thm (one-shot deviation principle).6.9 Fix Thm.6.11 Both B(X) and CB (X) are complete
some Markov strategy g, and let V g be the with the metric d(f , g) ≡ supx∈X |f (x) − g(x)|.
corresponding value function. Then, g is a Thm (Blackwell’s sufficient cond.).6.12 Let
Markov solution if and only if T : B(X) → B(X) be such that
g(x) ∈ arg max (F(x, x0 )+β V g (x0 )) ∀x ∈ X . 1. Monotonicity: if f , g ∈ B(X) are such that
x0 ∈Γ (x)
f (x) ≤ g(x) for all x ∈ X then T f (x) ≤ T g(x)
for all x ∈ X, and
Alternative formulation
2. Discounting: there is β ∈ [0, 1) such that
An alternative formulation of (SP) sepa- for any a ∈ R+ , T [f + a](x) ≤ T f (x) + a β
rates states and actions. This approach is for all x ∈ X.
easy to generalize to stochastic settings. Then T is a contraction.
Define T : CB (X) → RX given by 2. P is irreducible if for all x, x0 there is t
0
T [V ](x) = sup (F(x, x0 ) + β V (x0 )) . such that (P t )x,x > 0.
x0 ∈Γ (x) 3. x is an absorbing state of P if P x,x = 1.
Note that V solves the Bellman equation if
and only if V = T [V ] ≡ T V . Def. p is a probability vector if px ≥ 0 and
P x
x∈X p = 1. ∆(X) denotes the set of prob-
Thm.6.13 T f ∈ CB (X) for all f ∈ CB (X). ability distributions.

Thm.6.14 T : CB (X) → CB (X) is a contraction Def. p∗ ∈ ∆(X) is a stationary distribution of

and therefore has a unique fixed point. P if p∗ ≡ P | p∗ .

Note that if the distribution of states at

Properties of V ∗
time t is pt = p∗ , then pt+1 = p∗ .
Thm.6.15 Assume X ⊂ R, and F(·, x) and Γ Thm.6.18 If P is irreducible and aperiodic then
are increasing for all x. Then, the unique V ∗ it has a unique stationary distribution p∗ .
satisfying V = T V is increasing. If F(·, x) is Furthermore,
strictly increasing for all x, so is V ∗ . |
1. limt→∞ P t = 1 p∗ , where 1 ∈ RX has 1 in
Thm.6.16 Assume that X and Γ are convex and all components (i.e., 1x = 1 for all x).
F is concave. Then, the unique V ∗ satisfying 2. For any p ∈ ∆(X), limt→∞ (P | )t p = p∗ .
V ∗ = T V ∗ is concave. Moreover, if F(·, x) is
strictly concave for all x, so is V ∗ . Settings
Thm (Env., Benveniste&Sheinkman).6.17 For convenience, we now omit the label
Suppose F is strictly concave and Γ is con- and superindex to indicate alternative set-
vex. In this case, the optimal policy corre- tings.
spondence is a function g ∗ (·). If x ∈ int(X)
and g ∗ (x) ∈ int(X) then the fixed point of T , Def. A setting is hX, C, P , F, βi, where:
V ∗ , is differentiable at x and 1. X a finite set of states.
V ∗0 (x) = F1 (x, g ∗ (x)) . 2. C ⊂ Rm is the set of actions.
3. Let P : X ×C → ∆(X) is the law of motion.
0
We use P x,x (c) to denote the probabil-
6.5 Finite X ity that state x transitions to state x0 in
Assume now finite states, |X| = N ∈ N. We some given period where the action is c.
interpret RX as both the set of functions 4. F : X ×C → R is the per-period payoff func.
from X to R and the set of |X|-dimensional 5. β ∈ (0, 1) is the discount factor.
vectors R|X| .
A Markov strategy g ∈ C X generates a Mar-
0 0
Def. P ∈ RX×X is a stochastic matrix if kov chain Pgx,x ≡ P x,x (g x ), where g x ≡ g(x).
0
• P x,x ≥ 0 for all x, x0 , and Def. Given P , the probability of (x1 , ..., xt ) is
0
• x0 ∈X P x,x = 1 for all x.
P
xt 0 ,xt 0 +1
Pr((xt0 )tt0 =1 |g) ≡ t−1
Q
t 0 =1 Pg .
0
We interpret P x,x as the probability that Def. The expected payoff for a Markov
state x transitions to state x0 in some given chain P when the initial state is x1 is
period. t−1
x P t00 −1 x 00 x 00
Vg 1 ≡ lim Pr((xt0 )tt0 =1 |g)
P
β F t (g t ) .
Def. 1. P is aperiodic if P x,x > 0 for all x. t→∞
(xt 0 )tt 0 =1 t 00 =1
Thm.6.19 For any strategy g, the expected pay- The continuation payoff is divided be-
off Vg : X → R is the unique solution to tween the current period’s payoff and the
0 0
Vgx = x0 ∈X (Fgx + β P x,x Vgx ) for all x .
P expected continuation payoff.
In matrix form, Vg = Fg +β Pg Vg , or Vg = (IN − 0
Thm.6.20 If, for all x and x0 , P x,x (·) is contin-
βPg )−1 Fg where IN is the (N×N )-identity, and uous, F x (·) upper semi-continuous, and C is
where Fgx ≡ F x (g x ) for all x ∈ X. compact, the Bellman equation has a unique
1
If x is absorbing then Vgx = 1−β Fgx . solution V∗ ∈ RX . Furthermore, there exists
If Pg is irreducible and aperiodic, and p∗g is an optimal policy g∗ ∈ RX so that
0 0
V∗x = F x (g∗x ) + β x0 ∈X P x,x (g∗x ) V∗x ,
P
its stationary distribution, then
x0 (1−β) V x0
lim (1−β) Vgx = lim x0 ∈X p∗g
P or V∗ = Fg∗ +β Pg∗ V∗ .
g
β→1 β→1
x0 x0 Thm (one-shot deviation principle).6.21
P
= x0 ∈X p∗g Fg .
Fix some strategy g : X → C, and let V g be
The Bellman Equation the corresponding value function. Then, the
strategy is optimal if and only if
The Bellman equation can be written as
 
x x P x,x 0 x 0
g ∈ argmax F (c) + β x0 ∈X P (c) Vg
x,x0 (cx ) V x0 .
 
V∗x = max x x P
x
F (c ) + β 0
x ∈X P ∗ c∈C
c ∈Γx for all x ∈ X.

7 Notation
- Absolute value of x: |x| - Intersection between A and B: A ∩ B
- Argument of the maximum: argmax - Hypograph of f : hypf
- Ball of radius ε around x: Bε (x) - Jacobian of f : Df
- (x) Belongs to A: x ∈ A - Limit of (xn )n : limn→∞ xn
- Boundary of A: ∂A - Maximum of A: max(A)
- Cardinality of A: |A| - Maximum of f on A: maxx∈A f (x)
- Cartesian product of A and B: A×B - Norm of v: kvk
- Closure of A: A - Power set of A: 2A
- Composition of g and f : g ◦ f - Restriction of f to A: f |A
- Complement of A: Ac - Sequence: (xn )n
- Correspondence from X to Y : Γ : X ⇒ Y - Set of functs. from X to Y : Y X
- Derivative of f : f 0 (x), f (1) (x), df (x)/dx - Set of cont. functs. from X to Y : C 0 (X, Y )
- Epigraph of f : epif - Subsequence: (xkn )n
- Exists some x in X: ∃x ∈ X - (B) Subset of A: B ⊂ A
- For all x in X: ∀x ∈ X - Supremum of A: sup(A)
- Function from X to Y : f : X → Y , f ∈ Y X - Supremum of f on A: supx∈A f (x)
- Interior of A: int(A) - Union between A and B: A ∪ B