Announcements

More Lambda Calculus

Work on your project (probably

background reading)
I am looking at your proposals, but come
talk to me if you have concerns

Meeting 17, CSCI 5535, Spring 2009

Plan

Recall

Last Time (Introduce Lambda Calculus)

Syntax
Substitution

Goal: Come up with a core language thats as

Goal
small as possible and still Turing
complete

Today (Lambda Calculus in Real Life)

Operational Semantics
Evaluations strategies
Equality
Encodings
Fixed Points

This will give a way of illustrating important

language features and algorithms

Lambda Syntax

Combinators

The -calculus has 3 kinds of expressions (terms)

e ::= x
Variables
| x. e
Functions (abstractions)
| e1 e2
Application
x. e is a one-argument anonymous function
with body e
e1 e2 is a function application

A -term without free variables is closed or a

combinator
Some interesting combinators:
I
= x. x
Explain
K
= x. y. x
these
informally
S
= f. g. x. f x (g x)
D
= x. x x
Y
= f. ( x. f (x x)) ( x. f (x x))
Theorem: any closed term is equivalent to one
written with just S, K and I
Example: D = S I I
(well discuss this form of equivalence later)

Informal Semantics

Informal Semantics

All weve got are functions, so all we can do is

call them!

All weve got are functions, so all we can do is

call them!
The evaluation of
( x. e) e
Binds x to e
Evaluates e with the new binding
Yields the result of this evaluation

Like a function call, or like let x = e in e

Example:
( f. f (f e)) g evaluates to g (g e)
7

Informal Semantics

Operational Semantics

All weve got are functions, so all we can do is

call them!
The evaluation of
( x. e) e

Many operational semantics for the -calculus

All are based on the equation
(
x. e1) e2 = [e2/x]e1
usually read from left to right
This is called the -rule and the evaluation
step a -reduction
The subterm ( x. e1) e2 is a -redex
We write e e to say that e -reduces to e
in one step
We write e * e to say that e -reduces to e
in 0 or more steps

Binds x to e
Evaluates e with the new binding
Yields the result of this evaluation

Like a function call, or like let x = e in e

Example:
( f. f (f e)) g evaluates to g (g e)

Remind you of the small-step opsem term rewriting?

9

Operational Semantics

10

Examples of Evaluation
The identity function:
(
x. x) E [E / x] x = E
Another example with the identity:
(
f. f (
x. x)) (
x. x)

betabeta
-reduction

(x.e1) e2 [e2/x]e1

11

12

Examples of Evaluation

Examples of Evaluation

The identity function:

(
x. x) E [E / x] x = E
Another example with the identity:
(
f. f (
x. x)) (
x. x)
[
x. x / f] f (
x. x)) =
[
x. x / f] f (
y. y)) =
(
x. x) (
y. y)
[
y. y / x] x = y. y

A non-terminating evaluation:
(
x. xx) (
y. yy)
[
y. yy / x] xx = (
y. yy) (
y. yy)

Try T T, where T = x. x x x

14

13

Evaluation and the Static Scope

Another View of Reduction

The definition of substitution

guarantees that evaluation respects
static scoping:

The application
x. e

( x. ( y. y x)) (y ( x. x)) z. z (y ( v. v))

e
xxx

(y remains free, i.e., defined externally)

Becomes:

If we forget to rename the bound y:

( x. ( y. y x)) (y ( x. x)) * y. y (y ( v. v))

g

(y was free before but is bound now)

(terms can grow substantially through -reduction!)

15

16

Normal Forms

Structural Operational Semantics

A term without redexes is in normal form

A reduction sequence stops at a normal form

We define a small-step reduction relation

(x. e1) e2 [e2/x]e1

If e is in normal form and e * e then

e is identical to e

e1 e1
e1 e2 e1 e2

e2 e2
e1 e2 e1 e2

e e
x. e x. e
This is a non-deterministic semantics
Note that we evaluate under (where?)

K = x. y. x is in normal form
K I is not in normal form
17

18

Lambda Calculus Contexts

Lambda Calculus Contexts

Define contexts with one hole

H ::= |

Define contexts with one hole

H ::= | x. H | H e | e H
Write H[e] to denote the filling of the
hole in H with the expression e
Example:
H = x. x H[ y. y] = x. x ( y. y)
Filling the hole allows variable capture!
H = x. x H[x] = x. x x
19

Contextual Operational Semantics

20

Contextual Operational Semantics

e e
H[e] H[e]

(x. e1) e2 [e2/x]e1

21

Contexts allow concise formulations of

congruence rules (application of local
reduction rules on subterms)
Reduction occurs at a -redex that can be
anywhere inside the expression
The latter rule is called a congruence or
structural rule
The above rules to not specify which redex
must be reduced first

22

The Order of Evaluation

The Diamond Property

In a -term there could be more than

one instance of ( x. e1) e2, as in:
( y. ( x. x) y) E

A relation R has the diamond property if whenever

e R e1 and e R e2 then there exists e such that e1 R e
and e2 R e
e
R
R

Could reduce the inner or outer

Which one should we pick?

R
e

inner
outer
(
y. [y/x] x) E = (
y. y) E [E/y] (
x. x) y = (
x. x) E

e2

e1

(
y. (
x. x) y) E

23

does not have the diamond property

* has the diamond property (Church-Rosser
property, confluence property)
simplest known proof is quite technical

24

A Diamond In The Rough

Beta Equality

Languages defined by non-deterministic

sets of rules are common

Let = be the reflexive, transitive and

symmetric closure of
= is (
)*

Logic programming languages

Expert systems
Constraint satisfaction systems

That is, e = e if e converts to e via a

sequence of forward and backward

And thus most pointer analyses

Dataflow systems
Makefiles

It is useful to know whether such

systems have the diamond property
25

26

The Church-Rosser Theorem

Corollaries

If e1 = e2 then there exists e such that

e1 * e and e2 * e

e1

e2

If e1 = e2 and e1 and e2 are normal

forms then e1 is identical to e2

Proof (sketch): apply the diamond

property as many times as necessary

From CR we have e. e1 * e and e2 * e

Since e1 and e2 are normal forms they are
identical to e

If e * e1 and e * e2 and e1 and e2

are normal forms then e1 is identical to e2
All terms have a unique normal form.

27

28

Evaluation Strategies

Evaluation Strategies Summary

Church-Rosser says that independent of the

reduction strategy we will find 1 normal form
But some reduction strategies might find 0

Normal Order

( x. z) (( y. y y) ( y. y y))
( x. z) (( y. y y) ( y. y y))

Evaluates the left-most redex not contained in

another redex
If there is a normal form, this finds it
Not used in practice: requires partially evaluating
function pointers and looking inside functions

Call-By-Name (lazy)

( x. z) (( y. y y) ( y. y y)) z

Dont reduce under , dont evaluate a function

argument (until you need to)
Does not always evaluate to a normal form

There are three traditional strategies

normal order (never used, always works)
call-by-name (rarely used, cf. TeX)
call-by-value (amazingly popular)

Call-By-Value (eager or strict)

29

Dont reduce under , do evaluate a functions

argument right away
Finds normal forms less often than the other two30

Normal-Order Reduction
A redex is outermost if it is not contained
inside another redex
Example:
S (K x y) (K u v)
K x, K u and S (K x y) are all redexes
Both K u and S (K x y) are outermost
Normal order always reduces the leftmost
outermost redex first

Bonus: Evaluation
Strategies

Theorem: If e has a normal form e then

normal order reduction will reduce e to e

Why Not Normal Order ?

Call-by-Name

In most (all?) programming languages,

functions are considered values (fully
evaluated)
Example:
x. D D = (with normal order)
Thus, no reduction is done under lambda

Dont reduce under

Dont evaluate the argument to a function call
A value is an abstraction

No popular programming language uses

normal order

e1 n* x. e1 [e2/x]e1 n* e
x. en* x. e

e1 e2 n* e

Call-by-name is demand-driven: an expression is not

evaluated unless needed
It is normalizing: converges whenever normal order
converges
Call-by-name does not necessarily evaluate to a normal
form. Example: D D

33

34

Call by Name Example

Call-by-Value Evaluation

Example:

Dont reduce under lambda

Do evaluate the arguments to a function call
A value is an abstraction

(y. (x. x) y) ((u. u) (v. v)) n

32

e1 v* x. e1 e2 v* e2 [e2/x]e1 v* e

(x. x) ((u. u) (v. v)) n

x. ev* x. e

(u. u) (v. v) n

e1 e2 v* e

Most languages are primarily call-by-value

But CBV is not normalizing: (x. I) (D D)
CBV diverges more often than normal order or
even CBN

v. v

35

36

Call by Value Example

Evaluation Strategy Considerations

Example:

Call-by-value:
easy to implement
well-behaved (predictable) with respect to side-effects

(y. (x. x) y) ((u. u) (v. v)) v

Call-by-name:
More difficult to implement (must pass unevaluated
expressions)
The order of evaluation is harder to predict (e.g., difficulty
with side-effects)
Has a simpler theory than call-by-value
Allows the natural expression of infinite data structures (e.g.
streams)
Terminates more often than call-by-value

(y. (x. x) y) (v. v) v

(x. x) (v. v) v
v. v

37

38

Which is Better?

Caveats

The debate about whether languages should

be strict (CBV) or lazy (CBN) is nearly 20
years old

The terms lazy and strict are not used consistently in

the literature
Call-by-value and call-by-name are well defined
There are parameter passing mechanisms besides callby-value and call-by-name that cannot be naturally
expressed in the lambda calculus

This debate is typically confined to the

functional programming community (where it is
sometimes intense)

by reference
value-result

These additional mechanisms deal with side-effects

Many languages have a mixture of parameter passing
mechanisms

Outside the functional community CBN is

rarely considered (but remember TeX)

40

39

Functional Programming

How is -calculus related

to real life?

The -calculus is a prototypical

functional language with:

no side effects
several evaluation strategies
lots of functions
nothing but functions (pure -calculus does
not have any other data type)

How can we program with functions?

How can we program with only
functions?

42

Programming With Functions

Referential Transparency

Functional programming is a programming style

that relies on lots of functions
A typical functional paradigm is using

In pure functional programs, we can reason

equationally, by substitution
Called referential transparency
let x = e1 in e2 ===
[e1/x]e2
In an imperative language a side-effect in e1 might
invalidate the above equation
Why?

functions as arguments or results of other

functions
Called higher-order programming

Some impure functional languages permit

side-effects (e.g., Lisp, Scheme, ML, Python)
references (pointers), in-place update, arrays,
exceptions
Others (and by others we mean Haskell) use
monads to model state updates

43

The behavior of a function in a pure functional

language depends only on the actual arguments
Just like a function in math
This makes it easier to understand and to reason
about functional programs

44

How Complex Is Lambda?

Expressiveness of -Calculus

Given e1 and e2, how complex (a la CS

theory) is it to determine if:
e1 * e and e2 * e

The -calculus is a minimal system but can express

data types (integers, booleans, lists, trees, etc.)
branching, recursion

This is enough to encode Turing machines

We say the lambda calculus is Turing-complete
Corollary: e1 = e2 is undecidable

That means we can encode any computation we

want in it ... if were sufficiently clever

45

46

Encodings

Encoding Booleans in -Calculus

Still, how do we encode all these

constructs using only functions?
Idea: encode the behavior of values
and not their structure

What can we do with a boolean?

we can make a binary choice (= if exp)

A boolean is a function that, given two

choices, selects one of them:
true
=def x. y. x
false
=def x. y. y
if E1 then E2 else E3 =def E1 E2 E3

47

48

Encoding Booleans in -Calculus

Lets try to define or

What can we do with a boolean?

Recall:

we can make a binary choice (= if exp)

true
false
if E1 then E2 else E3

A boolean is a function that, given two

choices, selects one of them:

=def x. y. x
=def x. y. y
=def E1 E2 E3

Intuition:

true
=def x. y. x
false
=def x. y. y
if E1 then E2 else E3 =def E1 E2 E3

or a b = if a then true else b

Either of these will work:

Example: if true then u else v is

or
or

(x. y. x) u v (y. u) v u

=def a. b. a true b
=def a. b. x. y. a x (b x y)

49

50

This is getting painful to check

More Boolean Encodings

Lets use OCaml

Think about how to do and and not

Without peeking!

51

52

Encoding and and not

Encoding Pairs in -Calculus

and a b

What can we do with a pair?

and
and

not a
not
not

= if a then b else false

we can access one of its elements

(= field access)

=def a. b. a b false
=def a. b. x. y. a (b x y) y

= if a then false else true

=def a. a false true
=def a. x. y. a y x

A pair is a function that, given a boolean,

returns the first or second element
mkpair x y =def
fst p
=def
snd p
=def
fst (mkpair x y)

true x y
53

b. b x y
p true
p false

(mkpair x y) true
x
54

Computing with Natural Numbers

Encoding Numbers -Calculus

What can we do with a natural number?
we can iterate a number of times over some function
loop)

(= for

A natural number is a function that given an operation f

and a starting value z, applies f a number of times to z:
0
=def f. z. z
1
=def f. z. f z
2
=def f. z. f (f z)
Very similar to List.fold_left and friends

These are numerals in a unary representation

Called Church numerals

The successor function

succ n
or
succ n
Addition
plus n1 n2
Multiplication
mult n1 n2
Testing equality with 0
iszero n
Subtraction

=def f. s. f (n f s)
=def f. s. n f (f s)
=def n1 succ n2
=def n1 (add n2) 0
=def n (b. false) true

Is not instructive, but makes a fun exercise

55

56

Computation Example

Toward Recursion

What is the result of the application add 0?

Given a predicate p, encode the function find

such that find p n is the smallest natural
number which is at least n and satisfies p
Ideas? How do we begin?

(n1. n2. n1 succ n2) 0

n2. 0 succ n2 =
n2. (f. s. s) succ n2
n2. n2 =
x. x

By computing with functions we can

express some optimizations
But we need to reduce under the lambda
Thus this never happens in practice

57

58

Encoding Recursion

The Fixed-Point Combinator Y

Given a predicate p, encode the function find

such that find p n is the smallest natural
number which is at least n and satisfies p
find satisfies the equation

Let Y = F. (y.F(y y)) (x. F(x x))

find p n = if p n then n else find p (succ n)

Define

This is called the fixed-point combinator

Verify that Y F is a fixed point of F
Y F (y.F (y y)) (x. F (x x)) F (Y F)
Thus Y F = F (Y F)

Given any function in -calculus we can

compute its fixed-point (!)
Thus we can define find as the fixed-point
of the function F from the previous slide
Essence of recursion is the self-application y y

F = f.p.n.(p n) n (f p (succ n))

We need a fixed point of F

find = F find

or
find p n = F find p n
59

60

10

Expressiveness of Lambda Calculus

Encodings are fun
Yes! Yes they are!

But programming in pure -calculus is

painful
So we will add constants (0, 1, 2, , true,
false, if-then-else, etc.)
Next we will add types
61

11