Amortized Analysis

• We examined worst-case, average-case and best-case

analysis performance

• In amortized analysis we care for the cost of one

operation if considered in a sequence of n operations
- In a sequence of n operations, some operations may
be cheap, some may be expensive (actual cost)
- The amortized cost of an operation equals
the total cost of the n operations divided by n.
 Amortized Analysis

• Think of it this way:

– You have a bank account with 1000$ and you want
to go shopping and purchase some items…

– Some items you buy cost 1$, some items you buy
cost 100$
– You purchase 20 items in total, therefore…

– …the amortized cost of each purchase is 5$

 Amortized Analysis

• AMORTIZED ANALYSIS:

You try to estimate an upper bound of the total

work T(n) required for a sequence of n operations…

Some operations may be cheap some may be expensive.

Overall, your algorithm does T(n) of work for n operations …

Therefore, by simple reasoning, the amortized cost of each

operation is T(n)/n
 Amortized Analysis

• Imagine T(n) (the budget) being the number

of CPU cycles a computer needs to solve the
problem

• If computer spends T(n) cycles for n operations,

each operation needs T(n)/n amortized time
 Amortized Analysis

• We prove amortized run times with the accounting

method. We present how it works with two examples:
– Stack example
– Binary counter example

• We describe Insert/Search/Delete/Join/Split in
Splay Trees. Accounting method can show that these
operations have O(logn) amortized cost (run time) and
they are “balanced” just like AVL trees
– We do not show the analysis behind the O(logn) run time
 Amortized Analysis: Stack Example

Consider a stack S that holds up to n elements and it has the

following three operations:

PUSH(S, x) ………. pushes object x in stack S

POP(S) ………. pops top of stack S

MULTIPOP(S, k) … pops the k top elements of S or pops the

entire stack if it has less than k elements
 Amortized Analysis: Stack Example

• How much a sequence of n PUSH(), POP()

and MULTIPOP() operations cost?
– A MULTIPOP() may take O(n) time

– Therefore (a naïve way of thinking says that): a sequence

of n such operations may take O(n n) = O(n2) time since we
may call n MULTIPOP() operations of O(n) time each

With accounting method (amortized analysis) we can

show a better run time of O(1) per operation!
 Amortized Analysis: Stack Example
Accounting method:
• Charge each operation an amount of dollars $:
– Some money pays for the actual cost of the operation
– Some is deposited to pay for future operations
– Stack element credit invariant: 1$ deposited on it

Actual cost Amortized cost

PUSH 1 PUSH 2
POP 1 POP 0
MULTIPOP min(k, S) MULTIPOP 0
 Amortized Analysis: Stack Example

• In amortized analysis with accounting method

we charge (amortized cost) the following $:

– We let a POP() and a MULTIPOP() cost nothing

– We let a PUSH() cost 2$:

• 1$ pays for the actual cost of the operation
• 1$ is deposited on the element to pay if POP-ed
 Amortized Analysis: Stack Example
credit
invariant c 1$
1$ b b 1$
a 1$
1$ a a 1$

Push(a) = 2$ Push(b) = 2$ Push(c) = 2$

1$ pays for push 1$ pays for push 1$ pays for push
and 1$ is deposited and 1$ is deposited and 1$ is deposited

MULTIPOP() costs nothing

because you have the 1$ bills to
pay for the pop operations!
 Accounting Method

• We charge operations a certain amount of money

• We operate with a budget T(n)

– A sequence of n POP(), MULTIPOP(), and PUSH()
operations needs a budget T(n) of at most 2n $

– Each operation costs

T(n)/n = 2n/n = O(1) amortized time

 Binary Counter Example

• Let n-bit counter A[n-1]…A[0] (counts from 0 to 2n):

– How much work does it take to increment the

counter n times starting from zero?

– Work T(n): how many bits do you need to flip

(0Æ1 and 1Æ0) as you increment the counter …
 Binary Counter Example

INCREMENT(A)
1. i=0;
2. while i < length(A) and A[i]=1 do
3. A[i]=0;
4. i=i+1;
5. if i < length(A) then
6. A[i] = 1

This procedure resets the first i-th sequence of 1 bits and sets
A[i] equal to 1 (ex. 0011 Æ 0100, 0101 Æ 0110, 0111 Æ 1000)
 Binary Counter Example
4-bit counter:
Counter value COUNTER Bits flipped (work T(n))
0 0000 0
1 0001 1
2 0010 3
3 0011 4
4 0100 7
5 0101 8
6 0110 10
7 0111 11
8 1000 15
A3A2A1A0
highlighted are bits that flip at each increment
 Binary Counter Example

• A naïve approach says that a sequence of n operations on a

n-bit counter needs O(n2) work
– Each INCREMENT() takes up to O(n) time. n INCREMENT()
operations can take O(n2) time

• Amortized analysis with accounting method

– We show that amortized cost per INCREMENT() is only O(1)
and the total work O(n)
– OBSERVATION: In example, T(n) (work) is never
twice the amount of counter value (total # of increments)
 Binary Counter Example

• Charge each 0Æ1 flip 2$ in line 6

– 1$ pays for the 0Æ1 flip in line 6

– 1$ is deposited to pay for the 1Æ0 flip later in line 3

• Therefore, a sequence of n INCREMENTS() needs

T(n)= 2n $

• …each INCREMENT() has an amortized cost of

2n/n = O(1)
 Binary Counter Example
Credit invariant $ $
0 0 0 0 0 0 0 1 0 0 1 0

• Charge 2$ for every 0Æ1 bit flip.

1$ pays for the actual operation $ $
• Every 1 bit has 1 $ deposited to 0 0 1 1
pay for 1Æ0 bit flip later

$ $ $ $ $
0 1 1 0 0 1 0 1 0 1 0 0
 Splay Trees
• Splay Trees: Self-Adjusting (balanced) Binary Search
Trees (Sleator and Tarjan, AT&T Bell 1984)

• They have O(logn) amortized run time for

– SplayInsert()
– SplaySearch()
– SplayDelete()
– Split()
• splits in 2 trees around an element
– Join() These are expensive
• joins two ordered trees operations for AVLs
 Splay Trees

• A splay tree has the binary search tree

property:

left subtree < parent < right subtree

• Operations are performed similar to BSTs. At

the end we always do a splay operation
 Splay Trees: Basic Operations
SplayInsert(x)
- insert x as in BST;
- splay(x);

SplaySearch(x) SplayDelete(x)
- search for x as - delete x as in BST;
in BST; if successful then
- if you locate x - splay() at successor
splay(x); or predecessor of x;
 Splay Trees: Basic Operations
• A splay operation moves an element to
the root through a sequence of zig, zig-zig,
and zig-zag rotation operations
– rotations preserve BST order

Splay(x)
- moves node x at
the root of the tree
perform zig, zig-zig, zig-zag
rotations until the element
becomes the root of the tree
Splay Trees: Splay(x) Operation

ZIG (1 rotation):
x
y

y
x zig
C A

A B B C
Splay Trees: Splay(x) Operation
ZIG-ZIG (2 rotations):

z
x

y y

D A z
x
C B
zig-zig

A B C D
Splay Trees: Splay(x) Operation
ZIG-ZAG (2 rotations):

z x

y zig-zag
y z
D
x
A A B C D

B C
 Splaying: Example
Splaying at a node splay(a):

k k
e
e
d d
G ZIG c G
c F ZIG-ZAG
F
b E a E
Aa A b
D B
B C C D
 Splaying: Example (cont.)

k a
e c k
a G e
ZIG-ZAG c d ZIG-ZIG
F A B G
b d
b F
A B E
C D E
C D
 Splaying: Homework

g
a
f
e f
H g
d A d
c
G splay(a) b e
F
b c G H
E
a B
D E F
why?
C
A B C D
 Splaying: Homework

g a
f
e splay(a)
H
A d g
f
c G d e
B b why? b c
a F A H
C B G
C D E F
D E
 Splaying

• We observe that the originally “unbalanced” splay

trees become more “balanced” after splaying

• In general, one can prove (see textbook) that it

costs 3logn $s to splay() at a node.

• Therefore, in an amortized sense, splay trees are

balanced trees.
Splay Trees: Join(T1,T2,x)
Join(T1, T2, x)
- every element in T1 is < T2
- x largest element (rightmost element) of T1
- it returns a tree containing x, T1 and T2
SplayMax(x); /* this splays max to root */
Splay(x);
right(x) = T2;

x
x

x
T1 T2
T1 T2 T1 T2
 Splay Trees: Split()
Split(T,x)
- it takes a single tree T and splits it
into two trees T1 and T2
- T1 contains x and elements of T smaller than x
- T2 contains elements of T larger than x
SplaySearch(x);
Splay(x); /* this brings x to root */
return left(x), x, right(x);

x
x

T T1 T2
T1 T2
 Splay Trees: Complexity
• The amortized complexity of the following is O(logn):
– SplayInsert()
– SplaySearch()
– SplayDelete()

• That is, in a sequence of n insert/search/delete operations on a splay

tree each operation takes O(logn) amortized time

• Therefore, Split() and Join() also take O(logn) amortized time

Split() and Join() CANNOT be done in O(logn) time with

other balanced tree structures such as AVL trees