Chapter 9

Limitations of Algorithm Power

Lower-Bound Arguments
Lower bounds arguments estimate the minimum amount of time or resources required to
solve a given problem.

Examples:
• sorting: lower bound (n log n)
• searching in a sorted array: lower bound (log n)
• multiplication of n × n matrices: lower bound (n2)

There are a number of methods that can be used to establish lower bounds
• Trivial lower bounds
• Information-theoretic arguments
• Adversary arguments
• Problem reduction
Trivial Lower Bounds
Trivial lower bounds are based on counting the number of items that must be
processed in input and generated as output to solve a problem.
Examples:
 Sorting Algorithm
 Searching Algorithm
 Graph Traversal Algorithm
 Matrix Multiplication
Information-Theoretic Arguments
Information-theoretic lower bound is based on the amount of information an algorithm needs to
produce to achieve its solution.
Adversary Arguments
Adversary argument method is a way to estimate the minimum amount of
computational effort that an algorithm would need to solve a problem.
Example
 Finding the maximum element
 Searching for an element in a sorted list
Problem Reduction
Problem reduction method is a technique used to obtain lower bounds on the number of
operations required to solve a problem.
Examples:

 Sorting problem
 Graph Connectivity problem
 Decision Trees

A decision tree is a powerful model that can be used to represent a sequence of decisions made by
an algorithm in the form of a tree.
A decision tree is composed of nodes and edges. Each internal node represents a decision. The
edges coming out of a node correspond to the possible outcomes of the decision. A leaf node
represents an outcome or a solution.
Structure of Decision tree
Root Node: The root node is the starting point of a tree.
Internal Nodes: Each internal node represents a decision point.
Leaf / Terminal Nodes: Leaf nodes represent the final class of the outcome.
Branches: Branches are connections between nodes. Each branch represents a response such as yes
or no.
Decision Trees for Sorting
Example 1
Most sorting algorithms are comparison based. Consider an example of selection
sorting for three elements in a list using decision tree.

The worst-case number of comparisons for selection sort is 𝒏𝒍𝒐𝒈𝟐 𝒏

Example 2

Consider an example of insertion sorting for three elements in a list using decision tree.
 The worst-case number of comparisons for insertion sort is 𝒏𝒍𝒐𝒈𝟐 𝒏

Decision Trees for Searching a Sorted Array

Decision trees can also used for searching an ordered array of n keys.

The wort case comparison requires 𝒍𝒐𝒈𝟐 (𝒏 + 𝟏)

P, NP AND NP COMPLETE PROBLEMS

Computational complexity refers to the amount of resources, such as time and memory
required to solve a problem using an algorithm. P, NP, NP-Complete are classifications
used to categorize problems based on their computational complexity.

A problem is solvable in polynomial time if its worst- case time efficiency belongs to
O(p(n)) where p(n) is a polynomial of the problem’s input size n.

Example 𝑶(𝒏), 𝑶(𝒏𝟐 ), 𝑶(𝒏 𝒍𝒐𝒈 𝒏) are polynomial time efficiencies.

Problems that can be solved in polynomial time are called tractable, and problems that
cannot be solved in polynomial time are called intractable.
 Class P Problems

DEFINITION: Class P is a class of decision problems that can be solved in polynomial

time by (deterministic) algorithms. This class of problems is called polynomial.
Example
1. Sorting Problem
2. Matrix Multiplication
3. Searching in a sorted array
The Class NP Problem

DEFINITION: Class NP is the class of decision problems that can be solved by non-
deterministic polynomial algorithms. This class of problems is called nondeterministic
polynomial.
Most decision problems are in NP
P ⊆ NP
Example
1. Travelling Salesman Problem
2. Knapsack Problem

NP-Complete Problems

DEFINITION: A decision problem D is said to be NP Complete if

 It belongs to class NP
 Every problem in NP is polynomial reducible to D.
 1

CHnprrR -6
Bctck bvac k NA d

d
PugF:,"3,
ktmclcinq is ah alaoa;tF,nte tpthn q,r,
Brc
U
ldnwo +'F,t ts t, a@* oltA sot,rlru,r, h '*
N .tte
r"b rD usina
[,a: huh *** q.ytptae h, it brtirb
b,nldr'"f a- s0* d'rotuk'o>r ircfizru::tb
% %'ott
gLnca o.
prJtUn w*uA ha ,u ur>*lroink , 'tFt I of."lfu'v
ro Pi [l ba t*r'P"eA
#at baiL t Salttfil tf,p '

8"v{' ,ol wllwd

Fat,F4etta'*c '

ilto a \oW'or>t A^{

LL) probbw S oarrek' So.t-

AE
1
U,L
%
ofirmal so lrrh". caD La
u;Mcl,
ryP tf{ baek{roe k i neM,
Solved wt "a 1
1ii) ao *PPU b"ck.lrac k ilta+frwd , #^e goltt*tsr, mtx|
(x,-. - - z..r)
bo LYA*< fblt n- tuPLe .1

a,s Arf)

ta)hero tf,s %i a.^e c.lwren fi** Eomz fiit,

SaV St

Ui;) =II-rE Sotu,tton vauho nwh salfrbff Ure cwtavtoo

fi;.,,,ch'on
?(*,-- xn)
-J
"lo r An) arobbm tt*gz bx4 ralw{r c-ar-' bo al;uidad inta
I

*no -A fiorlos ,

r. Pxpltc|b
A.?-nplictL
 Fx ['ei L br*1r6
t*1&@J
n,l-r

F*ptidt- Lonr{rarnFr a^z ftillL\ thaf fushtuV oach

'tLi bo bkz @f) ud.u! g
,"3 bvam
&a' vLn aa*
n'mh$
Uo*fu. i >o O,l S $ al.t Don rcyohue- toal'
1L
L

Xi=a@ft or S i=[,o,t )
hll "lupb, +T,at Bah\ - +11, ULP lr'eib u*4 tal nb
bl
4,,* ar' Po4n
DblL sobt}i'oo oru b* t"
L
I

F xanpb,

Ik A - @uu-n PTO
bten , *fv oltntnh "fi lf'e laPlz
rrxrst h ,50,b 8= \,rrr3,4
1lasen
F*tu 1

f\r#roPt, cr h cerx'l TAn

TFu i mPl,. oil: urx,lmirrh a/n nu|u *f@t dt'h-w'tne

t,,lhich -t* la,Pt, in 'tfz
% Jolwh'on "b "t '
'l*<
Sancgc1 ttt crf toxion fi.ew-tforv
h: 'tf,e imPlici\ esor|""ith an
F C'LATO Un a - ewo.n yobbm,
(il No lwo 7,** ca,t ba- @D 'tfe Barru eaLwnn
1ii) Alo 4wo ?** ca.h La o n +fr, Jame rtaXo'nL

trornrw
I
I 03

Sola[ie" ,s pato : ylll f/,c 4qDbs wlv'eh s ohY1il +fr'

wPbc.Ot @r,<'ftatnh &fr'* +f,,e ,sohrifeb sP* P

4fi"k inslanw I 'tr-,e lxobbn .

I
 3

f3o,ud; dr unc f.o"rs : -lAyS

^ ,3 a cfiwndfon wh;ch is
ff-
used yNavonl- u,x-yarnrio n ,$ bo&! +fr,ak earvp l-
Po%ibtl lr^d tb Ah Ah| u..tOs rcde

l-iro-\A.lrA*J
q+r.A. nodr 't*fls ls a b@1, wl,tc h ho.s bon aat eryulnd
at,d qtl uheg c hil&^en &ra r>ok u oL qo-{rr*ha
"fi J (l
rwdt /, ltvo tuod! u) bkc ln/d,tro ate
*n Xo,*""t
d is ealbd tf-rn d,
cu*rontfy

'Dooal
:a.\,1.ms Nr-d,
: fi hrolo ot-rena,lad is ol{F,ev t>ot b" exfdrd
a
$u.v{fvv sl flfie ohE
il* wln'el, a,1/ 'tt, cl, tdtzo herrye

bun aao,rcratLd
fldor"li Y&
E**l EEIfsHs0 NPi
14 It *olttlisrx dfr' f as-'eraro/ in x ( r:n) *rd
a,g Sson aA 'rTreU a^2 aln'fernirvd 7() bill .

fytrilLd
T)td tt,( ao* %I" p*-itu val'at
blaor i+fr,ro : {Z ac k trac k ( n)
q0
k =l
wl"ib ( t, L =o ) alu
{
if ( +t*.," t*tnai >t etn untriod *( x1 e{*t,l,vtrJ
' '. . :c[x ',y and B1r(xtil" xtil)=/,,ue)ll*o
t if, ( xtti -- - xlNi) o'WO
ts a W lo

lxin{:
( xl,I zt rl )
k =k*r
L
J,
8-W

!9 fu=k-t
I
 h

-l:'y- alqerttfrm dnscnib. if," back,lracki

$ottoi'T
hzcur{s1 o n
0 u ?roczu
Wgit\6

l) oritfrm : R Bact<lRnckCK)

i +r, ( uch eL(k) eb {l, .... xCu-,1)

r
"t,
1 xtrl) t=6)+tn
b ( Bx{r,, d,i,-.-.
.(
r.) (xr,l, xlx")) is ^
W +o

a,h ar>sww rwol,a tf''zo

yinb (rf ,J,- - . xt*l)
( r."
,h
") lfun
call R,BacxrrR ne A, ( Xu)
h
\
v

$- @,.,ten
E&..F*d\e'se,
Pnobbn
r.*#i..rt#i*.,

SbdhrnenF :
Yrr*'*frdfsr#

Tfu
cksa bsat d
r tfret
SO
obbn ,'J

ho
to plo*
lwo
A-
uze.b
?webs
att
on hvtt
c* oach slfe,
?
bu ba h ti,( gaff t t@a) OT in +f',r ?atfie celumn ot
1
0n +fN Samt olryorul

LvM Lo rx4rai

-lfu axphcrt urx'lra.tnts ak S = 1 tr L, 3, tr\

+\taJil,es
%
5 *r ra.oe,nk cohrmo rn mboo
 )

9* t,,'ci
u l- C-o
r-y--!
rx.lra, nh
-J
--*
:

-l$u, 1ff) ptrcit brx:,lrair,fr Ao ,

l. Irlo
L- Alo
4rpo

4wo
1* c-a,r) ho en +fP Sa,tw totD.
bu oh lfrt amt galumn
3 Alb
7*'^ Can S .

lwo
?t/!t
t>r eQ/h L" on +fx s aw dr'oqoml
a
..Il.g cbl.x b"*d
b* +1"e.
9'* 1/* h },ueo
b lam rs
Sk.rrn be-ln*,

3h
@,,run I

L <- @,raen L
3 @,.,un 3
tl
I <- @uu" *

W \ r, , x> )%s,xnl L e 4ff 3 ohtlion vocta r ,^tl*ro-

-tf€ ?* I lauel ,

x- CoUrr{r^ on wlneh '' I

t'
in fri ,rs t- fiP$ ^"d
lkk firs& 1/t!2b
catwon
ig PlP,ce€il

fr"1
t2
I Q,
L
3
4

I
 L

ltoa and **F ' c-rlrrnn. 7t shold w[-

ba placeol 'io sacod 6o us a'hd sorcnd
ulunn .( ol;oyon"ry/ ) y>taeo O' in
gword frorrr" 4f$rd to lonn '
t L 3
,
7@,
I
L >a XlQrl
3 )

Step z
Ada-6+."e
-rk tl*r ol ?ueon shoutd >oF ba in 'tfr''d
*pu P *t*n($o* @l,,mo @, * @=), +fn"d
ftr,rs
cD;, ( Az * 4s olingo'-lt1 oae| drA'
$rs A sLcond
4fitrd cpLumn ( e,L @g sa'ntt e'L*n)
lkrd *oa
, cofo*n ( 64, L@z ar"X orn, ,y
4krd Soul
fieurlB
e+fru, ) E, back t raak irl ond
0-qoh
st;gt Qt obu.
I L 3
't 2-3
,
&r @'
I
L
L L Ar)
3 3 -3

h 4

Lg lo*n
-Tfnn plaw Gs in' #*'ol frsu: s o+n"'l .

l>
I

2
3 Q* -l
q
 7

shP tt fln 6tann ahould rwl- be in to^#

$out+B
Aoo gtvs I erlo^n ( 4 , P @rt at& in clatat eet'*u') '
(
flsw aaeord col.crmn Og
I6q aA'e ln *a*e
$our*s q+
*Fnrrl l colomn ( eg+
eolnnn) , b
p uv'tf, rror,t2
NV in ,lro6o rld tu) t furln $ow 4
t'**
Wh*rr'n I @L ) @e a''ve in $arvle *to*")

, , Do BacHtack 1
@ t is skSad oncz
r LJ l> 3
I

L
& i
lt I >< @,

3 3
h a1 A

stw 5 -rk secend @,ruon should noF b' in

gotond 6oct) rfii'*'l cuLumn ( @r z@> oa;*6t )

Second 6o ets eourd .rh*n ( q,rq>*sarrte :h]

Saconal $ o us 4hivd eahunn (o, Yq> &7"*% )
in sotcT '
n' n 1c{au @ L $euv*n "n

I z 3
I

1
3
lr

SLap 6 -Tk tfn, d @wrn ' WL ear, 1r/" e" in

,tfu,d ffous

I
f ,rst La lunn

z 6>
3 'qt
4
 I
Sh 7 pl aw @a in $frmtB
soat tf,trd cok*n
bet *r* ) rtbffi fiso r rs F col,,mn
(e* [ Q,]
In d,ame Cetnnn ) , u) sorpld ee lurt, n
$o", tf''
cro

Co, * @,, ; D 6aw colo*n)

)L
,
@, L y=L
I

6, xz= h
3 33 !C3= I
--l q-
l

h j x+ =3

-Tk soluhon vachry f *, ,xy ,Ls,n)

Cr,h )
=J
rt o*f*r Pe$;bb aolu,ltan is
al 6o
llo,r>s ,

@r 2(1 =3
L @z x>z I
3 6* xz=h
ll &,, 2t4=L

qI- -goh.r,flon vwhr =f x, txvtxL,nrt)

= [: h,LJ

T
 +fre *i rs I s alr4ion is I hoon
q
flu e'lafr .e
lau 4ras fio4
bdntt'

'Xl= I
Xr L.

(4r

r
tl
r{ x
N
Ir ]

_llo
4,1 I
I

/, 0,
1
, 6'i

IL
t

a/, I
+
N * t1
It

L?i i

Az
(*,
I

Sob.knn '
G
 l0

3--eggl*- .?xb,k*-,

S be,Jnrm nh ,
-4Nb,#

Dtobts-ro t s bo ?6* A - @uu-r"t o^ Ex(

lfr/
aecl\ atkr
I

c [ve bo*"d 8o #rah rw 4rrro aLf,ac L<

7*o*
b*ru rn tfrL &rrte fr&a, ox in tfrz 3 a rrL{ C'o lan
n

\
0"{ @n +fre (tulw duV" r*t

fx c-) b CorNt
e*,r44.#
rai nh :

-ITrr. e-x-pL;etb r*inh a)\L S*4tt2t3,h,sr6,7'{1

eorvt4

bo.^^" i t irwlr'cates cpbno *rrv,ban

'I nl UuV C""<|-va;rk t

ti) At" *
lA ro eyu-tLM (4,h b* @n tfr" Saxtr rsa)
t
+
t'd) Al, Itwo q** czh b tk BN *a'trw coutril)

[ii;;
L
ca.h L* oil 4f" dr"g*t
Ato w)b
1y^ Same

h &?1 Lhm

trl 0rcerx (r,n)

q .f-0, f =r to n A"
q\ ,b U ,i ) tl'en
Pla&
{
,LLR') = t )
J{

i6 (n=n) +f"n
vovib (n C t : hJ ).,
o-lse
N Quoun s ( xrr , n) ;
3
\
3
 II

n oniffi no : yl"c"(w,i )
4
f, J= | b, b-t d.
r5 txtjl ,
I
)

fr!,f,r!* QlaLx ;

i6 (*u- (,ot il-i)= Ak[;-k))

n -l-
\2-W,a,
,1
WW)r,L
Lt
Wrru-
il"b )
)
]
g - b u-u-r) nor
fr"kE""-g $* fubh
I floL*tio n { A,g,*,L,1 ,I,z,s\
12.3 Y a*i
t

L 6r
3 6a
A &.,
, txc
b 6t
T 6r
g q(

R. Sohhon f r,1, h,6,Q,L,5'3)

r2 5
I
i @, I
?, @a

3 o3
It n
4
114

5 6
I
I
L"tg

b
-:
@o
"7

g
t- 6,
G,g
 IA

.f", w S,r,bsz
q\,
4,t, , ' ' ' ' sn 1%-n- f rti Fi
va
GSirnn &tuts
inbo6zx aur d (\
P
ssifir,'u i n h*T- -Tk ytobbn t3
.g
.h all a*b+nha "fr tff
tr
ven I ub w bo*e 3 uro
bt'd
h tl,
is
%d
f**wh-'
tri 5={ ll., 13,7+,'1 ) o.nd C = 3t. 'TF,,n &*lwo)
S rrbs,e ft ard L
{", tz,1\ ano) 4'+ ,t \
{, Solvo 'tf*s y, bltm LL{,,7 ' bnc klroc ki ntr:

I wdan
r, 4irsl wu &ott lt'e vo'lnos t r) in evohq
T
,

. s ={ 1, u, t3 ,zlrl
4

tj

iiJ
X
XX
X xX
+ x- rr#;cats d!^d wdto
r,
l" so hr,F, , ',/ uLW "1, lt', tB o'{ ,lr l) l, o j
\
p, . sol^rFton vo-ch"r z{ '/, ah3 o? \r,D,D,r\
 r8

tltqo, i-fqgi J'rn$a"bsoh ( i, ,-flhb , bobat )

r_-ff
i1 ( ptonn ts i2 ( 5,'v,tdghb, hL*t))
q
itr (r* 0
wvitu (x-tt , ;J );
o-lAz
q
xft+ rl =- bru-l. ;
Sumefis ubs<fs ( i+ r ' rasot\1hL+,9[i+lJ ,

:tol* I * s[ it r]) ;
sIi+J--fftu,
S**"bSu,bszFI
e
i+1, **A hb,bob"l -sfi+,J);
\
)

Rutarn( **0 h'L' + bDbal ;= Fl ) P& (r,u dghb ==N

ll *"i0ha + 3[i+fl ,"= l*l);

F *(o:tcik. '. I

LuV S=4g,/+,SrL), lv/ = rB

fxuvcix i pl
^r.frd*\'^".:&*ffi

l-o.b s=4f,,t0 1 1l', tB, t5)tg 3, Ivl =sa,D=6

 Hamiltonian Circuit Problem
A Hamiltonian Circuit is a path in a graph that visits every vertex exactly once and returns to
the starting vertex. It is also called as Hamiltonian Cycle. If such cycle exists, the graph is said
to be a Hamiltonian graph.
Algorithm
1. Select any point as the starting point
2. Add the starting point to the Hamiltonian path
3. For each vertex ‘v’ in the graph
a. If ‘v’ is not in the path and it is adjacent to the last node in the path, then add ‘v’ to the
path
b. If no vertex can be added to the path, then backtrack and remove the last vertex
in the path
c. If all vertices are in path and the last vertex is adjacent to the first vertex in the
path, then we found a Hamiltonian cycle
4. Repeat step 3 until all vertices have been visited.
Example
Let us illustrate the Hamiltonian circuit problem with an example graph shown below

Step 1 Draw the state space tree.

Step 2 The root node has three adjacent vertices 2,3,5. We select 2, it has two adjacent vertices 5
and 6. We proceed to 5, it has 2 two adjacent 3 and 4. The vertex 3 has one unvisited adjacent
vertex that is 4. The vertex 4 has one unvisited adjacent vertex 6. The vertex 6 has adjacent
vertices 2 and 4, since both of those nodes are already visited therefore node 6 is dead end. Now
backtrack from 6 4, 43 and 35
Step 3
The solution vectors are:
1. 1 264351
2. 1 264531
3. 1 346251
4. 1 354621
5. 1 526431
Complexity
Time Complexity: O(n!)
Space Complexity: O(n)

Branch and Bound

Definition
Branch and bound technique is used to solve optimization problem. It involves dividing the
search space of a problem into smaller sub-problems and using bounds to eliminate sub-problem
that cannot contain the optimal solution.
Example
• Travelling Salesman Problem
• Knapsack problem
• Assignment Problem

0/1 Knapsack Problem

• Given n objects and a knapsack of size M
• Each object associated with a profit 𝒑𝒊 and a weight 𝒘𝒊 .
• The total capacity cannot exceed M at any time.
• The problem can be mathematically stated as
𝑛

𝑀𝑎𝑥 ∑ 𝑝𝑖 𝑥𝑖
𝑖=1
𝑛

𝑆𝑢𝑏𝑗𝑒𝑐𝑡 𝑡𝑜 ∑ 𝑤𝑖 𝑥𝑖 ≤ 𝑀
𝑖=1
Where 𝑥𝑖 𝜖 {0,1},1 ≤i ≤n

Steps to solve knapsack problem using Branch and bound technique:

1. Sort all the objects based on their profit value to weight ratio in decreasing order
2. Initialize the root node which represents the empty knapsack
3. Calculate the Upper Bound
𝑼𝑩 = 𝒑 + (𝑴 − 𝒘)(𝒑𝒊+𝟏 /𝒘𝒊+𝟏 )
4. Each node creates two branches, one includes the current object and the other exclude the
current object.
5. For each node calculate the upper bound, if the upper bound is less than the maximum
profit then continue otherwise don’t explore the node
6. Repeat step 3 to 5
Example
Consider the knapsack problem of finding optimal solution where M=40, N=4
[𝑤1 , 𝑤2 , 𝑤3 , 𝑤4 ] = [20,25,10,15]
[𝑝1 , 𝑝2 , 𝑝3 , 𝑝4 ] = [20,40,35,45]
Solution:
𝑝
Calculate 𝑤𝑖 for all objects and arrange in descending order
𝑖

𝑝1 20 𝑝2 40 𝑝3 35 𝑝4 45
= =1, = = 1.6, = = 3.5, = =3
𝑤1 20 𝑤2 25 𝑤3 10 𝑤4 15

𝒑𝒊
Object Weight Profit
𝒘𝒊

1 10 35 3.5
2 15 45 3.0
3 25 40 1.6
4 20 20 1.0

Root Node
Create the root node. Initially w=0, p=0
UB=0+(40-0)*3.5=140

Node 1
Object 1 is included therefore w=10,p=35
UB=35+(40-10)*3.0 =125
Node 2
Object 1 is excluded therefore w=0,p=0
UB=0+(40-0)*3.0 =120

Likewise, explore all the nodes one by one

Node 3
Object 2 is included therefore w=25ie.,(10+15),p=80 ie., (35+45)
UB=80+(40-25)*1.6 =104
Node 4
Object 2 is excluded therefore w=10,p=35
UB=35+(40-10)*1.6 =83
Node 5
Object 2 is included therefore w=15, p=45
UB=45+(40-15)*1.6 =85
Node 6
Object 2 is excluded therefore w=0,p=0
UB=0+(40-0)*1.6 =64

By considering node 3,4,5 and 6 the highest upper bound is 104 therefore we explore node 3
Node 7
Object 3 is included therefore w=50 ie.,(10+15+25),p=120 ie., (35+45+40)
Weight is greater than knapsack therefore Not feasible
Node 8
Object 3 is excluded therefore w=25, ie.,(10+15),p=80 ie., (35+45)
UB=80+(40-25)*1 =90

By considering node 7 and 8 we explore only node 8 because node 7 is not feasible.
Node 9
Object 4 is included therefore w=45 ie.,(10+15+20),p=100 ie., (35+45+20)
Weight is greater than knapsack therefore Not feasible
Node 10
Object 4 is excluded therefore w=25ie.,(10+15),p=80 ie., (35+45)
UB=80+(40-25)*0 =80
Node 3 is already explored in second level of the tree, therefore by considering the remaining
nodes (4,5,6) explore Node 5 because the highest upper bound is 85
Node 11
Object 3 is included therefore w=40 ie.,(15+25),p=85 ie., (45+40)
UB=85+(40-40)*0 =85
Node 12
Object 3 is excluded therefore w=15, p=45
UB=45+(40-15)*1 =70

By considering node 11 and 12 we explore only node 11 because UB is high.

Node 13
Object 4 is included therefore w=45 ie.,(15+25+20),p=100 ie., (45+40+20)
Weight is greater than knapsack therefore Not feasible
Node 14
Object 4 is excluded therefore w=40ie.,(15+25),p=85 ie., (45+40)
UB=85+(40-40)*0 =85

Hence node 14 becomes the final node

The optimal subset={2,3}
The profit earned =85

Complexity
Time Complexity 𝑂(2𝑛 )
Space Complexity 𝑂(2𝑛 )