Module 5: Online Algorithms and Competitive analysis

Dr. Chandra Prakash

Assistant Professor
Department of Computer Science and Engineering

slides credit: Marius Minea Dan Sheldon, Akshay Krishnamurthy, Andrew McGregor
 Module 4: Online Algorithms and Competitive
Analysis
• PART 5.1: Types of algorithms
• PART 5.2: Buy vs. rent
• PART 5.3: Lost Cow Problem
• PART 5.4: Secretary Problem
• PART 5.5: MTF list Problem
 Offline Algorithms

• An offline algorithm has a full information in advance so it can compute the

optimal strategy to maximize its profit (minimize its costs).
• An offline algorithm is one which is given the whole problem data from the
beginning and is required to output an answer which solves the problem at hand.
• Offline algorithms produce the optimal solution as it is given the complete input.
 Online Algorithm
• In computer science, Online Algorithms are algorithms that need to make decisions
without full knowledge of the input.
• full knowledge of the past but no (or partial) knowledge of the future.
• algorithms for settings where inputs or data is arriving over time, and we need to
make decisions on the fly, without knowing what will happen in the future
• A strategy which at each point in time decides what to do based only on past
information and with no (or inexact) knowledge about the future.
– Can process its input piece-by-piece in a serial fashion, i.e., in the order that the input is fed to the
algorithm, without having the entire input available from the start.
• Online algorithms may produce the results that are not optimal as it doesn’t have the
complete input.
– For this type of problem we will attempt to design algorithms that are competitive with the
optimum offline algorithm, the algorithm that has perfect knowledge of the future.
 Online algorithms
• An online algorithm responds to a sequence of service requests; each an
associated cost.
– For example, a web page replacement policy maintains pages in a cache, subject
to a sequence of access requests, with the cost of a web page request being zero
if the page is in the cache and one if the page is outside the cache.
• In an online setting, the algorithm must completely finish responding to
a service request before it can receive the next request in the sequence.
– Making decision in the face of uncertainty
– Decision are taken in such as way that we don’t regreat
• If an algorithm is given the entire sequence of service requests in
advance, it is said to be an offline algorithm.
 Online algorithms
• To analyze an online algorithm we often employ a competitive analysis, where
we compare a particular online algorithm A to an optimal offline algorithm, OPT.
• Given a particular sequence P = (Pi ,P2, . .. , p,) of service requests, let
– cost (A, P) denote the cost of A on P and
– cost(OPT, P) denote the cost of the optimal algorithm on P.
• The algorithm A is said to be c-competitive for P if
– cost (A ,P) <c.cost(OPT,P)+b,
– for some constant b≥0.
• If A is c-competitive for every sequence P, then we simply say that A Is c-
competitive, and we call c the competitive ratio of A.
• If b =0, then we say that the algorithm A has a strict competitive ratio of c.
 Competitive Analysis
• Competitive analysis is a method of analyzing online algorithms, in which the
performance of an online algorithm is compared to the performance of an optimal offline
algorithm.

• The competitive ratio of an algorithm, is defined as the worst-case ratio of its cost divided
by the optimal cost, over all possible inputs.

• The competitive ratio of an online problem is the best competitive ratio achieved by an
online algorithm.

• The competitive ratio of an online algorithm ALG is the worst case (i.e., maximum) over
possible futures σ of the ratio:
ALG(σ)/OPT(σ),
•where ALG(σ) represents the cost of ALG on σ and OPT(σ) is the least possible cost on
σ
Buy vs Rent Problem
Renter’s Dilema
Ski-Rental Problem
 Buy vs Rent Problem
• The Ski-Rental Problem

• Assume that you are taking ski lessons.

• After each lesson you decide (depending on how much you enjoy it, and what is your
bones status) whether to continue to ski or to stop totally.
• You have the choice of either renting skis for
• 1$ a time or
• buying skis for y$.

• Will you buy or rent?

 Buy vs Rent Problem

• The Ski-Rental Problem

• If you knew in advance how many times t you would ski in your life then the
choice of whether to rent or buy is simple.

• If you will ski more than y times then buy before you start, otherwise always rent.

• The cost of this algorithm is min(t, y).

• This type of strategy, with perfect knowledge of the future, is known as an offline
strategy.
 Buy vs Rent Problem

• In practice, you don't know how many times you will ski. What should you do?

• An online strategy will be a number k such that after renting k-1 times you will buy skis
(just before your kth visit).

• Claim: Setting k = y guarantees that you never pay more than twice the cost of the offline
strategy.

• Example: Assume y=7$ Thus, after 6 rents, you buy. Your total payment: 6+7=13$.
 Buy vs Rent Problem

• Competitive Ratio

• An on-line algorithm A is c-competitive if there is a constant b for all sequences s of

operations

• A(s) < c OPT(s) + b

where A(s) is the cost of A on the sequence s and OPT(s) is the optimal off-line cost for
the same sequence.

• Competitive ratio is a worst case bound.

 Buy vs Rent Problem
• Theorem: Setting k = y guarantees that you never pay more than twice the cost of the
offline strategy.

• Proof: when you buy skis in your kth visit, even if you quit right after this time, t ≥ y.

•Your total payment is k-1+y =2y-1.

•The offline cost is min(t, y) = y.
•The ratio is (2y-1)/y = 2-1/y.

• We say that this strategy is (2-1/y)-competitive

 Buy vs Rent Problem
•Say you are just starting to go skiing. You can either rent skis for $50 or buy them for $500.
You don’t know if you’ll enjoy skiing, so you decide to rent.

•What is competitive ratio of the algorithm that says “buy right away”?
•The worst case is we only go skiing once. Here the ratio is 500/50 = 10.

•What about the algorithm that says “Rent forever”?

•Now the worst case is that we keep going skiing. So the competitive ratio of this
algorithm is unbounded.

•Here’s a nice strategy: rent until you realize you should have bought, then buy. (In our case:
rent 9 times, then buy).
•Let’s call this algorithm better-late-than-never. Formally, if the rental cost is r and the
purchase cost is p then the algorithm is to rent ⌈p/r ⌉ − 1 times and then buy.
 The Ski-Rental Problem

The general rule:

•When balancing small incremental costs against a big one-time cost, you
want to delay spending the big cost until you have accumulated roughly
the same amount in small costs.
 Exercise

The Lost Cow Problem

Old McDonald lost his favorite cow. It was last seen marching towards a
junction leading to two infinite roads. None of the witnesses can say if the cow
picked the left or the right route.
 The Lost Cow Problem

Old McDonald’s algorithm:

1. d=1; current side = right

2. repeat:
i. Walk distance d on current side
ii. if find cow then exit
iii. else return to starting point
iv. d = 2d
v. Flip current side
 Exercise
• Free Pizza problem
• N rooms
• OPT:
– know where pizza party is, go to that room
• Visit : 1, -2, 4, -8, 16
• Let says pizza is in room t
• OPT : spend t times
• We spend time :
• (1+1) + 2+2) + (4+4) ……. + 2m + 2m) + t
 List update problem
• Move to first [ MTF] list algorithm
• n item in a linked list
• 3 operation
– Access (x)
– Insert (x)
– Delete (x)
• After accessing an item allowed to make it anywhere closer toi the
front of list
Secretary Problem
 Secretary Problem
• The secretary problem is one of many names for a famous problem of the optimal
stopping theory.

• The problem has been studied extensively in the fields of

• applied probability,
• statistics, and
• decision theory.

• It is also known as the marriage problem, the sultan's dowry problem, the best choice
problem, etc.
 Secretary Problem
• The Secretary Problem also known as marriage problem, the sultan’s dowry problem, and the best
choice problem is an example of Optimal Stopping Problem.

• Problem Statement: Imagine an administrator who wants to hire the best secretary out of n rankable
applicants for a position.

• The applicants are interviewed one by one in random order. A decision about each particular applicant
is to be made immediately after the interview.

• Once rejected, an applicant cannot be recalled.

• During the interview, the administrator can rank the applicant among all applicants interviewed so far,
but is unaware of the quality of yet unseen applicants.
 Continue…
• The question is about the optimal strategy (stopping rule) to maximize
the probability of selecting the best applicant.
 Optimal Stopping
• In mathematics, the theory of optimal stopping or early stopping is concerned with the
problem of choosing a time to take a particular action, in order to maximize an expected reward
or minimize an expected cost.

• If the decision to hire an applicant was to be taken in the end of interviewing all the n
candidates, a simple solution is to use maximum selection algorithm of tracking the running
maximum (and who achieved it), and selecting the overall maximum at the end.

• The difficult part in this problem is that the decision must be made immediately after
interviewing a candidate.
 1/e law of Optimal Strategy

• According to this strategy the optimal win probability is always at least 1/e.

• The optimal stopping rule prescribes always rejecting the first n/e applicants that are
interviewed (where e is the base of the natural logarithm and has the value 2.71828)

• Then stopping at the first applicant who is better than every applicant interviewed so far (or
continuing to the last applicant if this never occurs).

• This strategy is called the 1/e stopping rule because the probability to select the best candidate is
1/e, in other words this strategy selects the best candidate about 37% of time.
 Sample Space and Selection Space
• If you think carefully, it might seem obvious that one cannot select the first candidate because the first
candidate has no one to compare with.
• A better strategy is to chose few candidates as sample to set the benchmark for remaining candidates.
• So the sample will be rejected and will only be used for setting benchmark.

• If a sample is too small, we don’t get information enough for setting the benchmark for remaining
candidates.

• If a sample is too large, though we get plenty of information but we have also rejected too many of the
potential candidates. This leaves us with very few candidates to choose from, and hence making the strategy a
poor one.
 Continue…
• The best strategy is to choose the perfect or optimal sample size (ideal sample size) which can
be done using 1/e law that is rejecting n/e candidates (this n/e is the sample size).
The optimal sample size and Probability of success

• Optimal Sample size k = n / e

• Probability of success is given by :

• where x = k / n
Continue…(Prob. Of Success is aprox. 0.368)
Paging Algorithm
 Paging Algorithm

• In paging, we have a disk with N pages, and

fast memory with space for k < N pages.

• When a memory request is made, if the page

isn’t in the fast memory, we have a page fault.

• We then need to bring the page into the fast

memory and throw something else out if our
space is full.

• Our goal is to minimize the number of misses

 Paging Algorithms
• Data brought from slower memory into cache

RAM

CPU
 Paging Algorithms
• Data brought from slow memory into small fast
memory (cache) of size k
• Sequence of requests: equal size pages
• Hit: page in cache,
• Fault: page not in cache
 Minimizing Paging Faults
• On a fault evict a page from cache
• Paging algorithm ≡ Eviction policy

• Goal: minimize the number of page faults

 Worst case
• In the worst case page, the number of page faults on n
requests is n.

E.g. cache of size 4, request sequence

p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11 p12
 Difficult sequences
Cache of size 4, request sequence
p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11 p12

Sequence is difficult, for one it never repeats pages so it is

impossible to have a page hit
 Compare to optimal
p1 p2 p3 p4 p5 p6 p7 p8 p9 p10 p11 p12 p13 … is hard for
everyone (i.e. 13 faults)

p1 p2 p3 p4 p5 p1 p2 p3 p4 p5 p1 p2 p3 p4 … 8 faults

Optimal algorithm knows the future

 Offline optimum
Optimal algorithm knows the future, i.e. offline OPT.

Compare online paging strategy to offline paging strategy

 Competitive Ratio
Defined as :

cost of online algorithm on I

sup
I
cost of offline optimum on I
 Paging Algorithm

•
 Paging- Cache Replacement Policies
Problem Statement:
•There are two levels of memory:
– fast memory M1 consisting of k pages (cache)
– slow memory M2 consisting of n pages (k < n).
• Pages in M1 are a strict subset of the pages in M2.
• Pages are accessible only through M1 .
• Accessing a page contained in M1 has cost 0.
• When accessing a page not in M1, it must first be brought in from M2 at a cost of 1 before it can
be accessed. This event is called a page fault.
 Paging- Cache Replacement Policies

Problem Statement (cont.):

If M1 is full when a page fault occurs, some page in M1 must be evicted in order to make room in
M1.

How to choose a page to evict each time a page fault occurs in a way that minimizes the
total number of page faults over time?
Paging- An Optimal Offline Algorithm
Paging- An Optimal Offline Algorithm
Online Paging Algorithms
Online Paging Algorithms
Online Paging Algorithms
Paging- a bound for any deterministic online algorithm
Paging- a bound for any deterministic online algorithm
MTF List Problem
Typically when we solve a problem we assume that we know all the data a priori.
However, in many situations the input is only presented to us as we proceed.

Definition:
The competitive-ratio of algorithm A is CA if for any n > N0
and for any sequence Rn,
where c is independent of n.
Definition 1:
An online algorithm Aon is -competitive if for all input
sequences  C Aon (  )    C OPT (  )

where: COPT is the cost of the optimal offline algorithm

C Aon is the cost of the online strategy

In order to evaluate the online strategy we will compare its performance with that of
the best offline algorithm.
This is also called competitive analysis.
Definition 2:
An online algorithm Aon is -competitive if for all input
sequences 
C Aon (  )    C OPT (  )  c

where:COPT is the cost of the optimal offline algorithm

C Aon is the cost of the online strategy

c is some constant independent of 

 The List Accessing Problem

Definition
Input: linked list x1 x2  xl
a sequence I of requested accesses I  1 2   n
where i,  i  x1 , x2 ,  , x. l 
The cost of accessing  i is the location of the item in the list counted from
the front.
Given I (online), our objective is to minimize the cost of accessing the items in
the list
 The List Accessing Problem
• While processing the accesses we can modify the list in two ways:

• free transpositions: after an access, the requested item may be moved

at no cost closer to the front of the list.

• paid transpositions: at any time we can swap two adjacent list items
at a cost of 1.
 Deterministic Online Algorithms
Move-To-Front (MTF)
Move the requested item to the front of the list.

Transpose (TRANS)
Exchange the requested item with the immediately preceding item in the list

Frequency-Count (FC)
Maintain a frequency count for each item in the list. Items are stored in non-
decreasing order of accesses. After item is accessed its frequency counter is
updated and item moved forward (if necessary) to maintain list order.
We will prove the following two facts:

Theorem 1:
The Move-To-Front algorithm is 2-competitive. c 2

Theorem 2:
Let A be a deterministic online algorithm for the List Accessing Problem. If A is c-
competitive, then .
Pay attention to the fact that in theorem 2 we prove a lower bound to the
competitiveness.
Proof 1:
Definitions: The potential function  : For anyt , 1 t n
(t) = The number of inversions in Move-To-
t
Front’s list with respect to OPT’s list, after
is served.
An inversion is a pair x,y of items such that x
occurs before y in Move-To-Front’s list and
after y in OPT’s list.

For any 1 t n
C MTF ( t ), C OPT ( t ) actual cost incurred

by MTF and OPT in serving  t

Move-To-Front and OPT start with the same list, so the initial potential is 0.

We will show that for any t

CMTF (t )  (t )  (t  1) 2COPT (t )  1 
n n

then C
t 1
MTF (t )  (n)  (0)  2COPT (t )  n
t 1

 CMTF ( I ) 2COPT ( I )  n  (0)  (n)

and because (0) 0, (n) 0 t

the theorem follows.

The amortized cost incurred by Move-To-Front on is defined as :
CMTF (t )   (t )   (t  1)
We will show inequality (*) For an arbitrary t.
Let: x = the item requested by  t .
k = number of items before x in MTF’s and OPT’s list
l = number of items before x in MTF’s list but follow x
in OPT’s list.
 C MTF (t ) k  l  1
COPT (t ) k  1

When MTF serve  t and moves x to the front of the list, l

inversions are destroyed and at most k new inversions are
created. CMTF (t )   (t )   (t  1) CMTF (t )  k  l 2k  1
Thus 2COPT (t )  1
Proof 2:
Consider a list of l items. n requests in I.
We construct a “bad” request sequence for A with cost C A ( I ) n l

Let OPT be the optimum static offline algorithm. OPT first sorts the
items in the list in order of nonincreasing request frequencies and
then serves I without making any exchanges.

If the list is sorted by request frequencies, the worst case is that all
frequencies are n/l (then we didn’t gain anything from sorting).
Thus accesses costs: l l
l 1

i 1
i f i i 1
i n
l n 
2
We can take instead of OPT the static offline algorithm because we prove a lower
bound.
c OPT c STATIC  A
Each request is made to the item that is stored at the last position in A’s list. n requests,
each will cause cost l, lead us to the cost nl.

If the frequencies are not equal the cost will be lower, because then we’ll put the more
frequent items closer to the beginning, causing more cheap accesses and less expensive
accesses.
Rearranging the list cost at most l(l-1)/2. Then the requests in I can be served at a
cost of at most n(l+1)/2.
Thus
l2 l 1 l 2  l 1 
COPT ( I ) n(l  1) / 2  l (l  1) / 2   n   nl  
2 2 2  2l 
3
 2  l
 2  COPT ( I )  nl C A ( I )
 l 1  l 1
 c 2  l 21 .

The theorem follows because the competitive ratio must hold for all list lengths.
 Reference

• CSE 521 - Algorithms On-line Algorithms, Tami Tamir

https://courses.cs.washington.edu/courses/cse521/03sp/online1.prn
.pdf