MEMORY

HIERARCHY
 NEED FOR AN IDEAL MEMORY

● Computers employ different types of memory

(semi-conductor,magnetic disks,USB sticks etc.)
to hold data and instructions .
● The execution speed of programs is dependent

directly on the speed with which data and

instruction is transferred between the processor
and the memory.
● Computer programmers have always wanted
unlimited amounts of fast memory.

● An ideal memory is the one which is

FAST,LARGE and INEXPENSIVE.

● But this is not practically possible !!!

● Let's take the following case for instance...

MEMORY PERFORMANCE IMPACT ON
PERFORMANCE
 Suppose a processor executes at

ideal CPI = 1.1
● 5% arith/logic, 30% ld/st, 20% control and that 10% of
data memory operations miss with a 50 cycle miss
penalty
CPI = ideal CPI + average stalls per instruction
= 1.1(cycle) + ( 0.30 (datamemops/instr)
x 0.10 (miss/datamemop) x 50 (cycle/miss) )
= 1.1 cycle + 1.5 cycle = 2.6
so 58% of the time the processor is stalled waiting for
memory!
 INTRODUCTION
● FAST,LARGE AND INEXPENSIVE MEMORY IS
IDEAL.
● BUT SPEED OF MEMORY ACCESS IS

INVERSELY PROPORTIONAL TO SIZE

ECONOMICAL SOLUTION??
MEMORY HIERARCHY
● The most economical solution that the scientists

thought that could help them in providing an

illusion of an unlimited fast memory is MEMORY
HIERARCHY.
WHAT IS MEMORY HIERARCHY?
● Arranging different memories at different levels of the
computer.

● Each vary in their speed, size, and cost.

● Since, fast memory is expensive,a memory hierarchy is

organized into several levels-each smaller,faster and
more expensive per byte than the next lower level
GOAL OF MEMORY HIERARCHY

● To provide a memory system with cost per byte

almost as low as the cheapest level of memory
and speed almost as fast as the fastest level!!!
 CHARACTERISTICS OF MEMORY
HIERARCHY
Processor
Inclusive– what
4-8 bytes (word) is in L1$ is a
subset of what
Increasing L1$ is in L2$ is a
distance 8-32 bytes (block) subset of what
from the is in MM that is
L2$
processor in a subset of is
access time 1 to 4 blocks in SM
Main Memory
1,024+ bytes (disk sector = page)

Secondary Memory

(Relative) size of the memory at each level

●The cache is the closest to the processor and
forms the upper level of the memory hierarchy.

● As we go far away from the processor the

memory becomes larger, slower and
inexpensive!!
 THE 3 PRINCIPLES!!!

● MAKE THE COMMON CASE FAST

● SMALLER IS FASTER
● THE PRINCIPLE OF LOCALITY
 THE PURPOSE OF CACHE
CACHES STORE FREQUENTLY ACCESSED
DATA.
THE CPU HAS FASTEST ACCESS TO THE
CACHE. (MAKING THE COMMON CASE
FAST)
CACHES ARE SMALL IN SIZE BUT HAVE
VERY LITTLE ACCESS TIME (SMALLER IS
FASTER).
THIS IS BECAUSE THE DESIGNER CAN USE
MORE POWER PER MEMORY CELL.
 PRINCIPLE OF LOCALITY
● THE MOST IMPORTANT PROGRAM
PROPERTY THAT WE REGULARLY EXPLOIT
IS LOCALITY OF REFERENCES.
● A PROGRAM SPENDS MOST OF ITS TIME

RE-USING THE DATA AND INSTRUCTIONS

THAT IT HAS USED RECENTLY!
THE 90/10 RULE
A program spends 90% of it's time in 10% of it's
code!
 2 TYPES OF LOCALITY

● TEMPORAL LOCALITY: states that recently

accessed items are likely to be accessed in the
near future
eg:-
for(i=0;i<5;i++)
sum+=i;
(Here the same variable 'sum' is accessed again
and again)
●SPATIAL LOCALITY:states that items whose
addresses are near one another tend to be
referenced close together in time.
Eg:-
for(i=0;i<10;i++)
a[i]+=5;
(Here the the consecutive memory locations are
accessed)
 TODAY, THERE ARE THREE PRIMARY
TECHNOLOGIES USED IN BUILDING
MEMORY HIERARCHIES

● THE FASTER MEMORIES ARE MORE

EXPENSIVE PER BIT THAN THE SLOWER
MEMORIES AND THUS SMALLER .
DRAM
SRAM
MAGNETIC DISK
 DRAM AND SRAM
● Main Memory uses DRAM for size (density)
Low power, cheap, slow
●

Dynamic: needs to be “refreshed” regularly (~ every 8

ms)
● Caches use SRAM for speed and technology

compatibility
Low density (6 transistor cells), high power,
expensive, fast
Static: content will last “forever” (until power
turned off)
CERTAIN IMPORTANT METRICS...
● HIT- If the data requested by the processor
appears in some block of the upper level, then it's
called a hit.
● MISS- If the data requested is not found in the

upper level and the lower level in the hierarchy is

accessed to retrieve the block containing the
requested data,the it's called a miss.
● HIT TIME- The time to access the upper level of

the memory hierarchy , including the time needed

to determine whether the access is a hit or a miss.
● MISS PENALTY- The time to replace a block in

the upper level from the lower level ,plus the time
to deliver this block to the processor.