MODULE 5

CACHE MEMORY – MAPPING FUNCTIONS – REPL ACEMENT ALGORITHMS ,

VIRTUAL MEMORY – PAGING AND SEGMENTATION.
 CACHE MEMORY

 Processor is much faster than the main memory.

o As a result, the processor has to spend much of its time waiting while
instructions and data are being fetched from the main memory.
o Major obstacle towards achieving good performance.
 Speed of the main memory cannot be increased beyond a certain point .
 The cache is a small and very fast memory, interposed between the
processor and the main memory.
 Its purpose is to make the main memory appear to the processor to be
much faster than it actually is.
 LOCALITY OF REFERENCE
 Efficiency of cache memory is based on the property of computer
programs known as “locality of reference”.

o Analysis of programs indicates that many instructions in

localized areas of a program are executed repeatedly, while
the others are accessed relatively less frequently.
o These instructions may constitute a simple loop, nested loops,
or a few procedures that repeatedly call each other.

 This is called “locality of reference”.

 LOCALITY OF REFERENCE
Temporal locality of reference:
 Recently executed instruction is likely to be executed again very soon.
 Temporal locality suggests that whenever an information item,
instruction or data, is first needed, this item should be brought into
the cache, because it is likely to be needed again soon

Spatial locality of reference :

 Instructions with addresses close to a recently executed instruction
are likely to be executed soon.
 Spatial locality suggests that instead of fetching just one item from the
main memory to the cache, it is useful to fetch several items that are
located at adjacent addresses as well.
 CACHE MEMORIES

 Processor issues a Read request, a block of words is transferred

from the main memory to the cache, one word at a time.
 Subsequent references to the data in this block of words are found
in the cache.
 CACHE MEMORIES

 At any given time, only some blocks in the main memory are
held in the cache. Which blocks in the main memory are in the
cache is determined by a “mapping function”.

 When the cache is full, and a block of words needs to be

transferred from the main memory, some block of words in the
cache must be replaced.
 This is determined by a “replacement algorithm”.
 CACHE HIT
 Existence of a cache is transparent to the processor. The processor
issues Read and Write requests in the same manner.
 If the Read or Write operation is performed on the appropriate cache
location, it is called a read or write hit.
 The main memory is not involved when there is a cache hit in a Read
operation.
 For a Write operation, the system can proceed in one of two ways.
o Write-through protocol: Contents of the cache and the main
memory may be updated simultaneously
o Write-back or copy-back protocol : Update the contents of the
cache, and mark it as updated by setting a bit known as the dirty
bit or modified bit. The contents of the main memory are updated
when this block is replaced
 CACHE MISS
 If the data is not present in the cache, then a Read miss or Write miss
occurs.
Read miss:
 If the addressed word in a read operation is not in cache read miss
occurs
 Block of words containing this requested word is transferred from the
memory into cache.
 After the block is transferred, the desired word is forwarded to the
processor.
 The desired word may also be forwarded to the processor as soon as it
is read without waiting for the entire block to be transferred. This is
called load-through or early-restart.
 CACHE MISS
Write-miss:
 During a write operation, if the addressed word is not in
cache write miss occurs
 If Write-through protocol is used, then the contents of the
main memory are updated directly.
 If write-back protocol is used, the block containing the
addressed word is first brought into the cache. The
desired word is overwritten with new information.
MAPPING FUNCTIONS
 MAPPING FUNCTIONS
 Methods for determining where memory blocks are placed in the cache.

 A simple processor example:

 Cache consisting of 128 blocks of 16 words each. Total size of cache
is 2048 (2K) words.
 Main memory is addressable by a 16-bit address.
 Main memory has 64K words and Main memory has 4K(4096) blocks
of 16 words each.

 Three mapping functions:

o Direct mapping
o Associative mapping
o Set-associative mapping.
 DIRECT MAPPING

 The simplest way to determine cache locations in which to store

memory blocks is the direct-mapping technique.
 Block j of the main memory maps onto block j modulo 128 of the
cache.
 If one of the main memory blocks 0, 128, 256, . . . is loaded into
the cache, it is stored in cache block 0. Blocks 1, 129, 257, . . . are
stored in cache block 1, and so on.
 Since more than one memory block is mapped onto a given cache
block position, contention may arise for that position even when the
cache is not full.
 Contention is resolved by allowing the new block to overwrite the
currently resident block.
DIRECT MAPPING
 Placement of a block in the cache is determined by its memory address.

 16-bit memory address can be divided into 3 fields.

 The low-order 4 bits select one of 16 words in a block.

 When a new block enters the cache, the 7-bit cache block field
determines the cache position in which this block must be stored.

 The high-order 5 bits of the memory address of the block are stored in 5
tag bits associated with its location in the cache..

 The tag bits identify which of the 32 main memory blocks mapped into this
cache position is currently resident in the cache.
DIRECT MAPPING

 As execution proceeds, the 7-bit cache block field of each

address generated by the processor points to a particular
block location in the cache.

 The high-order 5 bits of the address are compared with the

tag bits associated with that cache location.

 If they match, then the desired word is in that block of the

cache.

 If there is no match, then the block containing the required

ASSOCIATIVE MAPPING

 Most flexible mapping method, in which a main memory

block can be placed into any cache block position.

 12 tag bits are required to identify a memory block when it

is resident in the cache.

 The tag bits of an address received from the processor are

compared to the tag bits of each block of the cache to see if
the desired block is present.

 This is called the associative-mapping technique.

ASSOCIATIVE MAPPING

 When a new block is brought into the cache, it replaces

(ejects) an existing block only if the cache is full.

 The complexity of an associative cache is higher than that of

a direct-mapped cache.

 Reason is the need to search all 128 tag patterns to

determine whether a given block is in the cache
SET-ASSOCIATIVE MAPPING

 Another approach is to use a combination of the direct- and

associative-mapping techniques.

 The blocks of the cache are grouped into sets, and the
mapping allows a block of the main memory to reside in any
block of a specific set.

 The contention problem of the direct method is eased by

having a few choices for block placement.

 Also hardware cost is reduced by decreasing the size of the

SET-ASSOCIATIVE
MAPPING:EXAMPLE
SET-ASSOCIATIVE MAPPING

 Here a cache with two blocks per set is implemented.

 In this case, memory blocks 0, 64, 128, . . . , 4032 map into
cache set 0, and they can occupy either of the two block
positions within this set.
 Having 64 sets means that the 6-bit set field of the address
determines which set of the cache might contain the desired
block.
 The tag field of the address must then be associatively
compared to the tags of the two blocks of the set to check if
the desired block is present.
 The number of blocks per set is a parameter that can be
selected to suit the requirements of a particular computer.
 SET-ASSOCIATIVE MAPPING
 Four blocks per set can be accommodated by a 5-bit set field.

 Eight blocks per set by a 4-bit set field, and so on.

 The extreme condition of 128 blocks per set requires no set bits
and corresponds to the fully-associative technique, with 12 tag
bits.

 The other extreme of one block per set is the direct-mapping

method.
QUESTION

 A set-associate cache consists of a total of 64 blocks divided into

four-block sets. The main memory contains 4096 blocks, each
consisting of 128 words. How many bits are there in the main
memory address? How many bits are there in each of the TAG, SET
and WORD fields?

 Memory address length:

 4096(212) blocks and 128(27) words in each block.
 To uniquely identify a word: 12+7=19
 We are using 19 bit address.
QUESTION

 In set associative mapping: to uniquely identify a word from a block of

128 words-7 bits needed.
 So out of 19-bit memory address lower order 7 bits are used for word
identification.
 The remaining 12 is distributed between tag and set .

 64 blocks divided into 4 block sets. no:of sets =64/4=16=24.

 So 4-bits are used for identifying the set.

tag
 Remaining 8-bits set word
specify the tag
8 4 7
 REPLACEMENT ALGORITHMS

 In a direct-mapped cache, the position of each block is predetermined

by its address; hence, the replacement strategy is trivial.

 In associative and set-associative caches there exists some flexibility.

 When a new block is to be brought into the cache and all the
positions that it may occupy are full, the cache controller must decide
which of the old blocks to overwrite.

 The preferred approach is to keep blocks in the cache that are likely
to be referenced in the near future
 REPLACEMENT ALGORITHMS

 With respect to the property ‘Locality of Reference’, there is

a high probability that the blocks that have been referenced
recently will be referenced again soon.

 So when a block is to be overwritten, it is sensible to

overwrite the one that has gone the longest time without
being referenced.

 This block is called the least recently used (LRU) block,

and the technique is called the LRU replacement algorithm
 REPLACEMENT ALGORITHMS

 To use the LRU algorithm, the cache controller must track

references to all blocks as computation proceeds.

 Although it performs well for many access patterns, it can

lead to poor performance in some cases like when
accesses are made to sequential elements of an array that
is slightly too large to fit into the cache .

 Another reasonable rule would be to remove the “oldest”

block from a full set when a new block must be brought in.
 REPLACEMENT ALGORITHMS

 Several other replacement algorithms are also used in

practice.

 As this algorithm does not take into account the recent

pattern of access to blocks in the cache, it is generally
not as effective as the LRU algorithm.

 The simplest algorithm is to randomly choose the block

to be overwritten.
VIRTUAL MEMORY
 VIRTUAL MEMORY

 In most modern computer systems, the physical main

memory is not as large as the address space of the
processor.

 Virtual Memory is a storage mechanism which offers

user an illusion of having a very big main memory.

 It is done by treating a part of secondary memory as

the main memory.
 VIRTUAL MEMORY
 When we try to run a program, if it do not completely fit into the
main memory the parts of it currently being executed are stored in
main memory and remaining portion is stored in secondary storage
device.

 As these parts are needed for execution, they must first be brought
into the main memory, possibly replacing other parts that are
already in the memory.

 The operating Systems automatically moves data and programs

between main memory and secondary storage.
 VIRTUAL MEMORY
 Under a virtual memory system, programs, and hence the
processor, reference instructions and data in an address space
that is independent of the available physical main memory
space.

 The binary addresses that the processor issues for either

instructions or data are called virtual or logical addresses.

 A special hardware unit knows as MMU translates Virtual

Address into Physical Address.
 If a virtual address refers to a part of the program or data space
that is currently in the physical memory, then the contents of
 VIRTUAL MEMORY
 Otherwise, the contents of the referenced address must be brought into a
suitable location in the memory before they can be used.

 The Memory Management Unit (MMU), keeps track of

which parts of the virtual address space are in the physical
memory.

 When the desired data or instructions are in the main memory,

the MMU translates the virtual address into the corresponding
physical address.

 If the data are not in the main memory, the MMU causes the
operating system to transfer the data from the disk to the
memory.
 PAGING

 Paging technique plays an important role in implementing

virtual memory

 Paging is a memory management technique in which process

address space is broken into blocks of words of same size
called pages.

 Pages Commonly range from 2K to 16K bytes in length.

 PAGING

 They constitute the basic unit of information that is moved between

main memory and disk whenever translation mechanism decides
that a move is needed.

 The size of the process is measured in the number of pages.

 Similarly, main memory is divided into small fixed-sized blocks of

(physical) memory called frames and the size of a frame is kept the
same as that of a page
 PAGING-ADDRESS
TRANSLATION
 Virtual address generated by the processor consists of page number
(high-order bits) followed by an offset (low-order bits) that specifies the
location of a particular byte (or word) within a page.

 Physical address is represented by a frame number and the offset.

 A data structure called page map table is used to keep track of the
relation between a page of a process to a frame in physical memory.

Page table specifies the page no and the corresponding frame no in

memory
 PAGING-ADDRESS
TRANSLATION
 The starting address of the page table is kept in a page table base
register.

 By adding the virtual page number to the contents of this register, the
address of the corresponding entry in the page table is obtained.

 The contents of this location give the starting address of the frame if it
currently resides in the main memory

 Adding the offset to the frame will give the address of the specific word
in the memory.
 ADDRESS TRANSLATION-TLB

 The page table information is used by the MMU for every read and write
access.
 But due to its size page table cannot be included in MMU.
 The complete table is kept in the main memory.
 But a copy of only a small portion of the table corresponding to the most
recently accessed pages is accommodated within the MMU.
 They are stored in a small table, usually called the Translation Lookaside
Buffer (TLB) which acts as a cache for page table.
 Given a virtual address, the MMU looks in the TLB for the referenced page. If
the page table entry for this page is found in the TLB, the physical address
is obtained immediately.
 If there is a miss in the TLB, then the required entry is obtained from the
page table in the main memory and the TLB is updated
 SEGMENTATION

 Segmentation is a memory management technique in which

the memory is divided into the variable size parts.
 Segments usually correspond to natural divisions of a program such
as individual routines or data tables
 In a computer system using segmentation, a reference to a memory
location(virtual address) includes a value that identifies a segment
and an offset (memory location) within that segment.

 A hardware memory management unit (MMU) is responsible for

translating the virtual address into a physical address.
 SEGMENT TABLE

 MMU uses a segment table to map virtual address to physical

address.
 Segment table is a table that stores the information about all
segments and has two columns.
 First column stores the size or length of the segment.
 Second column stores the base address or starting address of the
segment in the main memory.
 The segment id in virtual address is compared with the segment
table entries and if a match is found the base address of the segment
is retrieved from table .
 Using the base address and offset we can retrieve the required word
from the segment in main memory.