M3 – Memory Hierarchy

3.2 – Virtual Memory, Address Translation

 Process/Program Address Space
Byte Address
0
CODE

. DATA

. HEAP

STACK
232-1
 Process/Program Address Space
Byte Address
0
CODE
●
Compiler assumes a
. DATA
linear address space
. HEAP – Byte 0 to Byte 232-1
.

STACK
232-1
 Process/Program Address Space
Byte Address
0
CODE ●
Compiler assumes a linear
. DATA address space
. HEAP
– Byte 0 to Byte 232-1

. ●
Virtual Address space

STACK
232-1
 Process/Program Address Space
Byte Address
0
CODE ●
Compiler assumes a linear
. DATA address space
. HEAP
– Byte 0 to Byte 232-1

Virtual Address space

. ●

●
The entire process data
structure may not be present
in MM at all times.
STACK
232-1
Paged Virtual Memory
Virtual
VirtualAddress
AddressSpace
Space
Byte
0 Virtual Page
Number 0

. Virtual Page
Number 1

.
. .
.
.
Virtual Page
Number N-1
232-1
Paged Virtual Memory

MAIN
MAINMEMORY
Page Number 0

Virtual
VirtualAddress
AddressSpace
Space

MEMORY
Byte
0 Virtual Page Page Number 13
Number 0

. Virtual Page
Number 1

.
. .
.
.
Virtual Page
Number N-1
232-1
Paged Virtual Memory

MAIN
MAINMEMORY
Page Number 0

Virtual
VirtualAddress
AddressSpace
Space

MEMORY
Byte
0 Virtual Page Page Number 13
Number 0

. Virtual Page
Number 1

. Page Number 1

HARD
HARDDISK
. Page Number 2

.
.

DISK
. .
.
Page Number N-1
Virtual Page
Number N-1
232-1
Paged Virtual Memory

MAIN
MAINMEMORY
Page Number 0

Virtual
VirtualAddress
AddressSpace
Space

MEMORY
Byte
0 Virtual Page Physical Page Number 13
Number 0 Physical
Address
AddressSpace
.
Space
Virtual Page
Number 1

. Page Number 1

HARD
HARDDISK
. Page Number 2

DISK
. .
. .
Page Number N-1
Virtual Page
Number N-1
232-1
 The Memory Hierarchy
Physical
Physical Virtual
Addresses
Addresses m em ory
Virtual
Virtual
Addresses Main m emory
Addresses
Cache
Regis ters

Words
Lines
(transf erred Pages
explicitly (transf erred
via load/store) automatically (transf erred
upon cache miss) automatically
upon page f ault)

Virtual
Virtual
Addresses
Addresses
Address Translation Table

Physical Page Numbers

VPN 0 PPN 0
VPN 1

.
.

.
.
VPN N-1
 Address Translation Table

Physical Page Numbers

Virtual Address VPN 0 PPN 0
VPN PO VPN 1

.
Virtual Address size --> 32bits�
4K Page --> 2^12B page --> 12b field�
VPN field --> (32 - 12)b --> 20b�
. PPN
Virtual Pages --> 2^20 Virtual Pages�
�
Size of each entry = 8B�
Size of Page Table = 8 x 2^20 B�
�
.
48b VA --> 2^36 pages --> 512GB .
40b VA --> 2^28 pages --> 2GB page table per process�
VPN N-1
 Address Translation Table

Physical Page Numbers

Virtual Address VPN 0 PPN 0
VPN PO VPN 1

.
. PPN

.
.
VPN N-1
 Address Translation Table

Physical Page Numbers

Virtual Address VPN 0 PPN 0
VPN PO VPN 1

.
. PPN

PPN PO
.
Physical Address .
VPN N-1
Virtual Memory
Virtual Memory
Virtual Memory
 Address Translation Table
MAIN
MAINMEMORY
MEMORY
Physical Page Numbers
Virtual Address VPN 0
VPN PO VPN 1

PPN

PPN PO
Physical Address VPN N
 Address Translation Table
MAIN
MAINMEMORY HARD
MEMORY HARDDISK
DISK
Physical Page Numbers Disk Addresses
Virtual Address VPN 0
VPN PO VPN 1

PPN Disk Address

PPN PO
Physical Address VPN N
 Address Translation Table
MAIN
MAINMEMORY HARD
MEMORY HARDDISK
DISK
Physical Page Numbers Disk Addresses V
Virtual Address VPN 0
VPN PO VPN 1

PPN Disk Address

PPN PO
Physical Address VPN N
 Implementation of Address Translation
●
Process always uses virtual addresses

LD R2, 16(R7) # VA --> [R7 + 16]

# VA --> PA (Page Table)
# Access the PA.

PC to IM (VA)
 Implementation of Address Translation
●
Process always uses virtual addresses
●
Memory Management Unit (MMU): part of CPU; hardware that
does address translation
 Implementation of Address Translation
●
Process always uses virtual addresses
●
Memory Management Unit (MMU): part of CPU; hardware that
does address translation
●
The page tables are (at best) present in the MM (OS virtual
address space)
– One main memory reference per address translation!
●
 Implementation of Address Translation
●
Process always uses virtual addresses
●
Memory Management Unit (MMU): part of CPU; hardware that
does address translation
●
To translate a virtual memory address, the MMU has to read the
relevant page table entry out of memory
–
 Implementation of Address Translation
●
Process always uses virtual addresses
●
Memory Management Unit (MMU): part of CPU; hardware that
does address translation
●
To translate a virtual memory address, the MMU has to read the
relevant page table entry out of memory
– Caches recently used translations in a Translation Lookaside Buffer
(Page Table Cache)
 Caches and Address Translation

Virtual Physical
Address Address
CPU MMU Cache
 Caches and Address Translation
Physically
PhysicallyAddressed
AddressedCache
Cache

Virtual Physical
Address Address
CPU MMU Cache
 Caches and Address Translation
Physically
PhysicallyAddressed
AddressedCache
Cache

Virtual Physical
Address Address
CPU MMU Cache

Virtual Cache Physical

Address Miss Address Main
CPU Cache MMU Memory
 Caches and Address Translation
No conflicts on Context Switch
Physically
PhysicallyAddressed
AddressedCache
Cache

Virtual Physical
Address Address
CPU MMU Cache
. PA frm TLB
. TLB miss --> MM --> HDD

Same VA for all processes. Flush the cache on context switch OR add PID to the flags.

Virtual Cache Physical

Address Miss Address Main
CPU Cache MMU Memory
. No translation time
Virtually
VirtuallyAddressed
AddressedCache
Cache
 Which is less preferable?
●
Physical addressed cache
–

●
Virtual addressed cache
–
 Which is less preferable?
●
Physical addressed cache
– Hit time higher (cache access after translation)
●
Virtual addressed cache
–
 Which is less preferable?
●
Physical addressed cache
– Hit time higher (cache access after translation)
●
Virtual addressed cache
– Data/instruction of different processes with same virtual address in
cache at the same time ...
●
 Which is less preferable?
●
Physical addressed cache
– Hit time higher (cache access after translation)
●
Virtual addressed cache
– Data/instruction of different processes with same virtual address in
cache at the same time ...
●
Flush cache on context switch, or
●
Include Process id as part of each cache directory entry
 Which is less preferable?
●
Physical addressed cache
– Hit time higher (cache access after translation)
●
Virtual addressed cache
– Data/instruction of different processes with same virtual address in
cache at the same time ...
●
Flush cache on context switch, or
●
Include Process id as part of each cache directory entry
– Synonyms
 Synonyms (Aliases)
VA-T1 VA-T2
P1 P2
t1 t2

Shared L2

Main Memory
t1:
t1:Read
Readxx
Shared Physical Page
X between P1 and P2
 Synonyms (Aliases)
VA-T1 VA-T2
P1 P2
t1 t2

Shared L2

t1:
t1:Read
Readxx
X
 Synonyms (Aliases)
VA-T1 VA-T2
P1 P2
t1 t2

Shared L2

t1:
t1:Read
Readxx
t2:
t2:Read
Readxx X
 Synonyms (Aliases)
VA-T1 VA-T2
P1 P2
t1 t2

Shared L2

t1:
t1:Read
Readxx
t2:
t2:Read
Readxx X
 Synonyms (Aliases)
VA-T1 VA-T2
P1 P2
t1 t2

Shared L2

●
L2 uses virtual addresses
X

22copies
copiesofofone
onephysical
physical
page in the cache!
page in the cache!
 VA 32b address size
64KB cache, 32B Block size
DM. Virtual Address
Page Size: 4KB
Tag: 16b Index: 11b BO: 5b
Virtual Page Number: 20b Page Offset: 12b
Virtually Addressed Cache
. Index 11b
4b from VPN + 7b from the PO

Physically Addressed Cache (32b PA)

. Index 11b
Physical Addr
. 4b from PPN + 7b from the PO
PPN 4b 7b --> index

Reduce this wait

2way SA cache--> Index bits: 10b

16way SA cache--> Index bits: 7b

No wait when Index field: 7b

Overlapped Operation

MMU

Virtual Tag check

Address using PA

CPU Cache
Indexing
using VA
 Overlapped Operation

MMU

Tag check
Virtual
using PA
Address

CPU Indexing
Cache
using VA

Virtually
VirtuallyIndexed
IndexedPhysically
PhysicallyTagged
TaggedCache
Cache(VIPT)
(VIPT)

Other
Otheroptions:
options:PIPT,
PIPT,VIVT
VIVT
 Direct mapped, 32 KB, 32B block,
Recall – Cache Access 32b main memory address
 Direct mapped, 32 KB, 32B block,
Recall – Cache Access 32b main memory address

32
32bbaddress
address
 Direct mapped, 32 KB, 32B block,
Recall – Cache Access 32b main memory address

32
32bbaddress
address
Tag
Tag Index
Index(10b)
(10b) 3Offset
22
3Offset
 Direct mapped, 32 KB, 32B block,
Recall – Cache Access 32b main memory address

32
32bbaddress
address
Tag
Tag Index
Index(10b)
(10b) 3Offset
22
3Offset
 Direct mapped, 32 KB, 32B block,
Recall – Cache Access 32b main memory address

32
32bbaddress
address
Tag
Tag Index
Index(10b)
(10b) 3Offset
22
3Offset

32B
32BBlock
Block
 Direct mapped, 32 KB, 32B block,
Recall – Cache Access 32b main memory address

32
32bbaddress
address
Tag
Tag Index
Index(10b)
(10b) 3Offset
22
3Offset

W
W11 W
W22......77 W
W88
 Direct mapped, 32 KB, 32B block,
Recall – Cache Access 32b main memory address

32
32bbaddress
address
Tag
Tag Index
Index(10b)
(10b) Offset
3Offset
3 22

W
W11 WW2Block
32B
32B ......77
2Block W
W88

W
W11 W
W22......77 W
W88
=?
 Direct mapped, 32 KB, 32B block,
Recall – Cache Access 32b main memory address

32
32bbaddress
address
Tag
Tag Index
Index(10b)
(10b) 3Offset
22
3Offset

W
W11 WW2Block
32B
32B ......77
2Block W
W88

W
W11 W
W22......77 W
W88
=? ...
No Yes
Cache Miss Cache Hit
 Direct mapped, 32 KB, 32B block,
Recall – Cache Access 32b main memory address

32
32bbaddress
address

Tag
Tag Index
Index(10b)
(10b) 33
Offset22
Offset

W
W11 WW2Block
32B
32B ......77
2Block W
W88

W
W11 W
W22......77 W
W88
=? ...

No Yes
Cache Miss Cache Hit To Processor
 Direct mapped, 32 KB, 32B block,
Recall – Cache Access 32b main memory address

32
32bbaddress
address
Tag is not needed until the cache line
Tag
Tag Index
Index(10b)
(10b) 3Offset
22
3Offset has been read

W
W11 WW2Block
32B
32B ......77
2Block W
W88

W
W11 W
W22......77 W
W88
=? ...

No Yes
Cache Miss Cache Hit To Processor
64 b VM address
VM Example
 64 b VM address
VM Example
Virtual Page No. (50) Page Offset (14)
 64 b VM address
VM Example
Virtual Page No. (50) Page Offset (14)

Index (8) Offset (6)

 64 b VM address
VM Example
Virtual Page No. (50) Page Offset (14)

Index (8) Offset (6)

26b
26b Cache
CacheBlock
Block
 64 b VM address
VM Example
Virtual Page No. (50) Page Offset (14)

Index (8) Offset (6)

26b
26b Cache
CacheBlock
Block

=?
L1 Hit/Miss
 64 b VM address
VM Example
Virtual Page No. (50) Page Offset (14)

Index (8) Offset (6)

(26)
Physical 26b Cache
CacheBlock
PhysicalAddress
Address(Tag)
(Tag) 26b Block

=?
L1 Hit/Miss
 64 b VM address
VM Example
Virtual Page No. (50) Page Offset (14)

Index (8) Offset (6)

(26)
Physical 26b Cache
CacheBlock
PhysicalAddress
Address(Tag)
(Tag) 26b Block

=?
L1 Hit/Miss

Index comes from the Virtual Address

(Virtually Indexed)
Tag comes from the Physical Address
(Physically tagged)
 64 b VM address
VM Example
Virtual Page No. (50) Page Offset (14)
(43) (7)
TLB Tag TLB Index Index (8) Offset (6)

(26)
Physical 26b Cache
CacheBlock
PhysicalAddress
Address(Tag)
(Tag) 26b Block

=?
L1 Hit/Miss

Index comes from the Virtual Address

(Virtually Indexed)
Tag comes from the Physical Address
(Physically tagged)
 64 b VM address
VM Example
Virtual Page No. (50) Page Offset (14)

(43) (7)
TLB Tag TLB Index Index (8) Offset (6)

(26)
Physical 26b Cache
CacheBlock
PhysicalAddress
Address(Tag)
(Tag) 26b Block

=? =?
L1 Hit/Miss
TLB Hit/
Page Fault

Index comes from the Virtual Address

(Virtually Indexed)
Tag comes from the Physical Address
(Physically tagged)
 64 b VM address
VM Example
Virtual Page No. (50) Page Offset (14)

(43) (7)
TLB Tag TLB Index Index (8) Offset (6)

(26)
Physical 26b Cache
CacheBlock
PhysicalAddress
Address(Tag)
(Tag) 26b Block

=? =?
L1 Hit/Miss
TLB Hit/
Page Fault
Physical Address (40)

Index comes from the Virtual Address

(Virtually Indexed)
Tag comes from the Physical Address
(Physically tagged)
 64 b VM address
VM Example
Virtual Page No. (50) Page Offset (14)

(43) (7)
TLB Tag TLB Index Index (8) Offset (6)

(26)
Physical 26b Cache
CacheBlock
PhysicalAddress
Address(Tag)
(Tag) 26b Block

=? =?
L1 Hit/Miss
TLB Hit/
Page Fault
Physical Address (40)
To L2
Tag Index Offset

Index comes from the Virtual Address

(Virtually Indexed)
Tag comes from the Physical Address
(Physically tagged)
 64 b VM address
VM Example
Virtual Page No. (50) Page Offset (14)

(43) (7)
TLB Tag TLB Index Index (8) Offset (6)

(26)
Physical 26b Cache
CacheBlock
PhysicalAddress
Address(Tag)
(Tag) 26b Block

=? =?
L1 Hit/Miss
TLB Hit/
Page Fault
Physical Address (40)
To L2
Tag Index Offset

L2
L2Cache
CacheBlock
Block
Index comes from the Virtual Address
(Virtually Indexed)
Tag comes from the Physical Address =?
(Physically tagged)
 Translation Lookaside Buffer
●
Cache of page table mappings
●
32 – 4096 entries long
– SA, FA, or DM
●
Dirty flag – use during page write back
●
Ref – used for LRU

VPN PPN Valid Ref Dirty Access

(tag) (data) Rights
 Page Fault
●
Virtual address generated by processor is not available in
main memory
 Page Fault
●
Virtual address generated by processor is not available in
main memory
●
Detected on attempt to translate address
– Page Table entry is invalid
 Page Fault
●
Virtual address generated by processor is not available in main
memory
●
Detected on attempt to translate address
– Page Table entry is invalid
●
Must be `handled’ by operating system
– Identify slot in main memory to be used
– Get page contents from disk
– Update page table entry
●
 Page Fault
●
Virtual address generated by processor is not available in
main memory
●
Detected on attempt to translate address
– Page Table entry is invalid
●
Must be `handled’ by operating system
– Identify slot in main memory to be used
– Get page contents from disk
– Update page table entry
●
Provide data to the processor
 Abstraction: Virtual vs. Physical Memory
●
Programmer sees virtual memory
– Can assume the memory is “infinite”
 Abstraction: Virtual vs. Physical Memory
●
Programmer sees virtual memory
– Can assume the memory is “infinite”
●
Reality: Physical memory size is much smaller than what the
programmer assumes
 Abstraction: Virtual vs. Physical Memory
●
Programmer sees virtual memory
– Can assume the memory is “infinite”
●
Reality: Physical memory size is much smaller than what the
programmer assumes
●
The system (system software + hardware, cooperatively) maps
virtual memory addresses are to physical memory
– The system automatically manages the physical memory space
transparently to the programmer
 Abstraction: Virtual vs. Physical Memory

+ Programmer does not need to know the physical size of memory nor
manage it  A small physical memory can appear as a huge one to the
programmer  Life is easier for the programmer
-- More complex system software and architecture

A classic example of the programmer/(micro)architect tradeoff

 Virtual Memory
●
What is the size of the Page Table?
●
Where is it stored?
●
What factors decide the size of a page?
– What are its side effects?
●
Page size is constant/variable?
●
 Paged Virtual Memory
●
48 bit Virtual Addresses, 40 bit Physical Addresses. Page size
= 16KB. How many entries in a process's Page Translation
Table? What is the size of the Page Translation Table?
 Summary
●
Process/Program Address Space
●
The Memory Hierarchy
●
Address Translation Table
●
Virtual Memory
●
Implementation of Address Translation
●
Synonyms (Aliases)
●
VM Example
●
Translation Lookaside Buffer
●
Page Fault
●
Abstraction: Virtual vs. Physical Memory
Slides contents from:

Hennessy & Patterson, 6ed. Appendix B and Chapter 2.

David Wentzlaff, ELE 475 – Computer Architecture.
MJT, High Performance Computing, NPTEL.
Extra
 Fast Translation
●
Address translation is on the critical path
●
Paging – 2 memory accesses!
– Address Translation Table + Data
●
Translation Lookaside Buffer (TLB): Part of MMU that
caches address translations
 Size of a Page
●
Page is the unit of Memory Management
●
Too large vs. Too small.
●
Page Offset field need not be translated
●
What if the Page Offset field was 12 bits? (Page size = 4KB)