Memory Hierarchy

cpe 252: Computer Organization 1

 Memory Organization
• Memory Hierarchy
– Memory hierarchy in a computer system :
• Main Memory : memory unit that communicates
directly with the CPU (RAM)
• Auxiliary Memory : device that provide backup
storage (Disk Drives)
• Cache Memory : special very-high-speed memory
to increase the processing speed (Cache RAM)
Auxiliary memory
Magnetic
tapes
Main
I/ O proc essor
memory
Magnetic
disks

C ac he
C PU
memory
 Memory Organization
– Multiprogramming
• enable the CPU to process a number of
independent program concurrently
– Memory Management System :
• supervise the flow of information between auxiliary
memory and main memory
 Memory Hierarchy is to obtain the highest possible access speed while minimizing
the total cost of the memory system
Register

Cache

Main Memory

Magnetic Disk

Magnetic Tape
Random Access Memory (RAM)
– RAM Chips

– Static RAM
• Consists of Flip-flops to store binary information.
• Static RAM is easier to use and having short Read/Write cycles
• Used mostly in Cache memory.

– Dynamic RAM
 Stores binary information in the form of electric charge stored inside the
capacitor.
 The capacitors are provided by MOS transistors.
 Refreshing circuit is required to refresh the memory.
• Dynamic RAM offers reduced power consumption and large storage
capacity.
• It is used to construct main memory.
 Main Memory

MAIN MEMORY
RAM and ROM Chips
Typical RAM chip
Chip select 1 CS1
Chip select 2 CS2
Read RD 128 x 8 8-bit data bus
RAM
Write WR
7-bit address AD 7

CS1 CS2 RD WR Memory function State of data bus

0 0 x x Inhibit High-impedence
0 1 x x Inhibit High-impedence
1 0 0 0 Inhibit High-impedence
1 0 0 1 Write Input data to RAM
1 0 1 x Read Output data from RAM
1 1 x x Inhibit High-impedence

Typical ROM chip

Chip select 1 CS1
Chip select 2 CS2
512 x 8 8-bit data bus
ROM
9-bit address AD 9
 Main Memory

MEMORY ADDRESS MAP

Address space assignment to each memory chip

Example: 512 bytes RAM and 512 bytes ROM

Hexadecimal Address bus

Component address 10 9 8 7 6 5 4 3 2 1
RAM 1 0000 - 007F 0 0 0 x x x x x x x
RAM 2 0080 - 00FF 0 0 1 x x x x x x x
RAM 3 0100 - 017F 0 1 0 x x x x x x x
RAM 4 0180 - 01FF 0 1 1 x x x x x x x
1 x x x x x x x x x
ROM 0200 - 03FF

Memory Connection to CPU

- RAM and ROM chips are connected to a CPU
through the data and address buses

- The low-order lines in the address bus select

the byte within the chips and other lines in the
address bus select a particular chip through
its chip select inputs
 Main Memory

CONNECTION OF MEMORY TO CPU

Address bus CPU
16-11 10 9 8 7-1 RD WR Data bus

Decoder
3 2 1 0
CS1
CS2

Data
RD 128 x 8
RAM 1
WR
AD7

CS1
CS2

Data
RD 128 x 8
RAM 2
WR
AD7

CS1
CS2

Data
RD 128 x 8
RAM 3
WR
AD7

CS1
CS2
RD 128 x 8 Data
RAM 4
WR
AD7

CS1
CS2
Data

1- 7 512 x 8
8
9 } AD9 ROM
 Numerical Problems
Q1. a) How many 128x8 RAM chips are needed to provide a memory capacity of 2048
bytes.
b) How many lines of the address bus must be used to access 2048 bytes of memory.
How many of these lines will be common to all chips.
c) How many lines must be decoded for chip select? Specify the size of decoders.

Q2. Extend the memory system of Fig.1 to 4096 bytes of RAM. List the memory-address
map and indicate what size decoders are needed.

Q3. A computer employs RAM chips of 256x8 and ROM chips of 1024x8. The computer
system needs 2K bytes of RAM, 4K bytes of ROM, and four interface units each with four
registers. A memory-mapped I/O configuration used. The two highest-order bits of the
address bus are assigned 00 for RAM, 01 for ROM, and 10 for interface registers.

How many RAM and ROM chips are needed.

Draw a memory-address map for the system.
Give the address range in hexadecimal for RAM, ROM and interface.
• 12-5 CacheCache
Memory
Memory
– Locality of Reference
• the references to memory tend to be confined within a few localized
areas in memory

– Cache Memory : a fast small memory

• keeping the most frequently accessed instructions and data in the fast
cache memory

– Hit Ratio
• the ratio of the number of hits divided by the total CPU references (hits +
misses) to memory
– hit : the CPU finds the word in the cache
– miss : the word is not found in cache (CPU must read main
memory)

– A computer with cache access time of 100ns, a main memory access time of
1000 ns, a hit ratio of 0.9 is having average access time of 200ns.
 Types of Mapping of Cache
Memory
– Mapping
• The transformation of data from main memory to cache memory
– 1) Associative mapping
– 2) Direct mapping
– 3) Set-associative mapping

– Associative Mapping:
main memory : 32 K x 12 bit word (15 bit address lines)
cache memory : 512 x 12 bit word
– CPU sends a 15-bit address to cache
» Hit : CPU accepts the 12-bit data from cache
» Miss : CPU reads the data from main memory (then data
is written to cache)
 Associative Mapping

Main memory
C PU
32K× 12 C ac he memory
512× 12

C P U address(15 bits)

Argument register

Address D ata

0 1 0 0 0 3 4 5 0

0 2 7 7 7 6 7 1 0

2 2 3 4 5 1 2 3 4
Direct mapping :
Direct Mapping
n bit memory address
Tag field (n - k) : Index field (k)
2k words cache memory + 2n words main memory
Tag = 6 bit (15 - 9), Index = 9 bit

6 bits 9 bits
Tag Index

00 000 000
512× 12
32K× 12
C ache memory
Octal
Hex Main memory address
Address = 9 bits
Address
Data = 12 bits
1FF
Address = 15 bits
3F 1FF Data = 12 bits
Memory
Direct Mapping
Index
 Direct mapping cache with block size of
8 words :
address Memory data address Tag Data
» 64 block x 8 word = 512
000000 1220 000 00 1220
cache words size
Index Tag Data 6 6 3

00777 2340 000 01 3450 Tag Block Word

Block 0
01000 3450 007 01 6578
Index
010
Block 1
01777 4560 777 02 6710
017
02000 5670
(b) C ache memory

02777 6710

770 02
Block 63
777 02 6710
(a) Main memory
Set-Associative
Set-associative mapping : (two-way)
Mapping
Index Tag D ata Tag D ata
000 0 1 3 4 5 0 0 2 5 6 7 0

777 0 2 6 7 1 0 0 0 2 3 4 0
 Questions
• A two-way set associative cache memory uses blocks of four words. The
cache can accommodate a total of 2048 words from main memory. The
main memory size is 128Kx32. Formulate all pertinent information required
to construct the cache memory. What is the size of cache memory.
• The access time of a cache memory is 100ns and that of main memory is
1000ns. It is estimated that 80 percent of the memory requests are for read
and the remaining 20 percent for write. The hit ratio for read access only is
0.9. A write-through procedure is used. What is the average access time of
the system considering only memory read cycles. What is the average
access time of the system for both read and write requests. What is the hit
ratio considering the write requests also
• A computer has a memory unit of 64Kx16 and a cache memory of 1K
words. The cache uses direct mapping with a block size of four words. How
many bits are there in the tag, index, block, and words fields of the address
format. How many bits are there in each word of cache, and how are they
divided into functions? Include a valid bit. How many blocks can the cache
accommodate?
Answer
Answer
– Replacement Algorithm : cache miss or full
• 1) LRU (Least Recently Used):
• 2) Random Replacement:
• 3) FIFO (First-In First-Out) :

– Writing to Cache :
– 1) Write-through : Every operation result write into cache as
well as main memory.
– 2) Write-back : After every operation cache result will update,
only final result need to copy into main memory.
Cache Initialization
• Cache is initialized :
– 1) When power is applied to the computer
– 2) When main memory is loaded with a complete set of
programs from auxiliary memory
– 3) Cache is initialized by clearing all the valid bits to 0.
• Valid bit
– Indicate whether or not the word contains valid data
 Virtual Memory

VIRTUAL MEMORY

Give the programmer the illusion that the system has a very large memory,
even though the computer actually has a relatively small main memory

Address Space(Logical) and Memory Space(Physical)

address space memory space

virtual address Mapping

(logical address) physical address

address generated by programs actual main memory address

 Virtual Memory

PAGE REPLACEMENT ALGORITHMS

FIFO Reference string
7 0 1 2 0 3 0 4 2 3 0 3 2 1 2 0 1 7 0 1
7 7 7 2 2 2 4 4 4 0 0 0 7 7 7
0 0 0 3 3 3 2 2 2 1 1 1 0 0
1 1 1 0 0 0 3 3 3 2 2 2 1
Page frames

FIFO algorithm selects the page that has been in memory the longest time

Optimal Replacement (OPT) - Lowest page fault rate of all algorithms

Replace that page which will not be used for the longest period of time
Reference string
7 0 1 2 0 3 0 4 2 3 0 3 2 1 2 0 1 7 0 1
7 7 7 2 2 2 2 2 7
0 0 0 0 4 0 0 0
1 1 3 3 3 1 1

Page frames
 Virtual Memory

PAGE REPLACEMENT ALGORITHMS

LRU (Least Recently Used)
- OPT is difficult to implement since it requires future knowledge

Replace that page which has not been

used for the longest period of time

Reference string
7 0 1 2 0 3 0 4 2 3 0 3 2 1 2 0 1 7 0 1
7 7 7 2 2 4 4 4 0 1 1 1
0 0 0 0 0 0 3 3 3 0 0
1 1 3 3 2 2 2 2 2 7
Page frames
 Question
The page request are in following order. If
there are three memory frames, show the
execution of the following pages using
a. FIFO b. Optimal c. LRU

6 0 1 3 2 5 0 4 7 1 3 2 6 0 4
 Input-Output Organization
 Peripheral Devices: Input & Output devices attached to computer
are called Peripheral.
 Input and output devices communicate alphanumeric information by
using ASCII 7bit code.
 To use computer efficiently, large number of programs and data
must be prepared in advance for execution with computer.
 I/O interface provides a method for transferring information between
internal storage and external I/O devices.
 Input-Output Interface
 The purpose of interfacing are as follows:
– Peripherals are electromechanical & electromagnetic devices
and are interacting with electronics devices(CPU).
– Data transfer rate of peripherals is usually slower than transfer
rate of CPU.
– Data codes and formats in peripherals differ from word format in
CPU
– Operating modes of peripherals are different from each other
and each one must be controlled without disturbing other.
 Input-Output Interface
 To resolve these differences, computer system includes
Interface units between CPU and peripherals to supervise and
synchronize all input and output transfers.

 Each peripheral has its own controller that operates a

particular electromechanical device.
 To communicate with a particular device, the processor places
a device address on the address lines.
Input-Output Interface
 Input-Output Interface

When the interface detects it own address,

it activates path between bus lines and the
device.
At the time address is made available in
address lines, the processor provides
function code(I/O command) in the control
lines.
 I/O versus Memory Bus
 In addition to communicating to I/O, processor must also
communicate with memory unit.
 There are three ways that computer buses can be used to
communicate with memory and I/O:
 Use two separate buses, one for memory and one for I/O. (IOP)
 Use one common bus for both memory and I/O but have
separate control lines for each. (Isolated I/O)
 Use one common bus for memory and I/O with common control
lines. (Memory Mapped I/O)
 Input-Output Transfer (IOP)
An IOP takes care of input and output tasks.
The CPU is assigned the task of initiating all
operations, but I/O instructions are executed
in IOP.
When an I/O operation is required, the CPU
informs the IOP where to find the I/O
program and then leaves the transfer details
Input-Output Transfer (IOP)
 Isolated I/O
 The distinction between memory and I/O transfer is made through
separate read and control lines.

 I/O read and I/O write are enabled during I/O transfer and Memory
read/write are enabled during memory transfer.

 In the isolated I/O configuration, CPU have distinct input and output
instructions where each of it will be associated with address of the
interface register.

 When the CPU fetches and decodes the I/O instruction, it places the
address associated with the instruction on the common address lines
and enables I/O read or I/O write control line.
 Isolated I/O
When the CPU fetches and decodes the
Memory instruction, it places the address
associated with the instruction on the
common address lines and enables Memory
read or Memory write control line.

The isolated I/O method isolates memory

and I/O addresses.
 Memory Mapped I/O
 In this configuration, same address space is used for both memory
and I/O.

 The computer treats interface register (I/O) as a part of memory

system.

 The assigned address cannot be used for storing memory words,

which reduces memory address range available.

 In a memory-mapped I/O organization, there are no specific, input

or output instruction.
Memory Mapped I/O
 Modes of Transfer
• Data transfer between the CPU and the I/O devices may
be handled in variety of modes. Some modes use the
CPU as an intermediate path and others transfer the
data directly to and from the memory unit.

• Data transfer to and from peripherals may be handled in

three ways:

– Programmed I/O

– Interrupt-initiated I/O

– Direct Memory Access (DMA)

 Programmed I/O
• Programmed I/O operations are the result of I/O
instructions. Each data item transfer is initiate by an
instruction in the program.

• Usually the transfer is to and from a CPU register and

peripheral. Other instructions are needed to transfer data
between memory and CPU.

• Once a data transfer is initiated, the CPU is required to

monitor the interface to see when a transfer can again be
made.
Programmed I/O
 Interrupt Initiated I/O
• In the programmed I/O CPU stays in
program loop until the I/O indicates that it
is ready for data transfer.

• This is time consuming process since it

makes CPU busy needlessly.

• This can be avoided by using an interrupt

facility.
 Direct Memory Access (DMA)
• The transfer of data between a fast storage device such as magnetic disk
and memory often limited to the speed of CPU.

• Removing the CPU and letting the peripheral device manage the memory
bus directly improve speed of transfer.

• Such transfer technique is called Direct Memory Access (DMA).

• A DMA controller takes over the buses to manage the transfer directly
between I/O device and memory.

• During DMA transfer, the CPU is idle and has no control over memory
buses.

• By using Bus Request(BR) and Bus Grant(BG) the buses are released to
DMA controller.
 DMA

• Data transfer ways:

– Burst Transfer: Here number of words are transferred in a block.

Example: Magnetic disk.

– Cycle stealing: Allows the DMA controller to transfer one data word
at a time after it must return the control of buses to CPU.
DMA Controller
 Initialization of DMA

• The CPU initializes the DMA by sending the following

information through the data bus.

– The starting address of the memory block where data are

available(for read) or where data are to be stored(for write).

– The word count, which is the number of words in the memory

block.

– Control to specify the mode of transfer such as read or write.

– A control to start the DMA transfer.

DMA Transfer in a computer
system
 Bus Arbitration
• The device that is allowed to initiate data transfers on
the bus at any given time is called Bus master.
• Bus arbitration is the process by which the next device
becomes Bus master and will do the data transfer.
• Two approaches: Centralized Arbitration and Distributed
Arbitration.

» Centralized Arbitration
 Types of Interrupt
• Major types of Interrupt are:
– External Interrupt: It comes from I/O devices, from timing device, or
from any other external source.
– Internal Interrupt: It includes register overflow, invalid operation
code, stack overflow etc.
– Software Interrupt or Hardware Interrupt: External and Internal
interrupts are initiated from signals that occur in the hardware of
the CPU. A software interrupt is initiated by executing an
instruction.
– Priority Interrupt: In case several sources will request service
simultaneously, in this case system must decide which device to
service first. For ex: Polling, Daisy Chain (Priority chain).
 Buses
• Data Bus: Bi-directional and transfers data.
• Address Bus: Uni-directional and sends the
address.
• Control Bus: R/W, BR,BG etc.

• Bus Structure: Single bus

 Buses
• Bus Structure: Multi bus
 Synchronous or Asynchronous
Data Transfer
• Internal operations in a digital system are synchronized by
means of clock pulses supplied by common pulse generator.

• If the registers in the interface share a common clock with

the CPU registers, the transfer is synchronous.

• Asynchronous data transfer requires that control signals be

transmitted between communicating units to indicate the
time at which data is being transmitted.

• Asynchronous data transfer can be accomplished by Strobe

& Handshaking.
 Strobe Control
• Strobe control:
– It employs single control line.
– It can be activated either by source or destination unit.
Handshaking