CSE 4293 – Computer Architecture

Memory
Organization
EEE, AUST
 Memory
Hierarchy
In the Computer System Design, Memory Hierarchy is an enhancement to organize
the memory such that it can minimize the access time. The Memory Hierarchy was
developed based on a program behavior known as locality of references.

CSE-4293 Computer
 The figure below clearly demonstrates the different levels of memory hierarchy :

Internal Memory Register

Cache
memory

Primary Memory Main

memory(RAM)

USB/ Flash Memory

Secondary Storage
Magnetic Disk/Hard Disk

Tertiary storage Magnetic Tape/ Tape drives

CSE-4293 Computer
 Memory
Capacity: Bytes
Register
KB/MB Cache
memory
GB Main
memory(RAM)

GB USB/ Flash Memory

T Magnetic Disk/Hard Disk

B
TB/PB Magnetic Tape/ Tape drives

CSE-4293 Computer
 Access Time:

1 ns
Register
10 Cache
ns memory
100 Main
ns memory(RAM)

10
USB/ Flash Memory
ms
10 Magnetic Disk/Hard Disk
ms
100 Magnetic Tape/ Tape drives
ms

CSE-4293 Computer
 Spee
d:
1 ns
Register
10 Cache
ns memory
100 Main
ns memory(RAM)

10
USB/ Flash Memory
ms
10 Magnetic Disk/Hard Disk
ms
100 Magnetic Tape/ Tape drives
ms

CSE-4293 Computer
 Cost per Bit:

Register
Cache
memory
Main
memory(RAM)

USB/ Flash Memory

Magnetic Disk/Hard Disk

Magnetic Tape/ Tape

drives

CSE-4293 Computer
 Frequently Uses:

Register
Cache
memory
Main
memory(RAM)

USB/ Flash Memory

Magnetic Disk/Hard Disk

Magnetic Tape/ Tape

drives

CSE-4293 Computer
 We can infer the following characteristics of Memory Hierarchy Design :

1. Capacity:
It is the global volume of information the memory can store. As we move from top to bottom in
the Hierarchy, the capacity increases.

2. Access Time:
It is the time interval between the read/write request and the availability of the data. As we
move from top to bottom in the Hierarchy, the access time increases.

CSE-4293 Computer
 3. Performance:
Earlier when the computer system was designed without Memory Hierarchy design, the speed gap
increases between the CPU registers and Main Memory due to large difference in access time. This
results in lower performance of the system and thus, enhancement was required. This enhancement
was
made in the form of Memory Hierarchy Design because of which the performance of the system
increases. One of the most significant ways to increase system performance is minimizing how far
down the memory hierarchy one has to go to manipulate data.

4. Cost per bit:

As we move from bottom to top in the Hierarchy, the cost per bit increases i.e. Internal Memory is
costlier than External Memory.

CSE-4293 Computer
 This Memory Hierarchy Design is divided into 2 main types:

1. External Memory or Secondary Memory –

Comprising of Magnetic Disk, Optical Disk, Magnetic
Tape i.e. peripheral storage devices which are accessible
by the processor via I/O Module.

2. Internal Memory or Primary Memory –

Comprising of Main Memory, Cache Memory & CPU
registers. This is directly accessible by
the processor.

CSE-4293 Computer
 A memory unit is the collection of storage units or devices together. The memory unit
stores the binary information in the form of bits. Generally, memory/storage is classified
into 2 categories:

 Volatile Memory: This loses its data, when power is switched off.

 Non-Volatile Memory: This is a permanent storage and does not lose any data
when
power is switched off.

CSE-4293 Computer
 The main memory occupies the central position because it is equipped to communicate
directly with the CPU and with auxiliary memory devices through Input/output processor
(I/O).

When the program not residing in main memory is needed by the CPU, they are brought in
from auxiliary memory. Programs not currently needed in main memory are transferred
into auxiliary memory to provide space in main memory for other programs that are
currently in use.

The cache memory is used to store program data which is currently being executed in the
CPU. Approximate access time ratio between cache memory and main memory is about 1 to
10

CSE-4293 Computer
 Main Memory

 The memory unit that communicates directly within

the CPU, Auxiliary memory and Cache memory, is
called main memory. It is the central storage unit of
the computer system.

 It is a large and fast memory used to store data during

computer operations. Main memory is made up of
RAM and ROM, with RAM integrated circuit chips
holing the major share.

CSE-4293 Computer
 RAM: Random Access Memory
 DRAM: Dynamic RAM, is made of capacitors and transistors, and must be
refreshed every 10~100 ms. It is slower and cheaper than SRAM.

 SRAM: Static RAM, has a six transistor circuit in each cell and retains/holds data,
until powered off.

 NVRAM: Non-Volatile RAM, retains its data, even when

turned off.
Example: Flash memory (pen drive).

CSE-4293 Computer
CSE-4293 Computer
DRAM vs. SRAM
 ROM: Read Only Memory

It is a non-volatile and is more like a permanent storage for information. It also stores

the bootstrap loader program, to load and start the operating system when computer is

turned on.

 PROM(Programmable ROM)

 EPROM(Erasable PROM)

 EEPROM(Electrically Erasable PROM)

CSE-4293 Computer
ROM
 Auxiliary
Memory
Devices that provide backup storage are called auxiliary memory.

For example:

Magnetic disks and tapes are commonly used auxiliary devices. Other devices used as
auxiliary memory are magnetic drums, magnetic bubble memory and optical disks.

It is not directly accessible to the CPU, and is accessed using the Input/Output
channels.

CSE-4293 Computer
 Cache Memory

The data or contents of the main memory that are used again and again by CPU, are stored

in the cache memory so that we can easily access that data in shorter time.

Whenever the CPU needs to access memory, it first checks the cache memory. If the data

is not found in cache memory then the CPU moves onto the main memory. It also transfers

block of recent data into the cache and keeps on deleting the old data in cache to

accommodate the new one.

CSE-4293 Computer
 CPU Accessing Different types of
memory

CSE-4293 Computer
 Memory Access
Methods
Each memory type, is a collection of numerous memory locations. To access data from any
memory, first it must be located and then the data is read from the memory location. Following are
the methods to access information from memory locations:

1. Sequential Access

2. Random Access

3. Direct Access

CSE-4293 Computer
 1. Sequential Access: This methods allows memory access in a sequence or in order. .(Accessing
data from magnetic tap) Memor
y
Search data from beginning to
end for any access request Time=?
Examples of sequential access memory include older
recordable media such as DVDs, CDs, and even
tapes. Hard drives are also sequential, rather than
random access memory.

Simple but slow process

CSE-4293 Computer
 2. Random Access: Main memories are random access memories, in which each memory
location has a unique address. Using this unique address any memory location can be
reached in the same amount of time in any order. Example: RAM

Memory

Search data from any

location at the same time Faster process

A web browser is an application on your computer, which means

RAM is called upon whenever you open a browser. Now, let's say
that you have a browser open for the entire time that you're using a
laptop. That means the data the browser requires to operate needs
to be stored in your laptop RAM.

CSE-4293 Computer
 3. Direct Access: In this mode, memory is divided in blocks. Data can access directly but
access time can be different.
Memor
Example: Magnetic/optical disk y

Block-
1
Blocks can be
access randomly

But each blocks are Block-

accessing sequentially 2

Combination of
random and sequential Block-
3
access methods

CSE-4293 Computer
 DMA (Direct Memory Access) Controller

The hardware device used for direct memory access is called the DMA
controller. DMA controller is a control unit, part of I/O device’s interface
circuit, which can transfer blocks of data between I/O devices and main
memory with minimal intervention from the processor.
Frees the processor of the data transfer responsibility
DMA Controller Operation
DMA controller has to share the bus with the processor to make the data transfer. The device that holds
the bus at a given time is called bus master. When a transfer from I/O device to the memory or vice verse
has to be made, the processor stops the execution of the current program, increments the program
counter, moves data over stack then sends a DMA select signal to DMA controller over the address bus.

If the DMA controller is free, it requests the control of bus from the processor by raising the bus request
signal. Processor grants the bus to the controller by raising the bus grant signal, now DMA controller is the
bus master. The processor initiates the DMA controller by sending the memory addresses, number of
blocks of data to be transferred and direction of data transfer. After assigning the data transfer task to the
DMA controller, instead of waiting ideally till completion of data transfer, the processor resumes the
execution of the program after retrieving instructions from the stack.

DMA controller now has the full control of buses and can interact directly with memory and I/O devices
independent of CPU. It makes the data transfer according to the control instructions received by the
processor. After completion of data transfer, it disables the bus request signal and CPU disables the bus
grant signal thereby moving control of buses to the CPU.

When an I/O device wants to initiate the transfer then it sends a DMA request signal to the DMA controller,
for which the controller acknowledges if it is free. Then the controller requests the processor for the bus,
raising the bus request signal. After receiving the bus grant signal it transfers the data from the device. For
n channeled DMA controller n number of external devices can be connected.
CSE-4293 Computer
 Datapath:

A data path is the ALU, the set of

registers, and the CPU's internal bus(es)
that allow data to flow between them.

CSE-4293 Computer
CSE-4293 Computer
CSE-4293 Computer
CSE-4293 Computer
 2 Level Memory
Organization

CSE-4293 Computer
  a two-level memory organization:
 a fast, smaller, and expensive level called "Level 1" (L1) cache.
 a slower, larger, and cheaper level called "Level 2" (L2) cache.

Example: Assuming
Memory Level Access
Access time for L1 cache: 1 nanosecond (ns) Time (ns)
Access time for L2 cache: 10 nanoseconds (ns) L1 Cache 1
Access time for main memory (RAM): 100 nanoseconds (ns) L2 Cache 10
Main Memory 100

Let's say the processor wants to access a specific memory location, and it turns out that the
data is not in the cache hierarchy at all (worst-case scenario). The total access time would
be:

Total access time = L1 Cache Access Time + L2 Cache Access Time + Main Memory Access
Time
Total access time = 1 ns + 10 ns + 100 ns
Total access time = 111 ns

CSE-4293 Computer
 Hit
ratio:
To calculate a hit ratio, divide the number of cache hits with the sum of the number of cache hits,
and the number of cache misses. For example, if you have 51 cache hits and three misses over a
period of time, then that would mean you would divide 51 by 54. The result would be a hit ratio
of 0.944.

CSE-4293 Computer
 Independent Memory Organization
An independent memory organization, multiple memory modules operate independently and can
be accessed simultaneously, providing parallel memory access.

Example: Assume the following parameters for each memory module:

L1
Memory Module A: Memory Module B: Memory
Capacity: 8 GB (Gigabytes) Capacity: 16 GB (Gigabytes)
Access Time: 20 ns Access Time: 25 ns

Now, let's suppose a processor wants to access a specific piece of data from both memory
modules. Since they operate independently, the access to each module can be considered in
parallel.
So, the total access time will be the time taken by the slower memory module:

Total Access Time = Max(Access Time for Module A, Access Time for Module B)
Total Access Time = Max(20 ns, 25 ns)
Total Access Time = 25 ns

CSE-4293 Computer
 Independent Memory Organization
Tavg: This is the average memory access time,
H1/H2: hit rate (probability of a cache hit) for L1/L2. L1
T1/T2: Access time for the L1/L2. L1
Memory
Tavg= H1T1+H2T2
Memory
Example: let for L1 CP
Hit rate (H1): 0.8 (80% chance of a cache hit)
Access time (T1): 5 nanoseconds (ns) U L2
For L2 Cache:
Hit rate (H2): 0.6 (60% chance of a cache hit) L2 Memory
Access time (T2): 15 nanoseconds (ns)
Then
Tavg = H1 * T1 + H2 * T2 The computer system takes approximately 13 ns to retrieve data
Tavg = (0.8 * 5 ns) + (0.6 * 15 ns) from memory, considering both cache levels and their respective
Tavg = 4 ns + 9 ns hit rates and access times.
Tavg = 13 ns

CSE-4293 Computer
Hierarchical
Memory Tavg=
Organization
CP
1) L1 Cache (Tavg_L1):
Tavg_L1 = H1 * T1= 0.9 * 2 ns = 1.8 ns U
2) L2 Cache (Tavg_L2). This includes the time taken to check
L1 Cache (Tavg_L1) as well:
(RAM)
Tavg_L2 = H2 * (T1 + T2) + (1 - H2) * Tavg_L1
= 0.8 * (2 ns + 10 ns) + (1 - 0.8) * 1.8 ns
Tavg_L2 = 9.96 ns L1
Memory
This includes the time taken to check both L1 Cache (Tavg_L1)
and L2 Cache (Tavg_L2) as well:
Tavg_Main = (1 - H1) * (1 - H2) * Tavg_L2 + (1 - H1) * T2 + T3
Tavg_Main = (1 - 0.9) * (1 - 0.8) * 9.96 ns + (1 - 0.9) * 10 ns +
100 ns L2
Tavg_Main = 0.02 * 9.96 ns + 0.1 * 10 ns + 100 ns
Tavg_Main = 0.1992 ns + 1 ns + 100 ns Memory
Tavg_Main = 101.1992 ns

CSE-4293 Computer
 3 Level Memory
Organization

CSE-4293 Computer
 Independent Memory
Organization
L1
Memory

CP
L2
U Memory

Tavg=

L3
Memory

CSE-4293 Computer
 Hierarchical
Memory CP
Organization
U

L1
Memory
Tavg=

L2
Memory

L3
Memory
CSE-4293 Computer
 Problem:
1
Consider a system with 2-level caches. Access times of L1 and L2 are 1ns, 10ns. Hit ratio
of
L1 0.8. What is average time within Cache For..
i) Independent Organization
ii) Hierarchical Organization

CSE-4293 Computer
 Problem:
2
Consider a system with 3-level caches. Access times of L1, L2 and L3 are 1ns, 10ns,
100ns.
Hit ratio of L1 and L2 are 0.5 and 0.4. What is average time within Cache For..
i) Independent Organization
ii) Hierarchical Organization

CSE-4293 Computer
 Problem:
3
Consider a system with 2-level caches with a main memory. Access times of L1 and
L2 and main memory are 1ns, 10ns, 100ns. Hit ratio of L1 and L2 are 0.4 and 0.5.
What is average time of system ignoring search time within Cache.

CSE-4293 Computer
 Thank You

CSE-4293 Computer 4