Computer Organization & Architecture

Chapter Four

Memory System Organization and

Architecture

1 14 August 2023
 4.1 COMPUTER MEMORY SYSTEM
OVERVIEW
 Memory is used for storing programs and data that are required to
perform a specific task.
 Memory is an essential component of computers.
 Main memory is directly connected to the central processing unit via a
memory bus.
 The CPU continuously reads instructions stored there and executes
them as required.
 Typical computer system is equipped with a hierarchy of memory
subsystems.
1. Internal to the system (directly accessible by the processor)
2. External (accessible by the processor via an I/O module).
2 14 August 2023
4.1.1. Characteristics of Memory Systems
 Memory systems classified according to their key characteristics:
 Location
 Capacity
 Unit of transfer
 Access method
 Performance
 Physical type
 Physical characteristics
 Organization

3 14 August 2023
 Cont…

 Location: refers to whether it is in CPU, internal or external to the

computer
 Internal/main: example processor register, main memory, cache which are
directly accessible by CPU
 External/secondary: example optical disk, magnetic disk, taps which are
accessed by CPU through I/O modules
 CPU: In the form of CPU registers and Small amount of Cache
 Capacity:
 For internal memory typically expressed in bytes (8 bits) or words (8, 16, 32
bits)where as external store the programs and data in terms of bytes, KB,
MB…
 Unit of transfer:
4 Internal: the number of data lines into and out of memory,
14 AugustMay
2023 be equal to
 Cont…
 Related concepts
 Word: the “natural” unit of organization of memory, usually equal to
size of integer or instruction length
 Addressable units: Smallest location which can be uniquely
addressed
 Usually a word
 Some systems allow memory addressing at byte level
 For a length of A address bits, number of addressable units N is 2A
=N
 How many address line(bits) are needed to access 4GB RAM?
 4GB in byte is 4,294,967,296 or 232 bytes then one address line
5 32 address lines needed in byte 31 in word addressable
14 August 2023 (16 bit
 Cont…
 Access method:
 Sequential access:
 Access to records is made in a specific linear sequence
 Access time depends on location of data and current
location of read/write mechanism
 E.g. Magnetic tape, CD-ROM
 Direct access:
 Individual blocks have unique address
 Access by jumping to vicinity plus sequential search
 Access time – depends on location of the data and current
location of read/write mechanism, e.g. Disk

6 14 August 2023
 Cont…
 Random access:
 Individual addresses identify locations exactly
 Each location has its own (wired) addressing mechanism
 Access time is independent of location or previous access
 any location can be selected at random and directly addressed
and accessed, E.g. RAM
 Associative
 Data is located based on a portion of its content
 Comparison done simultaneously on all words
 Access time – independent of location or previous access
E.g. Cache
7 14 August 2023
 Cont…
 Physical type:
 A variety of physical types of memory have been employed
 the most common today are: semiconductor memory, magnetic
surface memory, and optical & magneto-optical memory
 Physical characteristics:
 volatility/ Non-volatility, erasable / read-only, power consumption
 Organization: is a key design issue and it is the physical
arrangement of bits to form words.

8 14 August 2023
 4.1.2. The Memory Hierarchy
 The design constraints on a computer’s memory can be summed up by
three questions
i. How much?
ii. How fast?
iii. How expensive?
 The memory unit that communicates with directly with CPU is called main
memory.
 Not enough storage space
 Contains programs and data currently needed are stored here

9 14 August 2023
 Cont…
 Devices that provide backup storage are called auxiliary memory.
 Most common are magnetic disks and tape drives.
 Used to store system programs, large data files, backup data
 Not urgently needed data are stored here
 Total memory capacity of a computer can be visualized as a
hierarchy of components.
 A variety of technologies are used to implement memory systems,
and across this spectrum of technologies, the following relationships
hold: :
o Smaller capacity, faster access time and greater costs per bit
o Greater capacity, slower access time and smaller costs per bit
10 14 August 2023
 The Computers' Memory Hierarchy

11 14 August 2023
 Memory Hierarchy

• Registers
• L1 Cache
• L2 Cache
• Main memory
• Disk Cache
• Disk
• Optical
• Tape
Access Cost per Capacity Frequency of
time bit access by
CPU
12
Main memory organization and operations
 The main memory is assumed at the second level of
the hierarchy.
 Types:
1. RAM (Random access memory)
 The main memory mainly consists of RAM, which is
available in static and dynamic mode.
 DRAM: Dynamic Random Access Memory
 High density, low power, cheap, slow
 Dynamic: need to be “refreshed” regularly
 Main Memory is DRAM
 SRAM: Static Random Access Memory
 Low density, high power, expensive, fast
 Static: content will last “forever”(until lose power)
 Cache uses SRAM

13 14 August 2023
 Comparison Chart

14 14 August 2023
 Performance of Main Memory:
 A good indication of the performance is given by two
parameters: latency and bandwidth.
 The term memory latency is used to refer to the amount of
time it takes to transfer a word of data to or from the memory.
 In block transfers, the term latency is used to denote the time
it takes (how long) to transfer first word of data
 When transferring blocks of data, it is of interest to know how
much time is needed to transfer an entire block
 Since blocks can be variable in size, it is useful to define a
performance measure in terms of the number of bits or bytes
that can be transferred in one second.
 This measure is often referred to as the memory bandwidth

15 14 August 2023
…Continued
 Access time (latency)
 For random-access memory: time between presenting the address
and getting the valid data
 For non-random-access memory: time to position the read/write
mechanism
 Memory cycle time
 Applies to random-access memory
 Time may be required for the memory to “recover” before next access
 Represents (access time + recovery time)
 Transfer rate: rate at which data can be moved into/out of a memory unit

16 14 August 2023
2. ROM (Read only memory)
 ROM is used for storing programs that are Permanently
resident in the computer.
 The ROM portion of main memory is needed for storing
an initial program called bootstrap loader, which is to
start the computer software operating when power is
turned on.
 There are five basic ROM types:
• ROM - Read Only Memory
• PROM - Programmable Read Only Memory
• EPROM - Erasable Programmable Read Only Memory
• EEPROM - Electrically Erasable Programmable Read Only
Memory
• Flash EEPROM memory standalone memory storage.
17 14 August 2023
 4.2. Cache Memory
 Has small storage capacity/amount and fast speed of memory

 Conceptually, it is found between main memory and CPU

 Stores data from some frequently used addresses(of main

memory).

 May be located on CPU chip or module

 reducing the total execution time of the program

Figure 4.2: Single Cache

18 14 August 2023
 …Continued
• The basic characteristic of cache memory is its fast access time.
• Therefore, very little or no time must be wasted when searching the words
in the cache
 When CPU needs to access memory, the cache is examined .
 If the word is found in the cache, it is read from the fast memory called Hit
.
 If the word addressed by the CPU is not found in the cache, the main
memory is accessed to read the word called Miss.
 The performance of cache memory is frequently measured in terms of a
quantity called hit ratio
 Hit ratio = hit / (hit+miss)
 Miss ratio= 1 - hit ratio
• Processor loads data from Memory and copies into cache.
• This results in extra delay, called miss penalty.
19 14 August 2023
 Average Memory Access Time

 Example 1, find performance of two levels of

memory:
 L1: 1,000 words, access time T1 = 0.01 μs
 L2: 100,000 words, access time T2 = 0.1 μs
 AMAT=(hit time)+(Miss rate)*(Miss penalty)
 Hit ratio H = fraction of all memory accesses
where the word is found in faster memory (L1)
 Assume H = 95%, then the average access time
is:
0.95 * 0.01 μs + 0.05 * (0.01 μs + 0.1 μs)
= 0.015 μs
20 14 August 2023
 Example 2
 A computer system has a MM access time of 200 ns
 supported by a cache having a 10 ns access time and a
hit rate of 99%.
 Suppose access to cache and main memory occurs
concurrently. (The accesses overlap.)
 The AMAT is:
 0.99(10 ns) + 0.01(200 ns) = 9.9 ns + 2 ns = 11.9 ns (The
accesses not overlap. )
 0.99(10 ns) + 0.01(10 ns + 200 ns)
 = 9.9 ns + 2.01 ns = 12 ns.

21 14 August 2023
 Cache/Main Memory Structure

Figure 4.3: Cache/Main Memory

Structure

22 14 August 2023
Figure 4.3 depicts the structure of a cache/main-memory system

 Main memory consists of up to 2n addressable words, with each word

having a unique n-bit address
 For mapping purposes, this memory is considered to consist of a number
of fixed-length blocks of K words each (i.e., there are M = 2n/K blocks in
main memory).
 The cache consists of m blocks, called lines, each line contains K words,
plus a tag of a few bits.
 The length of a line, not including tag and control bits, is the line size
example 32 bits or 4 bytes
 The number of lines is considerably less than the number of main memory
blocks (m << M)
 Tags – identify which particular block of main memory is in each cache line

23
currently stored 14 August 2023
 4.2.1. Cache Operation
 CPU requests contents of memory location

 Check cache for this data

 If present

• get from cache (fast)

 If not present

• read required block from main memory to cache

• deliver data to CPU

24 14 August 2023
 4.2.2. Elements of Cache Design
 The following are the elements of cache design issues/parameters:
 Cache Size
 Mapping Function(direct, associative, set associative)
 Replacement Algorithm(LRU, LFU, FIFO, random)
 Write Policy(write through, and write back)
 Line Size
 Cache Size
 Factors that influence cache size:
 Cost and
 Speed which are restricted to between 8kb to 64kb for L1.

25 14 August 2023
 Mapping Function

26 14 August 2023
 Cont…
• The main memory consists of 16 Mbytes, with each byte directly
addressable by a 24-bits address (224 =16M), thus, for mapping
purpose, we can consider main memory to consists of 4M
blocks of 4 bytes each.

• Main memory size = 16 MB = 224 bytes

• Address length = 24 bits

• Number of blocks in main memory = 224 / 22 = 222 blocks

27 14 August 2023
 Direct Mapping
 If a block of memory from the main memory can be placed in
exactly one place, we have a cache which is direct-mapped.
 Each block of main memory maps to only one cache line
 Mapping is expressed as: i = j modulo m
Where
i = cache line number
j = main memory block number
m = number of lines in cache
 The mapping function: implemented by the address of main
memory
 For example, let 16-byte main memory and a 4-byte cache (four
1-byte blocks).
 For instance, with the four-block cache here, address 14
would map to cache block 2. 14 mod 4 = 2
 Memory locations 0, 4, 8 and 12 all map to cache block 0.
 Addresses 1, 5, 9 and 13 map to cache block 1, etc

28 14 August 2023
Figure 4.7: Direct Mapping

29 14 August 2023
 Example of Direct mapping
Select this paragraph to edit

30 14 August 2023
 Example of Direct mapping

31 14 August 2023
 Full Associative Mapping
 If the block can be placed anywhere, the cache is fully
associative.

 A main memory block can load into any line of cache.

 Any block from memory can be put in any free cache block
(i.e. no restriction)

32 14 August 2023
 Fully Associative Implementation

33 14 August 2023
Example of Full Associative mapping

34 14 August 2023
 Set Associative Mapping
 If there are a restricted set of places that the block can be
placed, the cache is set associative.
 A compromise that provides strengths of both direct and
associative approaches
 Cache is divided into a number of sets of lines Each set
contains a fixed number of lines
 A given block maps to any line in a given set determined
by that block’s address
 Given, S sets, block i of MM maps to set i mod s
 Within the set, block can be put anywhere
 Let k = number of cache blocks per set = n/s
 K comparisons required for search

35 14 August 2023
Select this paragraph to edit

36 14 August 2023
Sample of 2-ways set associative Cache Map

37 14 August 2023
Example of 2-way Set Associative Cache Map
Select this paragraph to edit

38 14 August 2023
 Replacement Algorithm
 On a miss block during cache memory is full, a new block must be
brought in.
 For full-associative and set associative mapping replacement
algorithm is needed:
 Least Recently Used (LRU): Replace that block in the set that
has been in the cache longest with no reference to it.
 First-In-First-Out: Replace block that has been in the cache
longest
 Random replacement: Not based on usage, pick a line at
random from among the candidate lines

39 14 August 2023
Example On Replacement Algorithms

40 14 August 2023
 Write Policy
 If a block has been modified in cache, when should we update the
corresponding block in main memory?
 Write through:
 All writes done in both cache and main memory
 Update both levels of hierarchy
 Later when block is evicted, no write back is needed to
memory.
 Write back:
 Writes are done in cache only, minimize writes in memory
 Update only cached copy
 Processor can continue quickly
41  Later when block is evicted, entire block is14 written back to
August 2023
 Number of Caches
 Increases in circuit densities → caches can be placed inside
processor chip
 Internal caches
 Very short data paths (within the chip)
 Much faster access than through bus
 Multi-level caches
 L1, L2, L3, …
 L1 on-chip, L2 on-board, etc.

42 14 August 2023
 4.3. Virtual Memories
 Virtual memory allows a program to treat its memory
space as single contiguous block that may be
considerably larger than main memory.
 A memory management unit takes care of the mapping
between virtual and physical addresses
 CPU and OS translate virtual addresses to physical
addresses by MMU.
 VM “block” is called a page.
 VM translation “miss” is called a page fault.

43 14 August 2023
 Page Fault
 When a program generates an access request to a page
that is not in the main memory, a page fault is said to
have occurred.
 The whole page must be brought from the disk into the
memory before access can proceed
 When it detects a page fault, the MMU asks the operating
system to intervene by raising an exception (interrupt).
 Page tables are used to translate the virtual addresses
seen by application in to physical addresses used by
the hardware to process instruction.
44 14 August 2023
 Example of VM

45 14 August 2023
 External Memory
 External memory includes:
i. Magnetic disk
ii. Optical
iii. Magnetic tape

i. Magnetic Disk
 Disk substrate coated with magnetizable material (iron oxide…rust)
 Substrate used to be aluminium or aluminium alloy materials, but now
glass substrates are introduce
 Glass substrate has the benefits
 Improved in surface uniformity- to increases disk reliability
 Reduction in surface defects- helps to reduce read/write errors
 Better stiffness to reduce disk dynamics
 Better ability to withstand shock and damage
46 14 August 2023
Magnetic Read and Write Mechanisms
 Recording on and retrieval data from disk operates via conductive
coil(s) called head(s)
 There are two heads: read head and write heads in many systems
 During read/write, head is stationary while platter rotates beneath
head
 Write
 Current through coil produces magnetic field
 Pulses sent to head
 Magnetic pattern recorded on surface
 Read
 Magnetic field moving relative to coil produces current
• Analogous to a generator or alternator
 Coil can be the same for read and write
47 14 August 2023
Data organization and formatting
 Data is organized on the platter in set of concentric rings called
tracks
 Each track is the same width as the head and there are thousands
of tracks per surface
 Figure 4.13 depicts the data layout
 Adjacent tracks are separated by gaps – helps to minimize
errors due to misalignment of head or interface of magnetic
fields
 Data are transferred to/from disk in sectors – there are typically
hundreds of sectors per track may be fixed or variable in lengths

48 14 August 2023
 Cont…

Figure 4.13: Disk Data Layout

49 14 August 2023
 Cont…
 Concentric rings or tracks
 There are thousands of tracks per platter surface
 Each track is the same width as head
 Gaps between tracks (intertrack gaps)
• protect data integrity by preventing or at least minimizes,
errors due to misalignment of head or interference from other
tracks
 Reduce gaps between tracks
 Increase capacity

50 14 August 2023
 Cont…
 Tracks divided into sectors
 Minimum block size is one sector
• may have more than one sector per block
 Typically hundreds of sectors per track
 May be fixed or variable in length
• Contemporary systems are fixed-length sectors are used,
typically with 512 bytes
 Gaps between sectors (intratrack or intersector) gaps
 Disk velocity – an information can be scanned at the same rate by
rotating the disk at a fixed speed
 Helps to track and sectors can directly addressed an individual blocks
51of data 14 August 2023
 Cont…
 since, bits are not read at the same rate, the solutions is using:
• Constant angular velocity (CAV)
• Multiple zone recording
 Multiple zone recording
• Divide disk into concentric zones – typical number is 16
• Each zone has fixed number of bits (and sectors) per track
• More complex circuitry to adjust for different data rates as heads
move farther out

52 14 August 2023
 Cont…
 Disk layout methods

Figure 4.14: Comparison of Disk Layout

53
Methods 14 August 2023
 Cont…
 Formatting
 Two kinds of formatting
• Low level – allows hard drive to find tracks and sectors
• High or O/S level – allows for file system
 File system cab be:
 FAT
 FAT32
 NTFS

54 14 August 2023
 Cont…
 Physical characteristics of disk system are:
 Head Motion
 Disk Portability
 Sides
 Platters
 Head Mechanism

55 14 August 2023
 Cont…
 Head motion  Disk Portability
 Fixed head  Removable disk
 Can be removed from drive and
 One read-write head per replaced with another disk
track  Provides unlimited storage
capacity
 Heads mounted on fixed arm  Disk can moved from one
 Movable head computer system to another
 Examples: floppy, ZIP, Jazz
 One read-write head per  Non-removable disk
side  Permanently mounted in the disk
drive
 Mounted on a movable arm
 Example: hard disk of a PC
 Heads move radially across
tracks

56 14 August 2023
 Cont…
 Sides and platters
Sides
Single sided
Double sided
Platters
Single platter
Multiple platter
•One read-write head per platter surface
•Heads are joined and aligned
•Aligned tracks on each platter form cylinders
•Data is striped by cylinder
57 14 August 2023
Disk performance parameters
 To read or write, the head must be positioned at the desired track
and at the beginning of the desired sector on that track
 Track selection involves moving the head in a movable head system
or electronically selecting one head on a fixed-head system
 Seek time - is the time taken to moving head to required track
 Rotational delay/latency – is the time it takes for the beginning of
the sector to reach the head
 Access time – is the time it takes to get into position to read/write
Access time = Seek time + Rotational delay
 Transfer time – is the actual time to transfer data

58 14 August 2023
Optical Storage
 CD-ROM
 is a non-erasable disk that stores digitized audio information
 680 MB giving over 70 minutes audio
 Constant linear velocity

59 14 August 2023
Magnetic Tape
 Magnetic Tape are: strip of plastic coated with magnetic recording
material

 Bits are recorded as magnetic spots in parallel tracks.

Eg. 9 tracks: 8 data bits + 1 bit parity
 Serial: Data is written as sequence of bits along each track

− Typical – serpentine recording

− Follow one track until end of tape, then continue (back)

on next track
 are slow and very cheap

60 14 August 2023
 Optical Storage Summary
Compact Disk (CD)- is a non-erasable disk that stores digitized audio information

Compact Disk Read-Only Memory (CD-ROM)- is a non-erasable disk used for

storing computer data, can holds more than 650 Mbytes

CD Recordable (CD-R) – is similar to CD-ROM and users can write to the disk
only once
CD Rewritable (CD-RW) –is similar to a CD-ROM and user can erase and rewrite
to the disk multiple times

Digital Versatile Disk (DVD) – is used to store a digitized, compressed

representation of video information and other large volume of digital data

DVD Recordable (DVD-R)- users can write to the disk only once and used only
one-sided disk

DVD Rewritable (DVD-RW) – users can erase and rewrite to the disk multiple
times, used one-side disks

61 14 August 2023