1

Computer Memory
CHAPTER 03
 2

• Key characteristics
Overview • Design Constraints
• Hierarchy
 3

Key characteristics
 4

Design Constraints
 How much (capacity) , how fast (access time), how expensive (cost),

 A variety of technologies are used to implement memory systems,

and across this spectrum of technologies, the following relationships
hold:

 Faster access time, greater cost per bit

 Greater capacity, smaller cost per bit

 Greater capacity, slower access time

 5

Memory Hierarchy

Moving down the hierarchy:

• Decreasing cost per bit
• Increasing capacity
• Increasing access time
• Decreasing frequency of access
of the memory by processor
 6

• Locality of reference
• Cache mapping schemes
Cache • Replacement policies
memory • Cache write policies
 7

Locality of reference
 The instruction currently being fetched is very close in memory to the
instruction to be fetched next. The same idea applies to the data
value currently being accessed (read/written) in memory.

 Three basic forms:

 Temporal locality - Recently accessed items tend to be accessed again in

the near future.

 Spatial locality - Accesses tend to be clustered in the address space

(arrays or loops).

 Sequential locality - Instructions tend to be accessed sequentially.

 The locality principle provides the opportunity for a system to use a

small amount of very fast memory to effectively accelerate the
majority of memory accesses.
 8

Cache Memory Principles

 Cache memory is designed to combine the memory access time of
expensive, high speed memory combined with the large memory size
of less expensive, lower-speed memory.

 Cache is faster than main memory maximize it utilization

 Cache is more expensive than main memory it is smaller
 9

Multiple level cache

 The size of cache memory can vary and can be organized into
multiple levels.

 Level 1 (L1) cache is smaller, typically 8K or 16K. L1 cache resides on

the processor,

 Level 2 (L2) cache is 256K or 512K. L2 cache resides between the

CPU and main memory.
 11

Cache Dynamics - Read

 When a memory read (or fetch) is issued by the CPU:

1. If the line with that memory address is in the cache (hit), the data is
read from the cache to the CPU.

2. If the line with that memory address is not in the cache (a miss), the
cache is updated by replacing one of its active lines by the line with
that memory address, and then the data is read from the cache to
the CPU.
 12

Cache Dynamics - Write

 When a memory write is issued by the CPU:

1. If the line with that memory address is in the cache, the data is
written to the cache, and the line is marked "invalid" (no longer is an
image of the corresponding memory line).

2. If the line with that memory address is not in the cache, the cache is
updated by replacing one of its active lines by the line with that
memory address. The data is then written from the CPU to the cache
and the line is marked "invalid.”

In both cases, the recently updated line in cache memory is no longer

valid (exact image of its corresponding lines in main memory)
 13

Cache mapping schemes

 There mapping techniques/Block replacement are available:

1. Direct Mapping:
2. Associative mapping
3. Set-Associative Mapping

 Mapping memory Blocks to Cache lines

Suppose:
 A cache has 24 = 16 lines, each with 23 = 8 bytes ➔ Cache size = 128 Bytes
 The main memory 256 bytes in the memory's address space
--> there are 8 bit addresses
 14

Direct mapping
 Each block has only one place it can appear in the cache.

16 lines = 24 = 128 words cache

Cache line (8 words)
1st 16 blocks of
...
8 words each
...

...

Address
lines 2nd 16 blocks of
8 words each

Tag Line (4) words (3)

 15

Cache mapping schemes

Suppose the CPU needs to read/write a word at 6A:
 Binary address = 48 = 0 1101 010
Address in cache memory : Tag = 0, Line = 1101 (13rd), Word = 010 (2nd )
Address the main memory: Block = 01101 (13), Word #: 2
 16

Direct Mapping
 Advantage: Simple and inexpensive to implement.

 Main disadvantage:
There is a fixed cache location for any given block → if a program
happens to reference words repeatedly from two different blocks
that map into the same line → the blocks will be continually swapped
in the cache → the hit ratio will be low (thrashing).
 17

Associative mapping
 Any block of main memory can be placed anywhere in the cache
memory.
 The fastest and most flexible mapping technique
 18

Set-Associative mapping
 Cache blocks are divided into sets. Set size is in the power of 2
 Example: the cache has 2 blocks --> 2-way set associative, the cache
has 4 blocks --> 4-way set associative
 20

The usefulness of a cache

 Consider a program loop which consists of 100 instructions, the loop is
executed for 1000 times; each instruction is stored in 1 memory
location.
 Suppose memory speed = 50ns to fetch one instruction, cache speed
= 5ns to fetch an instruction.
 Calculate the time needed to fetch instructions to execute the loop in
both case with and without cache memory?
 21

The hit, cache miss ratio

 Cache hit = a memory request that can be satisfied by the cache
 Cache hit ratio = #cache hit/#memory request
Calculate cache hit ratio for the above example
 22

Cache write policies

 If a cache block is modified, the cache write policy determines when
the actual main memory block is updated to match the cache block.
 There are two basic write policies:
 Write-through: updates both the cache and the main memory
simultaneously on every write.
 Write-back (copy back): only update blocks in main memory when the
cache block is selected as victim and must be removed from cache.
 23

Intel Core i7 Cache Hierarchy

 L1 I-Cache & D-Cache
 32KB, 8-way, access: 4 cycles

 L2 (unified)
I-Cache & D-Cache
 256 KB, 8-way, 11 cycles

 L3 (unified)
8 MB, 16-way, 30-40 cycles
 24

• Semiconductor main memory

• Types of ROM
Internal • Types of RAM
Memory • Chip logic
• Chip Packaging
 25

Organization
 The basic element of a semiconductor memory is the memory cell
which are:
 Two stable states, which can be used to represent binary 1 and 0.
 Capable of being written into (at least once), to set the state.
 Capable of being read to sense the state.
 26

Types of ROM
 ROM contain permanent pattern of data that can not be changed.

 ROM is non-volatile, it is created like any other integrated circuit chip,

with the data actually wired into the chip in the fabrication process.

 PROM (programmable ROM): may be written only once.

 EPROM (Erasable PROM): read and write electrically (old data must
be erased by exposing the package to an intense UV radiation).

 EEPROM (Electrically EPROM): data can be written into at any time

without earasing prior contents. The write operation takes
considerably longer than the read operation (several hundred
microseconds/byte).

 Flash Memory is EEPROM which can be erased entirely in a few secs.

 27

Types of RAM
 DRAM (Dynamic RAM): is made with cells that store data as charge
on capacitors. Because capacitors have a natural tendency to
discharge →require periodic charge refreshing to maintain data.

 SRAM (Static RAM): is a digital device that uses the same logic
elements used in the processor. In a SRAM, values are stored using
traditional FF logic-gate → hold its data as long as power is supplied.

 Both static and dynamic RAMs are volatile.

 A dynamic memory cell is simpler and smaller than a static memory
cell → DRAM is smaller cells and less expensive than a corresponding
SRAM.
 On the other hand, a DRAM requires the supporting refresh circuitry.
 SRAM: Cache memory, DRAM: Main memory
 28

Chip Logic
 Semiconductor memory comes in
packaged chips. Each chip
contains an array of memory cells.
The array is organized into W
words of n bits each.

 For example, a 16-Mbit chip could

be organized as 1Mx16-bit words
or 4Mx4-bit words
 29

Example: 1Mx8 bits RAM


 30

Example: 1Mx16-bits RAM


 31

Example: 1Mx16-bits RAM


 32

Example: 4Mx16-bits RAM


 33

Example: 8Mx8-bits RAM


 34

Overall Structure

 35

Chip Packaging example

 16-Mbit chip organized as 4Mx4-bits, standard chip package 24 pins
 37

• Magnetic disk
External • Solid State Drives
Memory • Optical Memory
 38

Construction
 A disk is a circular platter constructed of nonmagnetic material,
called the substrate, coated with a magnetizable material.

 Traditionally, the substrate has been an aluminum or aluminum alloy

material. Recently, glass substrates have been introduced.

 Benefits of glass substrates:

 Improvement in the uniformity of the magnetic film surface to

increase disk reliability,
 A significant reduction in overall surface defects to help reduce read-write
errors,

 Ability to support lower fly heights

 Better stiffness to reduce disk dynamics

 Greater ability to withstand shock and damage

 39

Construction
 For most disks, the magnetizable coating is applied to both sides of
the platter (double sided). Some less expensive disk systems use
single-sided disks.

 Hard disks accommodate multiple platters stacked vertically a

fraction of an inch apart with multiple head arms mechanism.
 40

Data organization and Formatting

 Data stores on the disk
platter in a concentric set
of rings, called tracks. Each
track is the same width as
the head. There are
thousands of tracks per
surface.

 In hard disks which have

multiple platters, the set of
all tracks in the same
relative position on the
platter is referred to as
Cylinder
 41

Performance parameters
 Seek time: the time required to move the disk arm to the required
track = Startup time + Traverse time + Settle time.

 For common disk size today (3.5’ = 8.9 cm), typical average seek time
is under 10ms.

 Rotational Delay = 1/2r (r: rotation speed in revolution per second)

 Transfer time:
 42

Timing comparison
 Consider a disk with an advertised average seek time of 4 ms,
rotation speed of 15,000 rpm, and 512-byte sectors with 500 sectors
per track. Suppose that we wish to read a file consisting of 2500
sectors for a total of 1.28 Mbytes.

Case 1: The file stored sequentially on disk:

 The file occupies 5 adjacent tracks (5x500 sectores/track = 2500

sectors) → the time to read the first track:

 Each successive track is read in 2 + 4 = 6 ms

 To read the entire file: T = 10 + (4 x 6) = 34ms = 0.034 s

 43

Timing comparison
 Consider a disk with an advertised average seek time of 4 ms,
rotation speed of 15,000 rpm, and 512-byte sectors with 500 sectors
per track. Suppose that we wish to read a file consisting of 2500
sectors for a total of 1.28 Mbytes.

Case 2: The file stored randomly on disk:

 For each sector:

 To read the entire file: T = 2500 x 6008 = 15020ms = 15.02 s

 44

Timing comparison
 Consider a disk with an advertised average seek time of 4 ms,
rotation speed of 15,000 rpm, and 512-byte sectors with 500 sectors
per track. Suppose that we wish to read a file consisting of 2500
sectors for a total of 1.28 Mbytes.

Case 2: The file stored randomly on disk:

 For each sector:

 To read the entire file: T = 2500 x 6008 = 15020ms = 15.02 s

 45

RAID
 The rate of improvement in secondary storage performance has
been considerably less than the rate for processors and main
memory.

 The mismatch has made the disk storage system, the main focus of
concern in improving overall computer system performance ➔ using
parallel components (multiple disks)
 46

RAID
 Industry has agreed on a standardized scheme for multiple-disk
database design, known as RAID (Redundant Array of Independent
Disks). RAID scheme consists of 7 levels (0 → 6). All levels share three
common characteristics:

1. RAID is a set of physical disk drives viewed by the operating system as

a single logical drive.

2. Data are distributed across the physical drives of an array in a

scheme known as striping, described subsequently.

3. Redundant disk capacity is used to store parity information, which

guarantees data recoverability in case of a disk failure (except RAID
0,1)
47
 RAID Comparison 48

Level Advantages Disadvantages Applications

0 I/O performance is greatly improved The failure of just one Video production
by spreading the I/O load across drive will result in all and editing Image
many channels and drives No parity data in an array Editing Pre-press
calculation overhead is involved being lost applications Any
Very simple design Easy to application requiring
implement high bandwidth
1 100% redundancy of data means Highest disk Accounting Payroll
no rebuild is necessary in case of a overhead of all RAID Financial Any
disk failure, just a copy to the types (100%)— application requiring
replacement disk Under certain inefficient very high availability
circumstances, RAID 1 can sustain
multiple simultaneous drive failures
Simplest RAID storage subsystem
design
2 Extremely high data transfer rates Very high ratio of No commercial
possible The higher the data transfer ECC disks to data implementations
rate required, the better the ratio of disks with smaller exist/ not
data disks to ECC disks Relatively word sizes— commercially viable
simple controller design compared inefficient Entry level
to RAID levels 3, 4, & 5 cost very high—
requires very high
transfer rate
requirement to justify
 RAID Comparison 49

Level Advantages Disadvantages Applications

3 Very high read data transfer rate Transaction rate Video production
Very high write data transfer rate equal to that of a and live streaming
Disk failure has an insignificant single disk drive at Image editing Video
impact on throughput Low ratio of best (if spindles are editing Prepress
ECC (parity) disks to data disks synchronized) applications Any
means high efficiency Controller design is application requiring
fairly complex high throughput
5 Highest Read data transaction rate Most complex File and application
Low ratio of ECC (parity) disks to controller design servers Database
data disks means high efficiency Difficult to rebuild in servers Web, e-mail,
Good aggregate transfer rate the event of a disk and news servers
failure (as compared Intranet servers Most
to RAID level 1) versatile RAID level
6 Provides for an extremely high data More complex Perfect solution for
fault tolerance and can sustain controller design mission critical
multiple simultaneous drive failures Controller overhead applications
to compute parity
addresses is
extremely high
 50

• Flash memory
Solid State • SSD
Drive (SSD) • Practical issues with SSD
 51

Flash memory
 In recent years, the cost and performance of flash memory has

evolved to the point where it is feasible to use flash memory drives to

replace HDDs.

There are two types of flash memory NOR and NAND.

 In NOR flash memory, the basic access unit is a bit, and the logical

organization resembles a NOR logic device.

 NOR flash memory can read and write data to specific locations, and

can reference and retrieve a single byte. NOR flash memory is used

to store cell phone operating system code and on Windows

computers for the BIOS program that runs at startup

 52

Flash memory
 In NAND flash memory, the basic unit is 16 or 32 bits, and the
logical organization resembles NAND devices.
 NAND reads and writes in small blocks. It is used in USB flash
drives, memory cards (in digital cameras, MP3 players, etc.),
and in SSDs.
 NAND provides higher bit density than NOR and greater write
speed.
 NAND flash read data on a block-wise basis (also known as
page access), where each block holds hundreds to thousands
of bits.
 53

SSD advantages over HDD

 High-performance input/output operations per second (IOPS):
Significantly increases performance I/O subsystems.

 Durability: Less susceptible to physical shock and vibration.

 Longer lifespan: SSDs are not susceptible to mechanical wear.

 Lower power consumption: SSDs use as little as 2.1 watts of power per
drive, considerably less than comparable-size HDDs.

 Quieter and cooler running capabilities: Less floor space required,

lower energy costs, and a greener enterprise.

 Lower access times and latency rates: Over 10 times faster than the
spinning disks in an HDD
 54

Practical issues
Tend to slow down as the device is used

 Flash memory is accessed in blocks (size of 512 KB), there are typically
128 pages per block.

 Steps to write a page on a flash memory:

1. The entire block must be read from the flash memory and placed in a
RAM buffer. Then the appropriate page in the RAM buffer is updated.

2. Before the block can be written back to flash memory, the entire
block of flash memory must be erased—it is not possible to erase just
one page of the flash memory.

3. The entire block from the buffer is now written back to the flash
memory
 55

Practical issues
Becomes unusable after a certain number of writes

 As flash cells are stressed, they lose their ability to record and retain
values (typical limit is 100,000 writes)
 56

• Compact Disk
Optical • CD Recordable

Memory • CD Rewritable
• Digital Versatile Disk
 57

Audio CD and CD-ROM

 58

CD Recordable (CD-R)
 For CDs and CD-ROMs, information is recorded by the pitting of the
surface of the medium, which changes reflectivity.

 For a CD-R, the medium includes a dye layer. The dye is used to
change reflectivity and is activated by a high-intensity laser.
 59

CD Rewritable (CD-RW)
 The CD-RW optical disk can be repeatedly written and overwritten,
as with a magnetic disk.

 The optical approach has been used is phase change.

 The phase change disk uses a material that has two significantly
different reflectivities in two different phase states:
 Amorphous state, in which the molecules exhibit a random orientation
that reflects light poorly;

 Crystalline state, which has a smooth surface that reflects light well.

 A beam of laser light can change the material from one phase to
the other.

 Current materials can be used for between 500,000 and 1,000,000

erase cycles.
 60

Digital Versatile Disk (DVD)

 An acceptable replacement for the analog VHS video tape.

 Replace the CD-ROM in personal computers and servers.

 DVD takes video into the digital age.

 DVD’s greater capacity is due to:

 Bits are packed more closely on a DVD: The spacing between loops
of a spiral on a CD is 1.6 μm, the minimum distance between pits
along the spiral is 0.834 μm. The DVD uses a laser with shorter
wavelength and achieves a loop spacing of 0.74 μm and a
minimum distance between pits of 0.4 μm (4.7GB)

 Dual-layer DVD (8.5GB): the lasers in DVD drives can adjust focus to
read a semireflective layer on top of the reflective layer.