Locality of Reference

CPU performance is limited by slower main memory access. Cache memory improves speed by storing frequently used data and instructions. Its efficiency relies on the Locality of Reference principle.

• Locality of Reference means programs often reuse the same data or nearby instructions rather than accessing random memory locations.
• It helps cache memory predict which information will be needed next.
• This reduces the average memory access time and increases overall CPU efficiency.
• The concept is the foundation of cache memory design and operation.
• Programs that exhibit strong locality utilize cache memory more effectively, resulting in higher hit ratios and fewer cache misses.

Formal Definition

Locality of reference is the property of a program that describes how memory references are grouped together in time and space during execution.

Types of Locality of Reference

There are mainly two types of locality of reference:

Temporal Locality (Locality in Time)

Temporal locality states that if a particular memory location (data or instruction) is accessed at one time, it is likely to be accessed again in the near future.

• Programs often contain loops, counters, and frequently used variables.
• Since these are used repeatedly within short intervals, storing them in cache memory ensures faster access.

Example:

for (int i = 0; i < 100; i++) {    sum = sum + a[i];    }

• The variable i, sum, and elements of the array a[i] are accessed repeatedly.
• Storing these in the cache helps avoid multiple main memory accesses.

Cache Operation Based on Temporal Locality:

When a CPU fetches a data item from main memory, it also keeps a copy in the cache, assuming it will be needed soon. If that assumption is correct, subsequent accesses result in a cache hit.

Spatial Locality (Locality in Space)

Spatial locality states that if a particular memory location is accessed, the nearby memory locations are also likely to be accessed soon. Instructions and data are stored in contiguous memory locations.

• When a program accesses one location, the next few instructions or data items are often located adjacent to it.
• Hence, caching nearby blocks of memory improves performance.

Example:

for (int i = 0; i < 10; i++) {    printf("%d", arr[i]);    }

• Here, consecutive elements of the array arr[i] are accessed.
• Once arr[0] is fetched, the CPU is likely to access arr[1], arr[2], etc., which are stored in nearby memory locations.

Cache Operation Based on Spatial Locality:

Cache memory utilizes block fetching. When one word of a block is accessed, the entire block (containing nearby addresses) is loaded into cache, anticipating future accesses.

Mathematical Representation of Average Memory Access Time (AMAT)

The performance of a cache memory system is often evaluated using the Average Memory Access Time (AMAT). It represents the average time required by the CPU to access data or instructions, considering both cache hits and cache misses.

Let:

• T_c = Time to access cache memory
• T_m = Time to access main memory
• h = Cache hit ratio (the probability that the required data is found in the cache)

Then, the average memory access time can be expressed as:

Average Memory Access Time (AMAT)= h × T_c + (1−h) × T_m

Here,

• The term (h × T_c) represents the time contribution from cache hits.
• The term (1 − h) × T_m represents the time contribution from cache misses, where data must be fetched from main memory.

A higher hit ratio (h) indicates better cache utilization and stronger locality of reference, resulting in lower AMAT and improved overall system performance.

Locality of reference

Visit Course