CS.

305
Computer Architecture
<local.cis.strath.ac.uk/teaching/ug/classes/CS.305>
Enhancing Performance with
Pipelining
Adapted from Computer Organization and Design, Patterson & Hennessy, 2005,
and from slides kindly made available by Dr Mary Jane Irwin, Penn State University.
Enhancing Performance with Pipelining CS305_07/2
Recap: Single Cycle vs. Multiple Cycle Timing
Clk Cycle 1
Multiple Cycle Implementation:
IFetch Dec Exec Mem WB
Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7 Cycle 8 Cycle 9Cycle 10
IFetch Dec Exec Mem
lw sw
IFetch
R-type
Clk
Single Cycle Implementation:
lw sw Waste
Cycle 1 Cycle 2
multicycle clock
slower than 1/5
th
of
single cycle clock
due to stage register
overhead
Enhancing Performance with Pipelining CS305_07/3
How Can We Make It Even Faster?
! Split the multiple instruction cycle into smaller and smaller steps
" There is a point of diminishing returns where as much time is spent
loading the state registers as doing the work
! Start fetching and executing the next instruction before
the current one has completed
" Pipelining (all?) modern processors are pipelined for
performance
" Remember the performance equation:
CPU time = CPI * CC * IC
! Fetch (and execute) more than one instruction at a time
" Superscalar processing stay tuned
Enhancing Performance with Pipelining CS305_07/4
A Pipelined MIPS Processor
! Start the next instruction before the current one has
completed
" improves throughput - total amount of work done in a given time
" instruction latency (execution time, delay time, response time -
time from the start of an instruction to its completion) is not
reduced
Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5
IFetch Dec Exec Mem WB lw
Cycle 7 Cycle 6 Cycle 8
sw
IFetch Dec Exec Mem WB
R-type
IFetch Dec Exec Mem WB
clock cycle (pipeline stage time) is limited by the slowest stage
for some instructions, some stages are wasted cycles
Enhancing Performance with Pipelining CS305_07/5
Single Cycle, Multiple Cycle, vs. Pipeline
Multiple Cycle Implementation:
Clk
Cycle 1
IFetch Dec Exec Mem WB
Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7 Cycle 8 Cycle 9Cycle 10
IFetch Dec Exec Mem
lw sw
IFetch
R-type
lw IFetch Dec Exec Mem WB
Pipeline Implementation:
IFetch Dec Exec Mem WB sw
IFetch Dec Exec Mem WB R-type
Clk
Single Cycle Implementation:
lw sw Waste
Cycle 1 Cycle 2
Enhancing Performance with Pipelining CS305_07/6
MIPS Pipeline Datapath Modifications
! What do we need to add/modify in our MIPS datapath?
" State registers between each pipeline stage to isolate them
Read
Address
Instruction
Memory
Add
P
C
4
Write Data
Read Addr 1
Read Addr 2
Write Addr
Register
File
Read
Data 1
Read
Data 2
16 32
ALU
Shift
left 2
Add
Data
Memory
Address
Write Data
Read
Data I
F
e
t
c
h
/
D
e
c
D
e
c
/
E
x
e
c
E
x
e
c
/
M
e
m
M
e
m
/
W
B
IF:IFetch ID:Dec EX:Execute MEM:
MemAccess
WB:
WriteBack
System Clock
Sign
Extend
Enhancing Performance with Pipelining CS305_07/7
Pipelining the MIPS ISA
! What makes it easy
" all instructions are the same length (32 bits)
can fetch in the 1
st
stage and decode in the 2
nd
stage
" few instruction formats (three) with symmetry across formats
can begin reading register file in 2
nd
stage
" memory operations can occur only in loads and stores
can use the execute stage to calculate memory addresses
" each MIPS instruction writes at most one result (i.e.,
changes the machine state) and does so near the end of the
pipeline (MEM and WB)
! What makes it hard
" structural hazards: what if we had only one memory?
" control hazards: what about branches?
" data hazards: what if an instructions input operands depend
on the output of a previous instruction?
Enhancing Performance with Pipelining CS305_07/8
Graphically Representing MIPS Pipeline
! Can help with answering questions like:
" How many cycles does it take to execute this code?
" What is the ALU doing during cycle 4?
" Is there a hazard, why does it occur, and how can it be fixed?
A
L
U
IM Reg DM Reg
Enhancing Performance with Pipelining CS305_07/9
Why Pipeline? For Performance!
I
n
s
t
r.
O
r
d
e
r
Time (clock cycles)
Inst 0
Inst 1
Inst 2
Inst 4
Inst 3
A
L
U
IM Reg DM Reg
A
L
U
IM Reg DM Reg
A
L
U
IM Reg DM Reg
A
L
U
IM Reg DM Reg
A
L
U
IM Reg DM Reg
Once the
pipeline is full,
one instruction
is completed
every cycle, so
CPI = 1
Time to fill the pipeline
Enhancing Performance with Pipelining CS305_07/10
Can Pipelining Get Us Into Trouble?
! Yes: Pipeline Hazards
" structural hazards: attempt to use the same resource by two
different instructions at the same time
" data hazards: attempt to use data before it is ready
An instructions source operand(s) are produced by a prior
instruction still in the pipeline
" control hazards: attempt to make a decision about program
control flow before the condition has been evaluated and the
new PC target address calculated
branch instructions
! Can always resolve hazards by waiting
" pipeline control must detect the hazard
" and take action to resolve hazards
Enhancing Performance with Pipelining CS305_07/11
I
n
s
t
r.
O
r
d
e
r
Time (clock cycles)
lw
Inst 1
Inst 2
Inst 4
Inst 3
A
L
U
Mem Reg Mem Reg
A
L
U
Mem Reg Mem Reg
A
L
U
Mem Reg Mem Reg
A
L
U
Mem Reg Mem Reg
A
L
U
Mem Reg Mem Reg
Single Memory Would Be a Structural Hazard
Reading data from
memory
Reading instruction
from memory
! Fix with separate instr and data memories (I$ and D$)
Enhancing Performance with Pipelining CS305_07/12
How About Register File Access?
Time (clock cycles)
I
n
s
t
r.
O
r
d
e
r
add $1,
Inst 1
Inst 2
add $2,$1,
A
L
U
IM Reg DM Reg
A
L
U
IM Reg DM Reg
A
L
U
IM Reg DM Reg
A
L
U
IM Reg DM Reg
Enhancing Performance with Pipelining CS305_07/13
How About Register File Access?
I
n
s
t
r.
O
r
d
e
r
Time (clock cycles)
Inst 1
Inst 2
A
L
U
IM Reg DM Reg
A
L
U
IM Reg DM Reg
A
L
U
IM Reg DM Reg
A
L
U
IM Reg DM Reg
Fix register file
access hazard by
doing reads in the
second half of the
cycle and writes in
the first half
add $1,
add $2,$1,
clock edge that controls
register writing
clock edge that controls
loading of pipeline state
registers
Enhancing Performance with Pipelining CS305_07/14
Register Usage Can Cause Data Hazards
I
n
s
t
r.
O
r
d
e
r
add $1,
sub $4,$1,$5
and $6,$1,$7
xor $4,$1,$5
or $8,$1,$9
A
L
U
IM Reg DM Reg
A
L
U
IM Reg DM Reg
A
L
U
IM Reg DM Reg
A
L
U
IM Reg DM Reg
A
L
U
IM Reg DM Reg
! Dependencies backward in time cause hazards
! Read before write data hazard
Enhancing Performance with Pipelining CS305_07/16
Loads Can Cause Data Hazards
I
n
s
t
r.
O
r
d
e
r
lw $1,4($2)
sub $4,$1,$5
and $6,$1,$7
xor $4,$1,$5
or $8,$1,$9
A
L
U
IM Reg DM Reg
A
L
U
IM Reg DM Reg
A
L
U
IM Reg DM Reg
A
L
U
IM Reg DM Reg
A
L
U
IM Reg DM Reg
! Dependencies backward in time cause hazards
! Load-use data hazard
Enhancing Performance with Pipelining CS305_07/17
stall
stall
One Way to Fix a Data Hazard
I
n
s
t
r.
O
r
d
e
r
add $1,
A
L
U
IM Reg DM Reg
sub $4,$1,$5
and $6,$1,$7
A
L
U
IM Reg DM Reg
A
L
U
IM Reg DM Reg
Can fix data
hazard by
waiting stall
but impacts CPI
Enhancing Performance with Pipelining CS305_07/18
Another Way to Fix a Data Hazard
I
n
s
t
r.
O
r
d
e
r
add $1,
A
L
U
IM Reg DM Reg
sub $4,$1,$5
and $6,$1,$7
A
L
U
IM Reg DM Reg
A
L
U
IM Reg DM Reg
Fix data hazards
by forwarding
results as soon as
they are available
to where they are
needed
xor $4,$1,$5
or $8,$1,$9
A
L
U
IM Reg DM Reg
A
L
U
IM Reg DM Reg
Enhancing Performance with Pipelining CS305_07/20
Forwarding with Load-use Data Hazards
I
n
s
t
r.
O
r
d
e
r
lw $1,4($2)
sub $4,$1,$5
and $6,$1,$7
xor $4,$1,$5
or $8,$1,$9
A
L
U
IM Reg DM Reg
A
L
U
IM Reg DM Reg
A
L
U
IM Reg DM Reg
A
L
U
IM Reg DM Reg
A
L
U
IM Reg DM Reg
Enhancing Performance with Pipelining CS305_07/22
Branch Instructions Cause Control Hazards
I
n
s
t
r.
O
r
d
e
r
lw
Inst 4
Inst 3
beq
A
L
U
IM Reg DM Reg
A
L
U
IM Reg DM Reg
A
L
U
IM Reg DM Reg
A
L
U
IM Reg DM Reg
! Dependencies backward in time cause hazards
Enhancing Performance with Pipelining CS305_07/23
One Way to Fix a Control Hazard
stall
stall
stall
I
n
s
t
r.
O
r
d
e
r
beq
A
L
U
IM Reg DM Reg
lw
A
L
U
IM Reg DM Reg
A
L
U
Inst 3
IM Reg DM
Fix branch
hazard by
waiting
stall but
affects CPI
Enhancing Performance with Pipelining CS305_07/24
Corrected Datapath to Save RegWrite Addr
! Need to preserve the destination register address in the
pipeline state registers
Read
Address
Instruction
Memory
Add
P
C
4
Write Data
Read Addr 1
Read Addr 2
Write Addr
Register
File
Read
Data 1
Read
Data 2
16 32
ALU
Shift
left 2
Add
Data
Memory
Address
Write Data
Read
Data
IF/ID
Sign
Extend
ID/EX EX/MEM
MEM/WB
Enhancing Performance with Pipelining CS305_07/26
MIPS Pipeline Control Path Modifications
! All control signals can be determined during Decode
" and held in the state registers between pipeline stages
Read
Address
Instruction
Memory
Add
P
C
4
Write Data
Read Addr 1
Read Addr 2
Write Addr
Register
File
Read
Data 1
Read
Data 2
16 32
ALU
Shift
left 2
Add
Data
Memory
Address
Write Data
Read
Data
IF/ID
Sign
Extend
ID/EX
EX/MEM
MEM/WB
Control
Enhancing Performance with Pipelining CS305_07/27
Other Pipeline Structures Are Possible
! What about the (slow) multiply operation?
" Make the clock twice as slow or
" let it take two cycles (since it doesnt use the DM stage)
A
L
U
IM Reg DM Reg
MUL
A
L
U
IM Reg DM1 Reg DM2
! What if the data memory access is twice as slow as
the instruction memory?
" make the clock twice as slow or
" let data memory access take two cycles (and keep the same
clock rate)
Enhancing Performance with Pipelining CS305_07/28
Sample Pipeline Alternatives
! ARM7
! StrongARM-1
! XScale
A
L
U
IM1 IM2 DM1
Reg
DM2
IM Reg
EX
PC update
IM access
decode
reg
access
ALU op
DM access
shift/rotate
commit result
(write back)
A
L
U
IM Reg DM Reg
Reg
SHFT
PC update
BTB access
start IM access
IM access
decode
reg 1 access
shift/rotate
reg 2 access
ALU op
start DM access
exception
DM write
reg write
Enhancing Performance with Pipelining CS305_07/29
Summary
! All modern day processors use pipelining
! Pipelining doesnt help latency of single task, it helps
throughput of entire workload
! Potential speedup: a CPI of 1 and fast a CC
! Pipeline rate limited by slowest pipeline stage
" Unbalanced pipe stages makes for inefficiencies
" The time to fill pipeline and time to drain it can impact
speedup for deep pipelines and short code runs
! Must detect and resolve hazards
" Stalling negatively affects CPI (makes CPI less than the ideal
of 1)