CSE431 L06 Basic MIPS Pipelining.

1 Irwin, PSU, 2005

CSE 431
Computer Architecture
Fall 2005

Lecture 06: Basic MIPS Pipelining Review
Mary Jane Irwin ( www.cse.psu.edu/~mji )
www.cse.psu.edu/~cg431

[Adapted from Computer Organization and Design,
Patterson & Hennessy, 2005, UCB]
CSE431 L06 Basic MIPS Pipelining.2 Irwin, PSU, 2005
Review: Single Cycle vs. Multiple Cycle Timing
Clk Cycle 1
Multiple Cycle Implementation:
IFetch Dec Exec Mem WB
Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7 Cycle 8 Cycle 9 Cycle 10
IFetch Dec Exec Mem
lw sw
IFetch
R-type
Clk
Single Cycle Implementation:
lw sw
Waste
Cycle 1 Cycle 2
multicycle clock
slower than 1/5
th
of
single cycle clock
due to stage register
overhead
CSE431 L06 Basic MIPS Pipelining.3 Irwin, PSU, 2005
How Can We Make It Even Faster?
Split the multiple instruction cycle into smaller and
smaller steps
There is a point of diminishing returns where as much time is
spent loading the state registers as doing the work
Start fetching and executing the next instruction before
the current one has completed
Pipelining (all?) modern processors are pipelined for
performance
Remember the performance equation:
CPU time = CPI * CC * IC
Fetch (and execute) more than one instruction at a time
Superscalar processing stay tuned
CSE431 L06 Basic MIPS Pipelining.4 Irwin, PSU, 2005
A Pipelined MIPS Processor
Start the next instruction before the current one has
completed
improves throughput - total amount of work done in a given time
instruction latency (execution time, delay time, response time -
time from the start of an instruction to its completion) is not
reduced
Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5
IFetch Dec Exec Mem WB lw
Cycle 7 Cycle 6 Cycle 8
sw
IFetch Dec Exec Mem WB
R-type
IFetch Dec Exec Mem WB
- clock cycle (pipeline stage time) is limited by the slowest stage
- for some instructions, some stages are wasted cycles
CSE431 L06 Basic MIPS Pipelining.5 Irwin, PSU, 2005
Single Cycle, Multiple Cycle, vs. Pipeline
Multiple Cycle Implementation:
Clk
Cycle 1
IFetch Dec Exec Mem WB
Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7 Cycle 8 Cycle 9 Cycle 10
IFetch Dec Exec Mem
lw sw
IFetch
R-type
lw IFetch Dec Exec Mem WB
Pipeline Implementation:
IFetch Dec Exec Mem WB sw
IFetch Dec Exec Mem WB R-type
Clk
Single Cycle Implementation:
lw sw
Waste
Cycle 1 Cycle 2
CSE431 L06 Basic MIPS Pipelining.6 Irwin, PSU, 2005
MIPS Pipeline Datapath Modifications
What do we need to add/modify in our MIPS datapath?
State registers between each pipeline stage to isolate them
Read
Address
Instruction
Memory
Add
P
C

4
Write Data
Read Addr 1
Read Addr 2
Write Addr
Register

File
Read
Data 1
Read
Data 2
16 32
ALU
Shift
left 2
Add
Data
Memory
Address
Write Data
Read
Data
I
F
e
t
c
h
/
D
e
c

D
e
c
/
E
x
e
c

E
x
e
c
/
M
e
m

M
e
m
/
W
B

IF:IFetch ID:Dec EX:Execute MEM:
MemAccess
WB:
WriteBack
System Clock
Sign
Extend
CSE431 L06 Basic MIPS Pipelining.7 Irwin, PSU, 2005
Pipelining the MIPS ISA
What makes it easy
all instructions are the same length (32 bits)
- can fetch in the 1
st
stage and decode in the 2
nd
stage
few instruction formats (three) with symmetry across formats
- can begin reading register file in 2
nd
stage
memory operations can occur only in loads and stores
- can use the execute stage to calculate memory addresses
each MIPS instruction writes at most one result (i.e.,
changes the machine state) and does so near the end of the
pipeline (MEM and WB)
What makes it hard
structural hazards: what if we had only one memory?
control hazards: what about branches?
data hazards: what if an instructions input operands depend
on the output of a previous instruction?
CSE431 L06 Basic MIPS Pipelining.8 Irwin, PSU, 2005
Graphically Representing MIPS Pipeline







Can help with answering questions like:
How many cycles does it take to execute this code?
What is the ALU doing during cycle 4?
Is there a hazard, why does it occur, and how can it be fixed?
A
L
U

IM Reg DM Reg
CSE431 L06 Basic MIPS Pipelining.9 Irwin, PSU, 2005
Why Pipeline? For Performance!
I
n
s
t
r.

O
r
d
e
r
Time (clock cycles)
Inst 0
Inst 1
Inst 2
Inst 4
Inst 3
A
L
U

IM Reg DM Reg
A
L
U

IM Reg DM Reg
A
L
U

IM Reg DM Reg
A
L
U

IM Reg DM Reg
A
L
U

IM Reg DM Reg
Once the
pipeline is full,
one instruction
is completed
every cycle, so
CPI = 1
Time to fill the pipeline
CSE431 L06 Basic MIPS Pipelining.10 Irwin, PSU, 2005
Can Pipelining Get Us Into Trouble?
Yes: Pipeline Hazards
structural hazards: attempt to use the same resource by two
different instructions at the same time
data hazards: attempt to use data before it is ready
- An instructions source operand(s) are produced by a prior
instruction still in the pipeline
control hazards: attempt to make a decision about program
control flow before the condition has been evaluated and the
new PC target address calculated
- branch instructions
Can always resolve hazards by waiting
pipeline control must detect the hazard
and take action to resolve hazards
CSE431 L06 Basic MIPS Pipelining.11 Irwin, PSU, 2005
I
n
s
t
r.

O
r
d
e
r
Time (clock cycles)
lw
Inst 1
Inst 2
Inst 4
Inst 3
A
L
U

Mem Reg Mem Reg
A
L
U

Mem Reg Mem Reg
A
L
U

Mem Reg Mem Reg
A
L
U

Mem Reg Mem Reg
A
L
U

Mem Reg Mem Reg
A Single Memory Would Be a Structural Hazard
Reading data from
memory
Reading instruction
from memory
Fix with separate instr and data memories (I$ and D$)
CSE431 L06 Basic MIPS Pipelining.13 Irwin, PSU, 2005
How About Register File Access?
I
n
s
t
r.

O
r
d
e
r
Time (clock cycles)
Inst 1
Inst 2
A
L
U

IM Reg DM Reg
A
L
U

IM Reg DM Reg
A
L
U

IM Reg DM Reg
A
L
U

IM Reg DM Reg
Fix register file
access hazard by
doing reads in the
second half of the
cycle and writes in
the first half
add $1,
add $2,$1,
clock edge that controls
register writing
clock edge that controls
loading of pipeline state
registers
CSE431 L06 Basic MIPS Pipelining.15 Irwin, PSU, 2005
Register Usage Can Cause Data Hazards
A
L
U

IM Reg DM Reg
A
L
U

IM Reg DM Reg
A
L
U

IM Reg DM Reg
A
L
U

IM Reg DM Reg
A
L
U

IM Reg DM Reg
Dependencies backward in time cause hazards
add $1,
sub $4,$1,$5
and $6,$1,$7
xor $4,$1,$5
or $8,$1,$9
Read before write data hazard
CSE431 L06 Basic MIPS Pipelining.16 Irwin, PSU, 2005
Loads Can Cause Data Hazards
I
n
s
t
r.

O
r
d
e
r
lw $1,4($2)
sub $4,$1,$5
and $6,$1,$7
xor $4,$1,$5
or $8,$1,$9
A
L
U

IM Reg DM Reg
A
L
U

IM Reg DM Reg
A
L
U

IM Reg DM Reg
A
L
U

IM Reg DM Reg
A
L
U

IM Reg DM Reg
Dependencies backward in time cause hazards
Load-use data hazard
CSE431 L06 Basic MIPS Pipelining.17 Irwin, PSU, 2005
stall
stall
One Way to Fix a Data Hazard
I
n
s
t
r.

O
r
d
e
r
add $1,
A
L
U

IM Reg DM Reg
sub $4,$1,$5
and $6,$1,$7
A
L
U

IM Reg DM Reg
A
L
U

IM Reg DM Reg
Can fix data
hazard by
waiting stall
but impacts CPI
CSE431 L06 Basic MIPS Pipelining.19 Irwin, PSU, 2005
Another Way to Fix a Data Hazard
A
L
U

IM Reg DM Reg
A
L
U

IM Reg DM Reg
A
L
U

IM Reg DM Reg
Fix data hazards
by forwarding
results as soon as
they are available
to where they are
needed
A
L
U

IM Reg DM Reg
A
L
U

IM Reg DM Reg
I
n
s
t
r.

O
r
d
e
r
add $1,
sub $4,$1,$5
and $6,$1,$7
xor $4,$1,$5
or $8,$1,$9
CSE431 L06 Basic MIPS Pipelining.21 Irwin, PSU, 2005
Forwarding with Load-use Data Hazards
A
L
U

IM Reg DM Reg
A
L
U

IM Reg DM Reg
A
L
U

IM Reg DM Reg
A
L
U

IM Reg DM Reg
A
L
U

IM Reg DM Reg
Will still need one stall cycle even with forwarding
I
n
s
t
r.

O
r
d
e
r
lw $1,4($2)
sub $4,$1,$5
and $6,$1,$7
xor $4,$1,$5
or $8,$1,$9
CSE431 L06 Basic MIPS Pipelining.22 Irwin, PSU, 2005
Branch Instructions Cause Control Hazards
I
n
s
t
r.

O
r
d
e
r
lw
Inst 4
Inst 3
beq
A
L
U

IM Reg DM Reg
A
L
U

IM Reg DM Reg
A
L
U

IM Reg DM Reg
A
L
U

IM Reg DM Reg
Dependencies backward in time cause hazards
CSE431 L06 Basic MIPS Pipelining.23 Irwin, PSU, 2005
stall
stall
stall
One Way to Fix a Control Hazard
I
n
s
t
r.

O
r
d
e
r
beq
A
L
U

IM Reg DM Reg
lw
A
L
U

IM Reg DM Reg
A
L
U

Inst 3
IM Reg DM
Fix branch
hazard by
waiting
stall but
affects CPI
CSE431 L06 Basic MIPS Pipelining.25 Irwin, PSU, 2005
Corrected Datapath to Save RegWrite Addr
Need to preserve the destination register address in
the pipeline state registers
Read
Address
Instruction
Memory
Add
P
C

4
Write Data
Read Addr 1
Read Addr 2
Write Addr
Register

File
Read
Data 1
Read
Data 2
16 32
ALU
Shift
left 2
Add
Data
Memory
Address
Write Data
Read
Data
IF/ID
Sign
Extend
ID/EX EX/MEM
MEM/WB
CSE431 L06 Basic MIPS Pipelining.26 Irwin, PSU, 2005
MIPS Pipeline Control Path Modifications
All control signals can be determined during Decode
and held in the state registers between pipeline stages
Read
Address
Instruction
Memory
Add
P
C

4
Write Data
Read Addr 1
Read Addr 2
Write Addr
Register

File
Read
Data 1
Read
Data 2
16 32
ALU
Shift
left 2
Add
Data
Memory
Address
Write Data
Read
Data
IF/ID
Sign
Extend
ID/EX
EX/MEM
MEM/WB
Control
CSE431 L06 Basic MIPS Pipelining.27 Irwin, PSU, 2005
Other Pipeline Structures Are Possible
What about the (slow) multiply operation?
Make the clock twice as slow or
let it take two cycles (since it doesnt use the DM stage)

A
L
U

IM Reg DM Reg
MUL
A
L
U

IM Reg DM1 Reg DM2
What if the data memory access is twice as slow as
the instruction memory?
make the clock twice as slow or
let data memory access take two cycles (and keep the same
clock rate)
CSE431 L06 Basic MIPS Pipelining.28 Irwin, PSU, 2005
Sample Pipeline Alternatives
ARM7




StrongARM-1


XScale
A
L
U

IM1 IM2 DM1
Reg
DM2
IM Reg
EX
PC update
IM access
decode
reg
access
ALU op
DM access
shift/rotate
commit result
(write back)
A
L
U

IM Reg DM Reg
Reg
SHFT
PC update
BTB access
start IM access
IM access
decode
reg 1 access
shift/rotate
reg 2 access
ALU op
start DM access
exception
DM write
reg write
CSE431 L06 Basic MIPS Pipelining.29 Irwin, PSU, 2005
Summary
All modern day processors use pipelining
Pipelining doesnt help latency of single task, it helps
throughput of entire workload
Potential speedup: a CPI of 1 and fast a CC
Pipeline rate limited by slowest pipeline stage
Unbalanced pipe stages makes for inefficiencies
The time to fill pipeline and time to drain it can impact
speedup for deep pipelines and short code runs
Must detect and resolve hazards
Stalling negatively affects CPI (makes CPI less than the ideal
of 1)
CSE431 L06 Basic MIPS Pipelining.30 Irwin, PSU, 2005
Next Lecture and Reminders
Next lecture
Overcoming data hazards
- Reading assignment PH, Chapter 6.4-6.5

Reminders
HW2 due September 29
th

SimpleScalar tutorials scheduled
- Thursday, Sept 22, 5:30-6:30 pm in 218 IST

Evening midterm exam scheduled
- Tuesday, October 18
th
, 20:15 to 22:15, Location 113 IST
- You should have let me know by now if you have a conflict