Tutorial

For
orma
mall A
Ass
sse
ert
rtio
ion
n base
basedd VeVeri
rifi
fica
cati
tion
on
in Ind
ndust
ustri
ria
al Sett
ttin
ing
g

 All o k J ain
 A ai n Raj
Raj S. Mitr
Mit r a
Cadenc
adencee Design
esig n Systems
Syst ems Texas Instruments
Instr uments
Noida Bangalore

Jaso
Jasonn Ba
Baumg
umgaartne
rtn er  Pallab Dasgupta
IB M
IBM Indian Institute
Institu te of Techno
Technolog
logyy
 Auu s t i n
 A Kharagpur 

Desi
sign
gn Au
Auto
tomati
mation
on Con
onferenc
ferencee, San Diego,
Diego , Ju
June
ne 8,
8, 2007
 Ag
 A g en
endda
 Introduction to Formal Verification

 Case Studies from TI: Protocol & Control Logic Verification

 Case Studies from IBM: Formal Processor Verification

 Verification Closure: Coverage Analysis & Integration with Simulation

 Ag
 A g en
endda
 Introduction to Formal Assertion Based Verification
■ What is the problem statement?
■ Temporal Logics
● LTL and CTL
● PSL and SVA

■ Model Checking
● CTL Model Checking
● LTL Model Checking

■ Symbolic Model Checking

● BDD and SAT based techniques
■  Ab
 Abstractions
 Case Studies from TI: Protocol & Control Logic Verification

 Case Studies from IBM: Formal Processor Verification

 Verification Closure: Coverage Analysis & Integration with Simulation

Pr ob
obll em Stateme
Statemennt
 Problem Statement
■ Verify an RTL design satisfies its functional requirements
 Tradi
radition
tional
al Soluti
Solution
on – Logic
Logic Si
Simulation
ulation
■ RTL design specified in HDLs
■ Requirements specified as test vectors
 Limitations of Logic Simulation
■ Limited set of test vectors
■ Exercise only a small fraction of design
■ Leads to undetected bugs
 Exam
xample – Float
oating poi
point divisi
division
on bug
bug in Pent
entium
■ 1012
1012 test vect
vectors
■ Cost $470 million
 All t er
 A ernn at
atii v e Sol
So l u t i o n – Fo
Forr m al A B V
 Formal Assertion Based Verification

 Mathematically reason about correctness of RTL design

 Properties specified in some form of “Temporal Logic”

Properties
Formal
 Assertion Does property hold
Based on the RTL design?
Verification
RTL design

 Benefits
■ Does not require user to provide test vectors
■ Does exhaustive verification
 Ag
 A g en
endda
 Introduction to Formal Assertion Based Verification
■ What is the problem statement?
■ Temporal Logics
● LTL and CTL
● PSL and SVA
■ Model Checking
● CTL Model Checking
● LTL Model Checking
■ Symbolic Model Checking
● BDD and SAT based techniques
■  Ab
 Abstractions
 Case Studies from TI: Protocol & Control Logic Verification
 Case Studies from IBM: Formal Processor Verification
 Verification Closure: Coverage Analysis & Integration with Simulation
Tempo
mpora
rall Lo
Logic
gics
s
 Logic extended with the notion of time

 Reason about propositions qualified in terms of time

 Tradeo
radeoffff betw
betwee
een
n expre
express
ssiibili
bility and
and com
complexi
plexity of verifi
verifica
catition
on

 Two popular forms of temporal logic for formal verification

■ Linear Temporal Logic (LTL)
■ Computation Tree Logic (CTL)
Line
Li nea
ar Temp
mpor
ora
al Lo
Logi
gic
c (LT
LTL)
L)
 Introduc
ntroduced
ed by Pnueli
nueli in 197
1977
7
 Propositional Logic + discrete time

 Time is viewed as a single linear sequence of events

 Properties specified over a single path

 Temporal operators to represent discrete tim

■ p is a propos
proposiition
tion – p sho
should
uld hold
hold at
at current titime
■ X p – p sho
should
uld hol
hold at
at next
next titime
■ F p – p sho
should
uld hold
hold in the future
future
■ G p – p sho
should
uld hold
hold gl
globall
obally
y
LTL Formulas

Xp

Fp

Gp

pUq

p Wq

Time
LTL
LTL - Exa
xample
mples
s
 Safety
afety Propert ies - G ¬(Critical1 ∧ Critical2)
Pro perties
■ Something bad never happens
 Liveness - F (Re
(Req1 ∨ Req2)
■ Something Good will eventually happen
 Fairness - G (Re
(Req1  F Critical1)
■ If something is requested, it eventually gets scheduled
 Strong
tr ong ir ness - GF (Re
Fairness (Req1 ∧ ¬ Critical2)  GF Critical1
■ If something is repeatedly requested, it repeatedly gets scheduled
Com
ompu
putatio
tationn Tree
Tree Lo
Logi
gic
c (CT
(CTL)
 A form of Branching Time Temporal Logic

 Introduced by Clarke and Emerson

 Temporal Operators as in LTL

■ Xp
■ Fp
■ Gp
 In addition, path operators
■ A – Over all paths
■ E – There
here exi
exists a path
path
Com
ompu
puta
tati
tion
on Tree
Tree Lo
Logi
gic
c
 AF
 A F p - On all pa th s , the property holds in the future
p aths

 AG
 A G p - On all pa th s , the property holds globally
p aths
Com
ompu
puta
tati
tion
on Tree
Tree Lo
Logi
gic
c

EF p - There exist path on which the property holds in the future

xi stss a path

EG p - There exist path on which the property globally holds

xi stss a path
CTL Ex am p l es
 Safety
afety Propert
Pro perties
ies
■  AG
 AG ¬(Critical1 ∧ Critical2)
 Fairness
■ AG
 AG (state1  EF (¬ state1))
■ Self deadlock check for state1

■  AG
 AGEF (resetState)
■  A powerful and useful resetability test

 Strong
tro ng fairn
fairneess
■ Cannot be specified in CTL
Com
ompari
pariso
sonn betwee
between
n LTL and
and CTL
 Expressibility
■ Neither is superior to the other 
■ LTL
LTL - Stron
trongg fairne
fairness
ss CTL - Deadloc
eadlock
k
■ CTL* - Superset
uperset of bot
both LTL
LTL and
and CTL
CTL
 Verification Complexity
■ LTL
LTL - Expon
xponen
entitial
al in
in size of
of form
formula
ula
■ CTL – Linear
Linear in
in size ofof form
formula
ula
 Compositional Verification
■ LTL
LTL – Yes CTL – No
 Am
 Amenable to Simulation
■ LTL
LTL – Yes CTL – No
 Preference
reference in the industry
industry – LTL
LTL flav
flavor 
or 
Com
ompari
pariso
sonn betwee
between
n LTL and
and CTL

L TL CTL

Expressibility
Strong Fairness Deadlock
(CTL* superset of LTL and CTL)

Verification
rif ication Comple
ompl exity
xit y
Exponential Linear  
(in size of formula)

Composit
ompo sition
ionaal Ve
Verification
rif ication Yes No

 Am
 A m enab
en abll e to
t o Si
Simm u l ati
at i o n Yes No

 Preference
reference in the indus
industr
try
y – LTL
LTL flav
flavor 
or 
Ind
ndus
ustr
tria
iall La
L ang
ngua
uage
gess
 Limitations of LTL and CTL
■ Mainly academic languages for experts
■ Unreadable by average user 
■ Open to mis-interpretation
■ Sometimes un-intuitive to express properties
 Industrial Languages
■ Property Specification Languages (PSL)
■ System Verilog Assertions (SVA)
■ Extending LTL with notion of sequences
● PSL al
also has
has a CTL flavor
avor - Not used very often
often
Ov er v i ew o f PSL

 Boolean Expressions
■ expressi ons Verilog xor VHDL
HDL expressions How
■ PSL/Sugar functions: rose(), fell(), prev(), ... When
What
 Temporal
mpo ral Operato
perators
rs
■ always , never , next , until , before, eventually , abort , ...
■ @ -> |-> |=> ; { } [* ] [= ] [-> ] && & | :
 Verification Directives
■ assert , assume, restrict , cover , ...
 Modeling Const
Construc
ructs
ts
■ HD s tatements used to model the environment
HDLL statements
 Invariants
 Som
 Something
ing that should never happen! 
For example
le:: An und
underflow s hould
hould never
never occur 
Verification
Directive
How Temporal
When Operator  Boolean
What Expression

assert_
ssert_no_
no_unde rflow : assert never  (rea
underflow readd && emp ty );
mpty
How When What
label (to apply) (expression (to check)
should
be true) This assertion
should hold
ho ld at
at
every
every ti
time
me step

assert_
ssert_no_
no_underflow : assert never  (rea
underflow readd AND emp ty );
empty
 Cond
ondit
itio
iona
nall Be
Beha
havi
vior 
or 

Example
le:: If A receive
ceivess a Grant,
nt, the
then B does not 
assert_if_GntA_no_GntB : assert
always (G (Gnt
ntAA -> ! GntB)
GntB ) @(pos
poseedge clk)
clk ) ;
assert_if_GntA_no_GntB : assert property (@(posedge clk)
(GntA) |-> (! GntB))
GntB)) ;
Implication  A t t h e ri
 At r i si ng clk,
operatorr (-
operato (->)
>) if GntA is high ,
expresses then GntB
GntB must mu st
“ if..
if....then” be low

GntA
if i f  if if if if i f  if  
then then
GntB

clk
 Mul
ulti
ti--Cyc
ycle
le Con
Condi
diti
tion
ona
al Be
B eha
havi
vior 
or 
Example
le:: A Grant is alwa
lways follow
llowed by Bus
Busy 
y 
asse
assert
rt_b
_bus
usy_
y_aft
after_G
er_Gnt
nt : assert
always (GntA OR GntB
GntB)) -> n ext Busy @(risi
(rising
ng_e
_edg
dgee (clk
(clk))
)) ;
assert_busy_after_Gnt : assert property (@(posedge clk)
(GntA || GntB) |=> (Busy)
(Busy))) ;
Impli cation
Implicati on (->)
(->) and
‘next’ t ogethe
ogetherr
express multi-cycle
conditional
GntA behavior 
i f  i f 
GntB then
i f 
then

then Now there

th ere is a
one-cycle delay
Busy between
betwee n ‘if’
‘i f’ and
‘then’
clk
 Mul
ulti
ti--Cyc
ycle
le Behavio
Behaviorr Usi
sing
ng Sequ
queenc
ncees
Example: A never receives a Grant in two successive cycles
assert_no_2_GntA : assert
never {GntA
{ GntA ; GntA
GntA}} @(posposeedge clk)
clk ) ;
assert_no_2_GntA : assert proppr ope erty
rt y (@
(@(posedge
po sedge clk)
cl k)
(GntA ##1 Gnt
GntAA ) |-> (0
(0))) ;
 A s eq
equu en
encc e is
is a
shortha
short hand
nd for a
series
se ries of ‘next’s

GntA
GntB

clk
 Compo
ompound
und Asse
A sserti
rtions
ons
Example: If Request
Request is followed by Grant, then next is Busy, and
and next is Done
Done

assert_GntA_ReqA_next_Busy_Done : assert always

(ReqA
(ReqA -> next (GntA -> ne
next
xt (Busy & & n ex t Done))) @(po
pose
sedg
dgee cl
clkk ) ;

assert_GntA_ReqA_next_Busy_Done : assert always

{ ReqA
ReqA;; GntA
GntA}} |=> {Busy
{Busy;; Done
Done}} @(posedge clk)
clk ) ; The two
asserti
assertions
ons are
ReqA i f  i f  i f  i f  i f  i f  i f  i f  equivalent.
next next next next next next

GntA i f  i f  i f  i f  i f  i f 
then then

Busy && next && next

Evaluation
starts again
again in
each
each cyc
cycle,
le,
Done overlapping
previous
evaluations
clk
 Ag
 A g en
endda
 Introduction to Formal Assertion Based Verification
■ What is the problem statement?
■ Temporal Logics
● LTL and CTL
● PSL and SVA
■ Model Checking
● CTL Model Checking
● LTL Model Checking
■ Symbolic Model Checking
● BDD and SAT based techniques
■  Ab
 Abstractions
 Case Studies from TI: Protocol & Control Logic Verification
 Case Studies from IBM: Formal Processor Verification
 Verification Closure: Coverage Analysis & Integration with Simulation
 Mod
odeel Check
Checkin
ing
g Properties Formal
 As
 A s s ert
er t i o n
Does property hold
Based
on RTL?
Verification
RTL design

Properties
PSL/SVA
Model Does property hold
Checking on the RTL design?

RTL design FSM model

 Model Checking
■ Does the model satisfy the property?
■ Does not require any test vectors
■ Does exhaustive verification
 How do
doees Mod
Modeel Che
Check
ckin
ingg wor
w ork?
k?

Reachable Starting  Simulation

State Space State Point
■ Requires test vectors
■ Follows specific path
x

x
x x
x

x
x
x
 Model Checking

x x
x
x
x
■ Requires no test vectors
x ■ Works on entire state

x
x
space
■ Breadth first search

Bugs triggered  Exhaustive

with simulation ■ Uncovers all possible bugs
 Example
r 
 RTL Design t [1]
 Modulo-3 counter 
t [0]

p [0]
 FSM Model
p [1]
r =1

r =0 r =0
0 1 2 3  CTL Property
r =1  EF (p = 2)
Start state

 Starti
arting from start state,
state, can
can the coun
countter even
eventtual
ually cou
count
nt upt
upto 2
CTL Mode
Mod el Chec
Chec k i n g
 Evaluate the CTL property on the model
■ Evaluate EF (p = 2) on the FSM

r =1

r =0 r =0
0 1 2 3
r =1

Least
Least fixed
f ixed Point

 Start state is a part of the Least Fixed Point

■ Indicates there is a path from start state to (p = 2)
■ EF (p = 2) HOLDS
HOLDSon on the FSM model
Lets
Le ts intr
i ntrod
oduce
uce a bu
bugg
r 
 RTL Design t [1]
 Buggy Modulo-3 counter 
t [0]

p [0]

p [1]
 FSM Model

r =1
r =0
 CTL Property
r =0  EF (p = 2)
0 1 2 3
r =1
CTL Mod
Modeel Che
Check
ckin
ingg on
o n the
t he bu
bugg
ggy
y de
d esi
sign
gn
 Evaluate the CTL property on the buggy model
■ Evaluate EF (p = 2) on the buggy FSM

r =1
Least
Least fixed
f ixed Point
r =0

r =0
0 1 2 2
r =1

 Start state is NOT part of the Least Fixed Point

■ Indicates there is NO path from start state to (p = 2)
■ EF (p = 2) DOES NOT HOLHOLD on the buggy FSM model
LTL Mod
Modeel Check
Checkin
ing
g
 Requires a new mathematical formulation
■ Omega automata (ω-automata)
 ω-automata
■ Au
 Automata over infinite words
■ Infinitely running automata

■ Single framework for expressing design and property

■ Various different forms:

● Buchi, Streett, Rabin, L-automata and L-process

 Buchi
uchi Automata
utomata
■  Ac
 Accepting state is visited infinitely often
 -aut om
omaata for de
desi
sign
gn
r =1 r 
t [1]

t [0]
r =0 r =0
0 1 2 3
r =1

(r  1,t 0) (r =0,t 1)

(t 0)
(r =1,t=0) (t 0)
(r =1,t≠0) (r =0,t≠2)

(r =0,t=1) (r =0,t=2)

(r =1,t=0) (t=0)
(t=0)
 -aut om
omaata for de
desi
sign
gn

(r  1,t 0) (r =0,t 1)

(t 0)
(r =1,t=0) (t 0)
(r =1,t≠0) (r =0,t≠2)

(r =0,t=1) (r =0,t=2)

(r =1,t=0) (t=0)
(t=0)

 Valid Behavior – (r =1,t=0) (r =0,t=1) (r =0,t=2) (r =0,t=0) ………

 Inva
Invalid Behavior – (r =1,t=1) ………

 L(D)
L(D) – Set of all val
valid beh
behav
aviiors of
of the des
desiign
 -aut om
omaata fo
forr pro
p rope
pert
rty
y
 Every LTL property can be expressed as ω-automata

 F (t = 2)

 Al
 Alternatively eventually! (t == 2)
t≠2

t=2

 Valid behavior of the property

■ *
(t ≠ 2) (t = 2) (true)
ω

■ …........................................ (t = 2) ……………………………..
…..
 L(P)
L(P) – Set of all val
valid beh
behav
aviiors of
of the prop
property
erty
 LTL Mod
Modeel Check
Checkin
ing
g
 LTL Model Checking is Language Containment

 Design behaviors is a subset of property behaviors

 L(D) ⊆ L(P) or L(D)

L(D) ∩ L(P) = ∅

(r  1,t 0) (r =0,t 1)

(r =1,t=0)
(t 0)
(t 0)
Cross
(r =1,t≠0) (r =0,t≠2)

(r =0,t=1) (r =0,t=2)
Product
(r =1,t=0) (t=0) (t=0)

t≠2
Complement Empty Language?
t=2
t=2
 LTL
LT L Mod
Modeel Che
Check
ckin
ingg on
o n our
o ur Exa
xamp
mple
le

F (t = 2)
(r  1,t 0) (r =0,t 1)
(t 0) t≠2
(r =1,t=0) (t 0)
(r =1,t≠0) (r =0,t≠2)
t=2
(r =0,t=1) (r =0,t=2)

(r =1,t=0) (t=0)
(t=0)
Will obtain a CEX – (r =1, t=0)ω
 Ad
 Add a constraint – G (r = 0) or alw
always(r == 0)
 Constraints are used to model environment and limit behaviors

 L(C)
L(C) – Set of all beha
behaviors
viors of
of the con
constrai
straint
nt
 Check L(C) ∩ L(D) ∩ L(P) = ∅. Property will pass
Com
ompl
ple
exi
xity
ty of Model
Model Che
Check
ckin
ingg
 CTL Model Checking
■ Evaluate property over the FSM structure
■ Notion of fixed point computation
■ Complexity is linear in the size of the formula
 LTL Model Checking
■ Notion of ω-automata
■ Translate design and property to ω-automata
■ Then LTL model check is simply a graph problem
■ Complexity is exponential in the size of the formula
 What problem remains? The State Explosion Problem
■ Complexity is linear in the number of states in design
■ Exponential in number of state bits in the design
 Ag
 A g en
endda
 Introduction to Formal Assertion Based Verification
■ What is the problem statement?
■ Temporal Logics
● LTL and CTL
● PSL and SVA
■ Model Checking
● CTL Model Checking
● LTL Model Checking
■ Symbolic Model Checking
● BDD and SAT based techniques
■  Ab
 Abstractions
 Case Studies from TI: Protocol & Control Logic Verification
 Case Studies from IBM: Formal Processor Verification
 Verification Closure: Coverage Analysis & Integration with Simulation
Bina
Bi nary
ry Deci
cisi
sion
on Dia
iagr
gra
ams

a
b

c out  Ordered Decision Tree

out

a
0 1

a b c out b b
0 1
0 0 0 0
0 0 1 1 c c c c
0 1 0 0 0 1
0 1 1 1
1 0 0 0 0 1 0 1 0 1 1 1
1 0 1 1
1 1 0 1
1 1 1 1
Bina
Bi nary
ry Deci
cisi
sion
on Dia
iagr
gra
ams

out
 Reduced Order BDD (ROBDD)
 Merge isomorphic nodes
a  Remove redundant tests

b b

out
c c c c
a

0 1 0 1 0 1 1 1 b

0 1
 BDD base
basedd Sym
Symbo
bolilic
c Mod
Modeel Check
Checkin
ing
g
 Build a symbolic FSM model using BDDs

 Perform fixed point computation on the symbolic FSM

 Determine if property holds or not on the symbolic FSM

Properties
PSL/SVA
Symbolic Does property hold
Model on the RTL design?
Checking

RTL design Symbolic FSM

Sym
ymbol
bolic
ic Mode
odell Che
Check
ckin
ingg on
o n our
o ur Exa
xampl
mple
e
r 
r 

p1

p0 n0
p0
p1 n1
r =1 n1 n1 n1

r =0 r =0 n0 n0
00 01 10 11
r =1
0 1

 FSM next state function can be represented symbolically as a BDD

 BDD allows both forward and backward state traversal in the FSM
Technically called a transition relation
Sym
ymbol
bolic
ic Mode
odell Che
Check
ckin
ingg on
o n our
o ur exa
xampl
mple
e
r =1

r =0 r =0
00 01 10 11
r =1

r 

p1

n1 p1
p0

r  ( 0
n0

1
 AND
 A ND n1 n1 n1
) = p0

1 0
n0 n0

0 1
B DD Base
Basedd Mo d el Check
heckii n g
 Benefits
■  Al
 All operations can be done symbolically
■  Av
 Avoids upfront state explosion of explicit FSM
 However, not a magic solution
■ Size of BDD dependent on variable order 
■ Exam
xample – for an n-bit adde
adder 
r 
● Best order
order is line
linear,
ar, Worst is
is expon
exponen
entitial
al
■ Finding the optimal order is a very hard problem
● NP-complete problem

 In practice works well if 

■ Transition relation compactly represented with BDDs
■ Reacha
chability
lity is fa
fast - BDDs do no
not get stuck
stuck in reorde
reorderirin
ng
SAT (S
(Sati
tisf
sfiabi
iabilility
ty)) Pro
robl
ble
em

 Given Boolean function f(x1, x2, …, xn)

 If it is possible to find assignments:
■ (x =a ), (x =a ), … (x =a ), such that
1 1 2 2 n n
■ f(a , a , …, a ) = 1?
1 2 n

 Works on Boolean formula in CNF

■ Conjunctive Normal Form or product of sums form

(¬a ∨ b) ∧ (¬b ∨ c ∨ d)

 This is an NP-Complete problem

Tra
ransf
nsfor
ormi
ming
ng de
desi
sign
gn int
intoo a SAT
SAT pr
probl
oble
em
 Can the output of a design take value 1?

CNF1 = (a ∨ ¬g) ∧ (b ∨ ¬g) ∧ (¬a ∨ ¬b ∨ g)

a g
b
c p

CNF2 = (¬g ∨ p) ∧ (¬c ∨ p) ∧ (g ∨ c ∨ ¬p)

 p can take value 1 if (CNF1 ∧ CNF2 ∧ p) is satisfiable

 r 
Boun
Bo unde
dedd Mod
Modeel Che
Check
ckin
ingg
 Unfold the circuit “k” times
p

r 0 r 1 r 2 r k-1

p0
p1 p2 p3 … pk
p0

Start state p0=0 Check property until k steps

 Take the unrolled circuit and convert into CNF

 Use SAT solver to check if safety property holds for k steps

 If no, then SAT solver will give a CEX of ≤ k steps

 Bounded Model Checking can only give failures for safety

Sym
ymbo
bolilic
c Mod
Modeel Che
Check
ckin
ingg Sum
Summa
mary
ry
 BDD based Model Checking
■ Works well for “quick” passes and failures
■ Passes and failures could be quite deep
 SAT based Bounded Model Checking
■ Good for shallow failures
■ For deep failures unrolling becomes a bottleneck
 Other techniques
■ Combining BDD, SAT and ATPG based techniques
■ Distribution

■  Ab
 Abstractions
 Ag
 A g en
endda
 Introduction to Formal Assertion Based Verification
■ What is the problem statement?
■ Temporal Logics
● LTL and CTL
● PSL and SVA
■ Model Checking
● CTL Model Checking
● LTL Model Checking
■ Symbolic Model Checking
● BDD and SAT based techniques
■  Ab
 Abstractions
 Case Studies from TI: Protocol & Control Logic Verification
 Case Studies from IBM: Formal Processor Verification
 Verification Closure: Coverage Analysis & Integration with Simulation
 Ab
 A b s t r ac
actt i o n
 Throw away information that is not required for proof 

 Ab
 Abstraction is key to scaling capacity of model checking
 One of most important abstraction
■ “Localization" abstraction
■ Throws away information not relevant to the given property

G (p  Xp)
Make g a pseudo-input

g always ((p==1)  next (p==1))

a
b p
 Ab
 A b s t r ac
actt i o n an
andd Ref
Refii n em
emen
entt

 Choose Initial Abstraction

 Model Check Abstracted Design
 Propert
ropertyy Pas
Pass
s – It is ind
indee
eed
d a Pass
ass on
full design!
 Justify Counter-example
 Justi
Justifified
ed?
? – Bug Found
ound!!
 Refine the model
■ Many ways of refinement
 Model Check Refined Design
Oth
theer Adva
Ad vanc
nceed Abst
Ab stra
ract
ctio
ionn Te
Tech
chni
niqu
quees
 Proof based Abstraction-Refinement
■ Combines SAT based BMC and BDD based model checking
■ Uses SAT based BMC to find a suitable abstraction

 Interpolation
■ SAT based Unbounded Model Checking
■ Uses
ses over-app
over-approxim
roximate
ate reacha
reachabil
biliity analysis
analysis
Ind
ndus
ustr
tria
iall For
Forma
mall ABV
A BV Too
ools
ls
 Various Commercial Tools
■ Caden
adence
ce – Incisive
Incisive Formal
ormal Veri
Verififier 
er 
■ Synopsy
ynopsyss – Magell
agellan
an
■ Mentor – 0-in
■ Jasp
Jasper
er – Jasp
JasperG
erGold
old
■ OneSpin, Real-Intent, Averant, Axiom
 Combination of Symbolic Model Checking and Abstractions

 Can handle local properties

■ Typically can scale to around 10K state bits
 Global properties are still difficult
■ Need advanced abstractions and compositional techniques
 Ag
 A g en
endda
 Introduction to Formal Verification
 Case Studies from TI: Protocol & Control Logic Verification
■ Methodology
● Planning for Formal Verification
● Property Coding Guidelines
● Formal Verification Flow
■ Case Studies
● Protocol Compliance : Bridge Validation
●  Ar
 Arbitration
● Interrupt Sorter 

● Memory Controller 

● SoCC
oC Conne
onnecti
ctivit
vity
y
● SoCP
oC Performan
erformancece

 Case Studies from IBM: Formal Processor Verification

 Verification Closure: Coverage Analysis & Integration with Simulation
Need for
f or Eff
ffic
icie
ient
nt Meth
thod
odol
olog
ogy
y
 State explosion problem (insufficient resources) very real !

 Ab
 Abstraction is the key
■ Au
 Automatic
■ Manual

 No golden abstraction(s) to solve state explosion problem

 An
 Analysis is important (apriori and dynamic)
■ For handling formal proof complexity
■ For predictability of FV usage and results
Realility
ty Bi
Bites!
tes!
 80% of medium / large sized modules experience state explosion
■ For one or more assertions
■ No golden abstraction(s) apply across all designs
 70% of initial counter-examples are spurious
■ More so because of incorrect constraints, rather than incorrect
assertions
■ Decomposition in practical scenarios lacks formalism
 50-70% of time is consumed in modeling / setup / coding
■ Rest is tool run time

Experience from application of FV on large set of Industrial designs

Need fo
forr Veri
rifi
fica
cati
tion
on Pla
lann
nnin
ingg
 Verification Objective: Close verification within a predicted schedule
■ Planning

● Complexity Estimation, Verification Plan, Partitioning and Abstractions

■ Execution

●  An
 Analyzing indeterminate results
●  Ad
 Adding / Removing / Optimizing Constraints
● Identification and Closure of False Failures

● Itera
Iteratition
ons
s could
could be large – due
due to adho
adhoc
c cons
constr
traint
aint updates
updates
■ Closure
losure – Signoff 
ignoff 
● Coverage analysis, Verification report

  Al
 All the above are inter-related
■ Closure has no meaning if results are indeterminate
■ Indeterminate results point to lack of proper abstraction
■ Lack of proper abstraction results from improper planning
For
orma
mall Ve
Veri
rifi
fica
cati
tion
on Deci
cisi
sion
onss
 Ab
 Abstraction decision
■ Identify sweet spots (control versus data path)
■ What parts of the design need to be abstracted out?
 Modeling decision
■ Choosing the correct way of modeling the property
■ Complex SERE’s versus simple properties
 Property decision
■ Which lan
langu
guag
age
e ? Whi
Which
ch AI
AIP’s ? Gl
Glue logic
logic ?
 Tool specific Decision
■ Chose the correct FV engine
■ Chose correct tool (abstraction) options
For
orma
mall Veri
Verifi
fica
cati
tion
on Pla
lann
nnin
ingg
 Predict the problems in advance
■ Based on design knowledge, tool, experience, etc
 Predict
■  Ap
 Approximate design complexity (design analysis)
■  Ap
 Approximate proof complexity (related to constraints and logic cone of
assertions)
 Decide
■ Whether to partition or not ? Where to partition ?
■  Ab
 Abstraction mechanism associated with the partitioning
■ How to code assertions / constraints properly?

■ What engines / methods to use for what scenarios?

Plann
lannin
ing
g Step-
Step-II: Ident
Identif
ify
y the
t he sw
sweeet spo
s pots
ts
 Sweet spots : Control logic verification
■ Bus Bridges , Arbitration Logic
■ Controllers (FSM logic, Memory Cntlr, Interrupt Cntlr)
■ Control dominated Logic (Stall, Pipeline …)
 Not so sweet spots
■ Complex Serial Protocols
■ Modules with large FIFO’s
■ Protocol interfaces with huge latency parameters
 Negative candidates : Datapath intensive blocks
■ Data Transformation Blocks (Filters …)
■ Blocks with complex arithmetic units (Multipliers, Adders)
Plann
lannin
ing
g Step-
tep-III: An
Anaaly
lyzze th
thee mo
modu
dule
le
 Identify the functionality of the block to be verified

 Identify the functionality of the surrounding blocks.

■ Check for well defined interfaces (standard protocols)
 Prepare detailed micro level bulleted English plan for 
■ As
 Assertions to be coded for block functionality
■ Constraints to be coded for functionality of surrounding blocks

■ Disable / Restrain any data-path interfaces (address-data bits)

Pl an ni
ninn g St
St ep -III: Es
Es t i mate Co m pl
pleex i t y
 Estimate Complexity for FV
■ (a) Number of flops in the module (first-crude estimate)
■ (b) Number of flops in logic cone of each assertion (finer estimate)
■ (c) Number of flops inferred because of constraint coding (adds to
complexity)
 Good Candidate : < M flops ( (b) + (c) )
? Not so good Candidate : M – N flo
flops ( (b) + (c) )
⌧ Negative candidates : > N flops ( (b) + (c) )
M ~ 1000 , N ~ 3000 Partitioning,
Restrictions needed
Plann
lannin
ing
g Step-
tep-IIV: Part
rtit
itio
ionn if nee
needed
ded
 Structural Partitioning
■ Partition a design into a set of functionally independent and spatially
disjoint components
 Functional Partitioning
■ Partition the design functionality itself into mutually exclusive logical
partitions
Str
truct
uctur
uraal Pa
Part
rtit
itio
ionin
ning
g
 Golden Rules for partitioning
■ Identify the sub-modules (specs must be clear) which can be verified
separately
■ Identify the ‘simple / sparse’ interface at which to cut
■ Partition the design and code constraints at the cut-points
■ Verify each partition in isolation (apply assume-guarantee)
 Coding Constraints at cut-point-interface
■ Localization of the verification process
■  Ab
 Abstraction is needed to benefit from decomposition
■ Golden Rule: “If surrounding RTL is replaced with equivalent constraints

 No gain!”
Fun
unct
ctio
iona
nall Pa
Part
rtit
itio
ioni
ning
ng

 Identify mutually exclusive functional partitions

■ Example: Different modes of operation

 Identify interactions between these partitions

■ Indep
ndepen
ende
dent
nt – do not
not im
impose
pose restri
restricti
ction
ons
s on
on each
each other
● Example: independent read and write operations
● Use pin tie-offs to cut off the other partition

■ They follow a sequence

● Example: a “read-exclusive” operation in OCP protocol should always be
followed by a write operation
● Identify the shared resources which play a role in one or more partitions

● Use constraints or simulation to skip over the previous partition

 As
 A s s u m e–Gu
e–Guar
aran
antt ee Reaso
Reas o n i n g
 Properties can be used as either assertions or assumptions

 As
 Assume guarantee reasoning
■ DUV is block A
● assert p A; assume pB p A
■ DUV is block B
● assert pB; assume p A
 A B
pB
 Using Protocol AIP’s
■ at master interface
● assume prop_slave* , assert prop_master*
■ at slave interface
● assume prop_master*, assert prop_slave*
Fun
unct
ctio
ional
nal As
Assu
sume–
me–G
Guara
uarant
nte
ee
 Properties can be coded in layers (bottom up fashion)

 Properties in a layer assume validity of properties in below layers

 Ap
 Application:
■ Prove the correctness of lower level properties
■  Ap
 Apply them as constraints thereafter to prove higher level properties
■ Example : Arbitration validation:

Base level property: “The

“The grants are zero-one-hot
zero-one-hot””
Higher level property : “check the arbitration scheme”
 Ag
 A g en
endda
 Introduction to Formal Verification
 Case Studies from TI: Protocol & Control Logic Verification
■ Methodology
● Planning for Formal Verification
● Property Coding Guidelines
● Formal Verification Flow
■ Case Studies
● Protocol Compliance : Bridge Validation
●  Ar
 Arbitration
● Interrupt Sorter 

● Memory Controller 

● SoCC
oC Conne
onnecti
ctivit
vity
y
● SoCP
oC Performan
erformancece

 Case Studies from IBM: Formal Processor Verification

 Verification Closure: Coverage Analysis & Integration with Simulation
Pro
rope
pert
rtyy Cod
Codin
ing
g Gui
Guide
deliline
ness
 Golden Rules:
■ Monito
nitor style
style vs
vs gene
generator
rator style
● For verifying RTL, do not create another one
■  At
 Atomic vs Expressive Sequential properties
● Expressiveness adds proof complexity
■ Separa
eparabil
biliity of functi
function
onal
al chec
checks
ks
● Verify unrelated functional aspects in isolation
● Code separate properties for individual I/O’s of module

● Reduce input state space via pin-constraints

● Prove partitioned spaces are mutually exclusive

■ Incremental Property specification and verification

● Build up layers of properties
● Usually simpler to prove properties separately

■  Av
 Avoid integers, counters
Pro
rope
pert
rtyy Cod
Codin
ing
g Gui
Guide
deliline
ness
 Guideline
uidelines
s – try the
these
se templat
emplates
es for each
each propert
property
y
■ Can it be written as “condition should/shouldn’t always happen”
■ Can it be written as
● prev_state  (implies)
(implies) current_state
■ Can it be written using ‘event
‘event bounded
bounded window’
window’ – something
something happening
happening
between two specified events
■ Can it be written without using number of clocks (i.e. counter)

Event1 Event2
Event1 Event2

P P P P

as s er t f r am e as s er t i ndo
 Ag
 A g en
endda
 Introduction to Formal Verification
 Case Studies from TI: Protocol & Control Logic Verification
■ Methodology
● Planning for Formal Verification
● Property Coding Guidelines
● Formal Verification Flow
■ Case Studies
● Protocol Compliance : Bridge Validation
●  Ar
 Arbitration
● Interrupt Sorter 

● Memory Controller 

● SoCC
oC Conne
onnecti
ctivit
vity
y
● SoCP
oC Performan
erformancece

 Case Studies from IBM: Formal Processor Verification

 Verification Closure: Coverage Analysis & Integration with Simulation
For
ormal
mal Ve
Veri
rifi
ficatio
cationn Steps
Steps
 Verification Setup
■ Sanity checks
■ Cover points
■ Constraint selection (incremental constraint addition)
 Formal Engine Selection

 Managing subsequent verification setup changes

■ Use structural coverage metrics
● Branch coverage, Expression coverage
● FSM checks

 Putting it all together 

For
ormal
mal Eng
Engin
ine
e Selecti
lection
on
 Engines targeted towards
■ Bug Finding
■ Complete Proof 
 Selection depends upon
 . A T P 
PG 
   
G
■ Estimated Sequential Depth (user decision)     l  e
    b
   a
    h
■ Design characteristics    c
   a
  e
● More breadth than depth owing to     R       T
      A
concurrent FSM’s FV       S
H    
● Large sequential depths owing to counters, serial
 y   
b   
shift registers r   
i    
d    
 r a c t i o n
 t
 A b s
Str
truc
uctu
tural
ral Cov
Coveerage Metr
tric
ics
s
 Metrics point towards ‘unreachable’ parts of the design under the set of
appli
applied
ed cons
constr
traints
aints – val
validates
dates the ‘sanit
sanity’
y’ of the
the cons
constr
traints
aints
 Various set of metrics supported by tools
■ Branch coverage (Dead code analysis)
■ FSM checks
■ Expression coverage
 Point to over-constrained, restricted verification environment

 Managing constraint addition / removal

■ For every major constraint change, coverage analysis must be done to
filter out constraint related problems
■ Compare with past data
For
orma
mall Veri
Verifi
fica
cati
tion
on Flo
loww

M R Model +
 o  e  O D
 d  f  
i  
n Properties
i    e
f   v   e
 y   e  c 
 a  t  
h  r   o
 s   e  C  m
 s   o  p
 u m n  o
m  o  s   s 
 p  d   t    e
r   , Model Checker 
 t  
i    e  a
 o l   i  
n  b A 
n  o
 s  r   s 
 t   Stuck ? None of the
 a r 
 a YES
 s   c   Ab
 Abstractions working
 s   t   Indeterminate
Indeterminate
 e  ,
r 
 t  
i  
 o Results
Results
n
 s 
YES NO
cex Bug Hunting
Spurious
Spurious PASS (Directed Closure ?
NO Simulation
NO YES assisted MC)
Bug Hunti
unting
ng Mode
 To find silicon bugs, or when the overall proof procedure does not
yield results
 Functional Partitioning
■ Break design state space into logical sub-spaces
 Directed simulation
■ Guide the system to known states
 Formal Verification
■ To explore for bugs in the logical partition, separately
■ Usually apply this technique with some over-constraints
 Bug
Bu g Hun
Hunti
ting
ng Examp
xample:
le: De
Dealiling
ng wi
with
th FIFO’s
 Interesting points to check: Overflow/Underflow
■ If possible, “reduce FIFO depth / width”
■ Else light-weight simulation to “reach near corner points”
(a) Dynamic Simulation
(c ) Ap
(c)  Apply FV from this
to Bring System to a state onwards
Known State

Rd ptr 

FIFO almost FULL

reset
Wr Ptr  Rd ptr  (b ) Ap
(b)  Apply pin-
Push Predefined constraints (upon pins
Pattern of Data (that can that have no further
catch corner cases** ) role) to reduce cone of
influence
 Ag
 A g en
endda
 Introduction to Formal Verification
 Case Studies from TI: Protocol & Control Logic Verification
■ Methodology
● Planning for Formal Verification
● Property Coding Guidelines
● Formal Verification Flow
■ Case Studies
● Protocol Compliance : Bridge Validation
●  Ar
 Arbitration
● Interrupt Sorter 

● Memory Controller 

● SoCC
oC Conne
onnecti
ctivit
vity
y
● SoCP
oC Performan
erformancece

 Case Studies from IBM: Formal Processor Verification

 Verification Closure: Coverage Analysis & Integration with Simulation
Brid
Br idge
ge : Veri
Verifi
fica
cati
tion
on Targ
rgeets
 Functional Coverage Points:  Structural Coverage Points
■ Protocol Compliance ■ Dead Codes

■ Ad
 Address Decoding ■ FSM state / transition

■ Data Integrity ■ Structural checks (Out of

■ Ar
 Arbitration bounds, FIFO full / empty)
■ Performance / Latency

Black Box Verification White Box Verification

Functional Intent Implied Intent

Manual Mostly Automated

Parallel Pro
roto
toco
cols
ls
 Control Variables
■ Define state of transaction (Command, Burst, Response)
■ Define legal / illegal states (reachable / unreachable)

■ Define legal / illegal transitions between legal states

 Qualifier Variables
■  Ad
 Add extra information to transaction (Address, Data)
 Every protocol can be decomposed into functional layers
■  At
 Atomic Transactions: basic transactions like read / write
● Handshake mechanism: Something holds until something else happens
● Basic definition for start / end of request, response
■ Complex Transactions: adds extra sequential information
● e.g. to bind atomic transactions in a “Burst mode”
● Basic definition involves start / end of transaction ‘windows’
■ Ordering: remember history of pipelined events
■ Out of order execution: properties for inter-thread transitions
Seri
riaal Pro
roto
toco
cols
ls
 DATA & CONTROL shared on same transmission line
■ Difficult to separate

 Data-frames are spaced over several clock-cycles

■ Requires “HISTORY” info: Detecting a pattern requires

“rem
“remem
embe
beriring
ng”” previous
previous train
train of sign
signals
als – need
needs
s an FSM
FSM
■ E.g. Whether a train of 10 bits is a pre-amble or a post-amble

depends on the first 3 bits in that sequence.

Increasi
Increasing effect
effect of
of history
history on
on current state deco
decoding
ding

Simple Parallel Protocol: Complex Parallel Protocol Serial protoc

protoco
ol :
CS = (current values only) CS = (current values, pipeline history, CS = (history)
thread_id)
CS: Current state of protocol (transaction)
Prope
roperty
rty Codin
odingg for
f or Proto
rotocol
cols
s
■ Generic and small FSMs for assertion writing (for each layer)
■ FSMs for a higher layer use output flags of lower layer FSMs
■ Prove assertions bottom-up, using proven assertions as constraints
How the Protocol looks How the Properties  As
 Assume-Guarantee
at it look at it

Parallelism
Semantic (threads, Semantic
Information
Layer  bursts) Layer 

Transaction Frame Frame Sequencing Transaction

Layer  Layer 
Packet packets’ Packet
Packet Packet Layer 
Layer  validity Dependency
Encoding Data Stream Data Decoding among
Layer  Integrity Layer  assertions

 As
 Assertions in a particular
level depends on those
 AXI-OCP
 A XI-OCP Br
Bridge
 Characteristics:
■ 1 AXI interface, 3 OCP-2.0 1 OCP
Interfaces  AXI
 A XI  AXI-OCP
 A XI-OCP
 AXI-OCP
 A XI-OCP
■ Flop Count : 560 2 OCP
Bridge
Bridge
■ Pipeline depth (AXI) : 5
3 OCP
■ Threads Supported

■ Functional reset happens 2

Other Signals
cycles after reset is asserted
 Scope of FV:  Modeling:
■ Protocol compliance check ■ Divide & Conquer 

■ No data integrity checks ● Verify each AXI-OCP path in

isolation
 Initialization: Use Simulation
● Use Addresses to restrict
■ 2 cycles IDLE after primary reset
other paths
is asserted
 AXI-OCP
 A XI-OCP Br
Bridge

 Verification:
 AI
 AIPs used for AXI and 1 OCP
■  AXI
 A XI
 AXI-OCP
 A XI-OCP
 AXI-OCP
 A XI-OCP
OCP  Assu
 As su me:
Master  2
Properties Bridge
Bridge
■ First prove each path is
 Assu
 As su me:
3 Slave

independent Properties

■ “If ad
address/thr
thread-id
-id is Prove: Other Signals
 A3/
 A3/T3, then MCmd
Cmd at Slave
Properties Prove:

OCP interface
face 1 & 2 is
is master
Properties

alwa
lways IDLE
DLE”
 Handling indeterminate proofs:
■ Ap
 Apply assume-
■ Reduce pipeline depth, # threads
guarantee
■ Prove each burst type separately,
■ Prove assertions in

bottom-up fashion using pin constraints

■ Bug hunting: initialization via

simulation
 AXI-OCP
 A XI-OCP Br
Bridge
 Project Statistics:
■ Number
umber of Assert
Assertiions : 125
125
■ Number of Constraints : 60
■ Number of Restrictions : 15
■ Total Time : 3 weeks (Tool runtime: 35%)
 Sample Results
■ BUGS / Anomalies
● “Data was presented on the OCP port even prior to the corresponding
command acceptance on AXI”
● “Sequence of generated addresses incorrect for incrementing bursts”
 I2C Con
Contr
trol
oller
ler [Gorai et al, DAC-
DAC-2
2006]

OCP I2C

Parallel to Serial Bridge: Bug condition:

■ 697 Flops ■ I2C
I2C Cont
Controll rolleer in
i n Master Receive Mode
■ Number of byte transferred is Odd 
byt es to be transferred
Scope of Verification: ■ OCP fills
fil ls FIFO
FIFO with
wit h “ addre
ddr esse
ss es to
t o be
b e read”
read” , and
and I2C
I2C
■ Root-cause a silicon bug transmits
transmi ts the address from f rom FIFO
IFO (addre
(addressss phase)
■ I2C
I2C receives data from fr om Slave (data(data phase) and and set
■ Prove that a software

workaround is robust RRDY flag

■ OCP
OCP to read read a byte
by te from
fr om FIFO
FIFO and cl clea
earr RRD
RRDY Y flag
Result: ■ I2C detects
detects “ I2C-stop” condition condit ion and set ARD A RDYY flag
■ Found trace for the bug ■ OCP
OCP to cl clea
earr ARDY Flag
■ B u g : Near
Near stop
sto p condi
c onditiotion,n, RRDY
RRDY cannot be cleared cleared
■ Workaround valid for a
■ Workaround
Workaround:: Clear ARDY and and then
th en clea
cl earr RRDY flag
fl ag
specific sequence of
commands only
 I2C Co n tr
troo l l er 
 Verification Steps
■ OCP and I2C VIP’s were configured and attached
■ Test pins, redundant OCP data pins were tied to constant values
■ Protocol Compliance checks
● Proven I2C assertions were then used as constraints
■ Directed simulation to
● Confi
onfigu
gured
red regi
registers for
for MR
MRx mode,
ode, for
for 1 byte tran
transfer 
sfer 
● Load transmit FIFO with data

● Take the design to end of I2C address phase

 The Event window where RRDY could be cleared divided into 4 parts
■ Based on Set Flags & Stop conditions

 Bug located: RRDY could not be cleared in one region only

■ Effort: 2 months, mainly to write I2C AIP. Reuse: 2 weeks
OCP A rb
rbii t er
 Characteristics:
OCP1
■ 4 OCP inputs (from masters) 1 OCP
● ocp1, ocp2 with pipeline OCP2
● ocp3, ocp4 no pipeline
2 OCP
OCP
OCP
■ 10 OCP outputs (to slaves) OCP3  Ar
 A
 Arrrbbiitter 
 A er 
■ Configurable Internal Register 7 OCP

■ Flops: 637 OCP4

10 OCP
■ Round Robin Arbitration with
priority-reset on 2 cycles IDLE
 FV Complexity
■ Bypass functionality ■ 32 bit address lines
 Scope of FV: ■ Internal Decoding Logic

■  Ar
 Arbitration Exceptions:
■ Protocol compliance check
● Bypass
■  Ar
 Arbitration Check
● Configuration register 

Bypass Check
OCP A r b i t er 
OCP1
Modeling:
1
■ Protocol Compliance Check OCP2
●  As
 Assume-guarantee 2
■ Bypass Check OCP
OCP
OCP3  Ar
 A
● Whenever there is request from  Arrrbbiitter 
 A er 
7
OCP3 to dedicated slave, it must
get serviced with zero-cycle OCP4
delay
■ Ar
 Arbitration Check
Bypass Check
● Disable Configuration reg
Identify serviced masters:
● Disable Bypass
■ Check for ByteEn pass-through
■ Define events: Start / End of
■ Tie ByteEn to unique values (to
Requests
identify master requests)
■ Map the 10 requests on Slave

side as 4 grants for masters

OCP A r b i t er 
 Create abstract arbiter model (assertions)
■ Round Robin Scheme with priority reset

OCP1

OCP
OCP
 Ar
 A
 Arrrbbiitter 
 A er 
OCP4

Model
Model ‘reque
‘r equests
sts’’ as - Model ‘grants’ as
as considered
considered
considered by arbiter  by arbite
arbit er 
- Map ‘10’
‘10’ grants to ‘4’ grants

‘4’ requests  Ab

 A b s t r act
ac t ‘4’ grants
 Ar
 A r b i t er
Model
OCP A r b i t er 
 Challenges
■ Due to decoding logic, the proof complexity is very high
● Ad
 Address constraints: Allow only some number of active slaves
● Restrict accesses: Take a set of 3 masters at a time

● Using proved properties as constraints

■ Pipelining creates problems for accurate arbitration checks

● Need to keep external history for pipeline: External buffer keeps track of
‘number of pending requests’  counter implementation

 Sample BUGS / Anomalies discovered:

■ Bypass should have zero-cycle latency, whereas 1 to 2 cycle latency was
possible under error conditions
■ Ar
 Arbitration Error in the first cycle : uninitialized registers
■Priority reset after idle cycles was not happening always
Int
nteerr
rrup
uptt Con
Contr
trol
olle
lerr [Bisw
[Bi swa
as et al, CDNLive-
NLive-2
2005]

 Characteristics:
■ Sorting between 32/48/96 interrupts (configurable)
■ Mask and Priority configurable for each interrupt
■ OCP Interface (for configuration)
■ Flop Count : 785 (for 32 interrupts)
■ Output after 8 clock cycles (sorting in groups of 4)
 Scope of FV:
Disabled
■ Protocol compliance check
■ Sorting Algorithm Verification In t er r u p t s Sorting
Sorting w i n n er  
 Al
 A
 Alllggoorriitthhm
 A m
OCP
0 1 1 0
Prop_s
i
Int
nteerr
rrupt
upt Cont
ontro
rolllle
er 
 Challenges
■ Mask & priority registers programmable via OCP interface
● Many combinations possible

 Solutions:
■ Prove integrity of OCP data writes to internal registers
■ Then apply stability constraints on register 
■ For intra-class checks, tie-off other interrupts
■ For inter-class checks, constrain values within each class to be same
■ Compare expected winner (computed by Verilog function) with actual
winner 
Memo
mory
ry Con
ontr
trol
olle
ler 
r 
 Characteristics:
■ Interface to SARAM core
■ Handshaking (with stall) with other bank-controllers
■ Total number of request buses (cpu, dma etc.) :9
■ Fixed Intra as well inter priority between requests
■ Pipelined CPU requests, DMA burst requests
■ Flop Count : 3500
■ Embedded SARAM, BIST controller 
 Scope of FV: Serialization check
● “Order of grants must be same as the requests”
● End-to-end property, encompasses all behavior of controller (mux, stall,

arbitration, etc.)
Memo
mory
ry Con
ontr
trol
olle
ler 
r 

Membist Disabled Pin Constraints Memory removed

SARAM
SARAM channel
channel
SARAM Cores
 Ar
 A r b i t r ati
at i o n l o g i c Strobe & inputs Output Stall Signals
Serialization
Stall
Stall behavio
behavior  r 

Input (global) Stall

signals
0
Ready Signals
Request and Address
Signals
(Constrained)
Memo
mory
ry Con
ontr
trol
olle
ler 
r 
 Solutions: White-box and Controlled Verification
■  Al
 Allow access to each core
● But intelligently disable parts/bits of address bus

■ Keep out other bank controllers

●  As
 Assume a floating primary input signaling their stall behavior 
■ Identify and code assertions for internal points

● E.g. Code and prove assertions for stall module

■ Then use them as constraints

 Results:
■ More than 30 bugs found, most of them corner cases (related to flush

functionality of pending buffers and stall behavior, which in turn lead to

serialization assertion failures)
■ Effort:

●  As
 Assertions : 215, Constraints : 12
● Total Time : 8 weeks (Tool runtime: 50%)
 Ag
 A g en
endda
 Introduction to Formal Verification
 Case Studies from TI: Protocol & Control Logic Verification
■ Methodology
● Planning for Formal Verification
● Property Coding Guidelines
● Formal Verification Flow
■ Case Studies
● Protocol Compliance : Bridge Validation
●  Ar
 Arbitration
● Interrupt Sorter 

● Memory Controller 

● SoCC
oC Conne
onnecti
ctivit
vity
y
● SoCP
oC Performan
erformancece

 Case Studies from IBM: Formal Processor Verification

 Verification Closure: Coverage Analysis & Integration with Simulation
SoC Conne
onnectivity
ctivity Verifica
rification
tion
 Bus Archit
Architec
ecture
ture Conn
Connec
ectitivit
vity,
y, I/O
I/O Pin-Mux
n-Muxiing Logic
Logic
 Errors caught easily and upfront in the design
■ Traditional verification involves exercising the complete communication

protocol from one end point to find any bug

■ Verification starts much earlier, issues get resolved quicker 

■ Errors detected: tieoffs, unconnected points, wrong connections

 Connectivity Description:
■ Interface Definition: Port, direction, width, high/low, etc.

■ Conne
onnecti
ctivit
vity
y table: IP
IPx-Porty
x-Porty to IPa-P
IPa-Portb
ortb
 Results:
■ Properties: ~ 5000 (at top-level of a 5M gate chip)

■ Time: 6 hours

■ Note: not
not chec
checking
king for val
validity
dity of
of xls itself,
tself, nor
nor IP functi
function
onali
ality
ty
 To be done: Interrupt and Clock-reset connectivities
SoC Conne
onnectivity
ctivity Verifica
rification
tion

Track 1

Connectivity PSL Gen

SPECIFICATIONS
XLS Scripts

PSL

RTL
Formal
Manual / Automated
Verification
Flow Generates RTL

Track 2

Connectivity related
bugs found
 SoC Ba
Bandwidth
ndwidth Ve
Verifica
rification
tion [Bhat
[B hatia
ia et al, DAC-20
DAC-2007
07]]

D
 Memory bandwidth validation: i  
 s 
 p
 C 
 a
DMA l   m E 
 a n  J 
■ Multiple masters accessing the  g P 
 y   e
 U r  i   E 
 a n
n  e  G
memory subsystem Memory
Traffic
i  
 t  
I  
 /  
F 

Controller 
■ Master starvation affects system

performance
Interconnect
■ Traditional methods: simulation,

FGPA / emulation, running real

 ARM I  
software on real silicon Bridge  S 
P 
 G
■ Formal Verification needed for r 
 a
 p
corner case analysis on real h 
i  
 c 
 s  A 
F  High Speed
RTL – earl
early
y enou
enough
gh to fifix design
design I  
 /  
F  E 
Serial Port
bottlenecks
 Properties:
■ Rate, e.g. " Requests are produced
produ ced every
every ‘n’ time
tim e units"
units " : t(Re
t(Request[i+[i+1]) - t(Re
t(Request[i])
[i]) = n
■ Latency, e.g. " Response is gene g enerate unit s after Request " :
ratedd no more than ‘k’ time units
t(Re
(R esponse[i])
[i]) - t(Re
(R equest[i])
[i]) <= k 
■ Throughput, e.g. "at least ‘W’ Request events events will
wi ll be produced in any
any period of ‘T’ time units" :
t(Re
t(Request[i+[i+W]) - t(Re
(R equest[i])
[i]) <= T 
 SoC Ba
Bandwidth
ndwidth Ve
Verifica
rification
tion

Verif
Verificatio
ication
n SS + Master
Master Mod els + PSL
Master
Mast
Ma er 1
ster

Mast
Master
er 2 Ch0 DUT
DB
M
.
. U Master
Mast er 2 Ch 1 Interconnect
. X
DI

SEL
EMIF
MIF
Master
Mast er 3
Master

Master
Mast er 4
Master
 Ag
 A g en
endda
 Introduction to Formal Verification
 Case Studies from TI: Protocol & Control Logic Verification
 Case Studies from IBM: Formal Processor Verification
■ Complexities of High-End Processors

■  Al
 Algorithmic Methods for Reducing Verification Complexity
■ Processor Verification Case Studies
● Sequential Equivalence Checking (SEC)

● Instruction Dispatch Case Study

● Instruction Fetch-Hang Case Study

● Floating-Point Unit Verification

● Load-Store Verification

 Verification Closure: Coverage Analysis & Integration with Simulation

POW
POWER
ER5
5 Ch
Ch i p : It’
t’ss Ugl
Uglyy

 Dual
Dual pSerie
pSeriess CPU
CPU
 SMT core (2 virtual procs/core)
 64 bit PowerPC
 276 million transistors
 8-way superscalar 
 Split L1 Cache (64k I & 32k D) per core
 1.92MB shared
shared L2 Cache
Cache >2.0 GHz
GHz
 Size: 389 sq mm
 2313 signal I/Os
 >>1,000,000 Lines HDL

POWE
POWER
R arch itectur
it ecture:
e:
 Symmetric multithreading
 Out-of-order dispatch and execution
 Various address translation modes
 Virtualization support
 Weakly ordered memory coherency
Moo
oore
re’s
’s La
Law:
w:  A
 AND
ND It’
It ’ s Get
Gettt i n g Ug
Ugll i er 
 Complexity increases for POWER5, POWER6,
POWER7, …

 Increase in # transistors / chip

■ Contributes somewhat to complexity

■  Al
 Alleviated by modularity
● N identical proc cores on chip

■  Al
 Alleviated by increased RAM size

 Increase in speed
■ Contributes significantly to HDL

complexity
■ CEC methodology requires 1:1 latch

correspondence between circuit, HDL

● CEC=Combinational Equiv Checking
Com
ompl
ple
exi
xiti
tie
es of
o f Hig
High-
h-E
End Pro
roce
cess
ssor
ors
s
 CEC methodology forces HDL to acquire circuit characteristics
■ Word-lev
ord-level
el operati
operation
ons,
s, isom
somorphi
orphisms
sms broken
broken by
by sel
self-test
f-test logi
ogic
● Self-test logic: much more intricate than mere scan chains
● Reused for functional obligations: initialization, reliability, …

■ Run-time monitoring logic entails similar complexities

Com
ompl
ple
exi
xiti
tie
es of
o f Hig
High-
h-E
End Pro
roce
cess
ssor
ors
s
 CEC methodology forces HDL to acquire circuit characteristics
■ Timing demands require a high degree of pipelining
●  An
 And multi-phase latching schemes

■ issues: redundancy added to HDL

Placement issues:
● Lookup queue routes data to 2 different areas of chip  replicate

■ Power-savings logic complicates even simple pipelines

■ Desig
Designn HDL
HDL becomes diffidif ficul
cult-to-pa
t-to-pars
rsee bit-le
bi t-level
vel representatio
representationn
● Industrial FPU: 15,000 lines VHDL vs. 500 line ref model

■ Sequential synthesis cannot yet achieve necessary performance goals

●  An
 And need integration of pervasive logic: self-test, run-time monitors
Com
ompl
ple
exi
xiti
tie
es of
o f Hig
High-
h-E
End Pro
roce
cess
ssor
ors
s
 Numerous techniques have been proposed for Processor Verification

 Satisfi
atisfiab
abiility Modulo
odulo The
Theori
ories
es replaces word-lev
ord-level
el operati
operation
ons
s by
by
function-preserving, yet more abstract, Boolean predicates
■ Replace complex arithmetic by arbitrary simple (uninterpreted) function

■ Reduce arithmetic proof to simpler data-routing check

 Numerous techniques to abstract large memories

 Numerous techniques to decompose complex end-to-end proofs

 …
 Com
ompl
ple
exi
xiti
tie
es of
o f Hig
High-
h-E
End Pro
roce
cess
ssor
ors
s
 Difficult
ifficul t to find place
placess to attempt
ttempt such abstractions on
high-
hig h-eend de
d esigns
sign s
■ Word-level info lost on highly-optimized, pipelined, bit-sliced designs
● Designs are tuned to the extent that they are “ almost wrong”
wrong” R. Kaivola

■ Time-consuming + error-prone to manually abstract the implementation

 Ab
 Abstractions may miss critical bugs
■ Remov
emoval
al of bitve
bitvector
ctor nonli
nonline
neari
aritities
es are lossy
ossy
● May mi
miss bugs
bugs due
due to roun
roundi
ding
ng mode
modes,
s, overfl
overflow,
ow, … if not careful

 Need to verify a more abstract model?

■ Developing, maintaining such a model is very expensive
■ Emerging SEC technology: may close abst
abstra
ract
ct vs CEC model gap
CEC model
Com
ompl
ple
exi
xiti
tie
es of
o f Hig
High-
h-E
End Pro
roce
cess
ssor
ors
s
 Indus
ndusttriall
rially,
y, “Form
“Formal
al Proce
Processo
ssorr Veri
Verifficat
cation”
on” refers
refers to proc com
compone
ponents
nts
■ E.g., verification of FPU, Cache Controller, Branch Logic

 “Dispat
spatch to Com
Compl
pleti
etion
on”” proofs for
for processors
processors as compl
complex
ex as Pent
entium,
um,
POW
POWER,
ER, … are
are intractable given today’s technologies

 In this tutorial, we focus on various case studies of the verification of

processor components
 Ag
 A g en
endda
 Introduction to Formal Verification
 Case Studies from TI: Protocol & Control Logic Verification
 Case Studies from IBM: Formal Processor Verification
■ Complexities of High-End Processors
■  Al
 Algorithmic Methods for Reducing Verification Complexity
■ Processor Verification Case Studies

● Sequential Equivalence Checking (SEC)

● Instruction Dispatch Case Study
● Instruction Fetch-Hang Case Study
● Floating-Point Unit Verification
● Load-Store Verification

 Verification Closure: Coverage Analysis & Integration with Simulation

Redu
duci
cing
ng Ve
Veri
rifi
fica
cati
tion
on Com
Compl
ple
exi
xity
ty
 We discussed characteristics of high-end processors that make them
large, complex, difficult to verify
■ High degree of pipelining

■ Multi-phase latching schemes

■  Ad
 Addition of sequential redundancy
■ Loss of word-level and isomorphic characteristics

 Some may be addressed using automatic vs. manual techniques

Semi
mi--For
ormm al Ver i f ic
icaat i on (SFV)
 SFV: hybrid between simulation and FV
■ Uses resource-b
resource-bou
ound
nded
ed formal
formal algos to amplify simulation results
■ Uses simulation to get deep into state space

 Hybrid approach enables highest cove

coverage
rage,, if design too big for FV
■ Scales to large designs, without costly “manual abstraction”

 Incomplete, like simulation

■ May trade proof capability for high semi-formal coverage
 Pipelining
 High-end designs often use aggressive pipelining to break
computations across multiple clock periods
■ Increases #state elements; diameter 

 Min-area peripheral retiming may be used to drop latch count

■ Used as an automatic, property-preserving transform
“Transform
“Transformation
ation-Bas
-Based
ed Verif
erification
cation using General
eneraliized Retim
etiming”
ng” CAV01
 Mul
ulti
ti--Phase Latc
Latchi
hing
ng Sch
cheemes
 High-end designs often use multi-phase level-sensitive latches to better distribute
propagation delays vs. edge-sensitive flops
■ Increases #state elements; diameter 

 Phase abstraction can automatically convert multi-phase to simple-delay model

■ Unfold next-state functions modulo 2, removing oscillator + ~1/2 latches

“ Automatic Genera
Generalize
lizedd Phase
Phase Abstraction for
f or Formal Ve
Verification”
rifi cation” ICC
ICCAD05
AD05
 Sequ
queent
ntial
ial Redu
dund
nda
anc
ncyy
 High-end designs use sequential redundancy to minimize propagation delays
 Pervasive logic  sequential, yet disabled, logic intertwined with all latches
■ Increases #state elements; breaks isomorphisms, word-level properties

 Numerous techniques exist to identify, merge redundant gates

“Expl
“Exploiti
oiting Susp
Suspected
ected Redu
Redunda
ndancy
ncy wi
without
hout Proving
roving It”
It” DAC05
Reduc
ducin
ing
g Com
Comple
plexi
xity
ty throu
th rough
gh Tra
Trans
nsfor
forms
ms
 High-end design complexities can be (partially) alleviated through
automated transformations and abstractions

 Our solution at IBM leverages transformation-based verification

■ Fully automated, scalable (S)FV toolset SixthSense

■ Iterate synergistic transforms to decompose large problems into simpler

sub-problems
● Localization, retiming, phase abstraction, redundancy removal, …

■ Leverage various proof+falsification engines to solve resulting problems

● Semi-formal search, bounded model checking, interpolation, …
 Tra
rans
nsfo
form
rma
ati
tion
on--Ba
Base
sed
d Veri
rifi
fica
cati
tion
on

Counterexample
Design +
Trace consistent with
Properties
Original Design
50000 registers

Redundancy SixthSense
Problem Removal
Redundancy-
decomposition Engine
removed
via synergistic
40000 registers trace
transforms  All transformations
are transparent to the user 
Localization
Redundancy-
Engine
 All results are in terms removed,
of original design localized trace
150 registers

Reachability
Engine
 Redu
duci
cing
ng Ve
Veri
rifi
fica
cati
tion
on Com
Compl
ple
exi
xity
ty
 Au
 Automated techniques are continually increasing in capacity

 However, for complex proofs, manual techniques are critical to push the
capacity barrier 
■ Choi
hoice of
of testbench
testbench boun
bounda
dariries
es
■ Manual abstractions to reduce design complexity

■ Underapp
nderapproxi
roxim
mation
ations
s and
and overapproxim
overapproximati
ation
ons
s
■ Strategic development of constraints and properties

 The
The best
best strat
strateg
egy
y often
often depe
depend
nds
s upon
upon som
some
e know
knowlledge
edge of
of avai
availabl
able algo
algos
s
 Ag
 A g en
endda
 Introduction to Formal Verification
 Case Studies from TI: Protocol & Control Logic Verification
 Case Studies from IBM: Formal Processor Verification
■ Complexities of High-End Processors
■  Al
 Algorithmic Methods for Reducing Verification Complexity
■ Processor Verification Case Studies

● Sequential Equivalence Checking (SEC)

● Instruction Dispatch Case Study
● Instruction Fetch-Hang Case Study
● Floating-Point Unit Verification
● Load-Store Verification

 Verification Closure: Coverage Analysis & Integration with Simulation

Pro
roce
cess
ssor
or Ve
Veri
rifi
fica
cati
tion
on Con
Conce
cept
ptss
 A verification Testbenchcomprises
■ Design components under verification

■ Input assumptions
● May be represented as constraints
● Or may be overridden by random drivers

 We refer to either scheme as a driver 

■ Properties to be checked
● Coverage metrics, assertions, equivalence vs. ref model

■ Initialization information
● Initial states impact reachable states which impacts pass
pass vs fail
fail
Pro
roce
cess
ssor
or Ve
Veri
rifi
fica
cati
tion
on Con
Conce
cept
ptss
 Verifying a proc component is basically the same as verifying any design
■ However, due to high-performance logic, verifying architectural
properti
properties
es typically
typically requir
requires
es a Tes
Testben
tbench
ch which is too large for proofs
proofs

 Options:
1. Develop unit-level Testbench without worrying about proof feasibility
2. Develop minimal Testbench encompassing only functionality to be
verified
Pro
roce
cess
ssor
or Ve
Veri
rifi
fica
cati
tion
on Con
Conce
cept
ptss
1. Develop unit-level Testbench without worrying about proof feasibility
■ Unit-l
nit-lev
evel
el testbenc
testbenche
hes
s often
often bui
built for
for si
sim regardl
regardles
ess
s
● Use of synthesizable language  reusable for FV, emulation, …

■ Leverage semi-formal verification for bug-hunting

● With luck, robust tool may yield some proofs regardless
● But likely need hand-tweaking of Testbench for proofs

 Easier for
fo r non
n on-e
-exp
xperts
erts to leverage (S(S)F
)FV
V
■ Manual abstraction is time-
time-consuming and difficult
d ifficult
■ Even if using experts to abstract, disperses formal spec effort

 Easier to specify desir

desireed prop
p ropeertie
rti es at unit
un it level
level
■ Verify functionality vs. verify blocks
■ Difficult to cover architectural properties on small blocks
 Pro
roce
cess
ssor
or Ve
Veri
rifi
fica
cati
tion
on Con
Conce
cept
ptss
2. Develop minimal Testbench encompassing only functionality to be verified
■ Requires dedicated effort to enable FV
● Checks and Assumptions may be reusable in higher-level sim
● But often need to develop a higher-level Testbench for sim

■ Block-level Testbench often more complex to define than unit-level

● More complex, prone to change, poorly documented input protocol

 Works very well if done by designer at design granularity level

■ E.g. designe
designerr of Data
Data Prefet
Prefetch
ch bui
building Tes
Testben
tbench
ch at
at that lev
level
el

 Highe
igh er cha nc e of pro
ch ance  proo
ofs, corner
corner-cas
-cas e bu
bug s on smaller
s maller Te
Testbench
st bench
■ Data Pref
Prefetch
etch is much
much smal
smalller tha
than
n enti
entire
re Load
Load-Store
-Store Unit
Unit!!
Basi
Ba sic
c CP
CPU Ar
Arch
chit
iteect
ctur
ure
e

Load-
Store
Unit (LSU)

Floating
Instr  Instr  Point Global
Fetch Decode Unit (FPU) Completion
Unit Unit Unit
Fixed-
(IFU) (IDU) (GCU)
Point
Unit (FXU)
Branch
Execution
Unit (BXU)
 Ag
 A g en
endda
 Introduction to Formal Verification
 Case Studies from TI: Protocol & Control Logic Verification
 Case Studies from IBM: Formal Processor Verification
■ Complexities of High-End Processors
■  Al
 Algorithmic Methods for Reducing Verification Complexity
■ Processor Verification Case Studies

● Sequential Equivalence Checking (SEC)

● Instruction Dispatch Case Study
● Instruction Fetch-Hang Case Study
● Floating-Point Unit Verification
● Load-Store Verification

 Verification Closure: Coverage Analysis & Integration with Simulation

Ps eu d o Cas e-St u d y : SEC
 Perf
erform
orman
ance
ce-cr
-criitica
call de
designs
signs undergo many functionality-preserving
transforms
■ Timing optimizations

■ Power optimizations

■  Ad
 Addition of test and run-time monitoring logic

 Using SEC to validate such changes can dramatically reduce verif 

resources
Outputs
Initialized
OLD Design

Inputs =?

Initialized
NEW Design
Ps eu d o Cas e-St u d y : SEC
 Due to synthesis limitations, transforms often done manually
■ Entail risk of (late) error introduction
■ Subop
uboptitim
maliti
alities
es tolerated to mi
minimi
nimize error
error risk
risk

 SEC is very easy to run:

run: no di
diffi
fficul
cult Testbe
stbench setup
setup

 SEC often runs quickly, and exhaustively checks equivalence

■ Eliminates risk; enables more aggressive, later optimizations

 Without
hout SEC
SEC, functi
unction
onal
al verif
verif is rerun upo
upon
n HD
HDL mod
modiificat
cation
■ CPU-months of simulation regressions
■ FV testsu
testsuitite
es re-run
re-run
Ps eu d o Cas e-St u d y : SEC
 SEC also useful for:
■ Reference-model style verification
● Less circuit-like model serves as extensive “property set”

■ Demonstrate bwd-compatibility of design evolutions

● E.g. if HW multithreading added to old design, SEC can confirm “single
thread”
thread” mode
ode of new matches old

■ Quantify: functional changes do not alter unintended behavior 

● If opcode1 handling changed, constrain and check equivalence across all
other opcodes
 Ag
 A g en
endda
 Introduction to Formal Verification
 Case Studies from TI: Protocol & Control Logic Verification
 Case Studies from IBM: Formal Processor Verification
■ Complexities of High-End Processors
■  Al
 Algorithmic Methods for Reducing Verification Complexity
■ Processor Verification Case Studies

● Sequential Equivalence Checking (SEC)

● Instruction Dispatch Case Study
● Instruction Fetch-Hang Case Study
● Floating-Point Unit Verification
● Load-Store Verification

 Verification Closure: Coverage Analysis & Integration with Simulation

 Ins
nstr
truc
ucti
tion
on Dis
ispa
patc
tchh Ca
Case Stu
tudy
dy
 Concerns the portion of the Instruction Decode Unit responsible for
routing valid instruction groups to execution units
Flushes / Rejects

     s      s      s

     n      n      n
     o      o
       i      o
       i
       i        t        t
       t
     c      c      c
Instr       u      u      u
     r      r      r
       t        t        t
     s      s      s
Fetch      n      n Instr       n
Staging        I Instr         I        I
Unit Dispatch To
Decoded Logic Buffer 
+ Logic Execution
Decoder  ICache Units
Line

Stall
Ins
nstr
truct
uctio
ion
n Dis
ispa
patc
tch:
h: Veri
rifi
fica
cati
tion
on Goa
oals
ls
 Verify that Dispatched instructions follow program order, despite:
■ Stalls
■ Flushes (which roll back the Dispatch flow to prior Instr Tag)
■ Branch
ranch Mispredicts
spredicts (sim
(similar to Fl
Flushe
ushes)
s)
■ Rejects (which force re-issue of instructions)
■ Bypass path
 Inst
nstruct
ruction
ion Disp
ispaatch: Logi
Logicc Bound
B ounda
arie
riess
 First choice: what logic to include in Testbench?
■ Indep
Indepen
ende
dent
nt veri
veriff of Ins
Instr
tr Buffer, Stagi
taging Logic,
Logic, Di
Dispatch
spatch Logic
Logic attrac
attractitive
ve
from size perspective, but hard to express desired properties
● Decided to include these all in a single testbench

■ Decoded instructions were mostly data-routing to this logic

● As
 Aside from special types (e.g. Branch), this logic did not interpret instructions
● Hence drove Testbench at point of decoded instruction stream

■ Thoug
houghh infrequ
infrequenentt duri
during
ng norm
normal
al exec
executi
ution
on,, this
this log
logiic mus
mustt react
react to
Rejects, Stalls, Flushes at any point in time
● Hence drove these as completely random bits
Ins
nstr
truc
ucti
tion
on Dis
Dispa
patc
tch:
h: Inp
Input
ut Mode
Modeliling
ng
 Second choice: how to model input behavior 
■ Needed to carefully model certain instruction bits to denote type
● Branch vs. Regular types

■ Other bits were unimportant to this logic

● Precise modeling: allow selection of exact legal decodings
 Manually intensive, and large constraints may slow tool

● Overapproximate modeling: leave these bits free

 Ideal
deal since
since overapproxi
overapproxim mation
ation ensures
ensures no missed bugs
bugs
 But large buffers imply large Testbench!

● Instead, used the bits strategically

 Tied some constant, to reduce Testbench size
 Randomized some, to help ensure correct routing
 Drove one bit as parity, to facilitate checks
 Encoded “program order” onto some bits, to facilitate checks
Ins
nstru
truct
ctio
ion
n Dis
Dispa
patch
tch:: Prop
Propeert
rtyy Mod
Modeeliling
ng
 Third choice: how to specify properties to be checked
■ Dispatches follow instruction order:
● Easy check
check since dri
driver
ver uses
uses bi
bits of instr to speci
specify
fy program order 
order 
● Check for incrementing of these bits at Dispatch

■ Flushes / Stalls roll back the Dispatch to the proper instruction

● Maintain
aintain a ref
reference
erence mod
model
el of corr
correc
ectt Dispat
spatch Instr Tag

■ Dispatched instructions are valid

● Check that output instructions match those driven:
 Correct “parity” bit
 Patterns never driven for a valid instruction are never read out
 Drove “illegal” patterns for instructions that must not be read out
 Ins
nstru
truct
ctio
ion
n Dis
Dispa
patch
tch:: Proo
Prooff Comp
Comple
lexi
xity
ty
 Recall that driver tricks were used to entail simpler properties
■ Check for incrementing “program counter” bits in Dispatched instr 

 Without such tricks, necessary to keep a reference of correct instruction

■ Captured when driven from Decoder; checked when Dispatched
● More work to specify
● Larger Testbench, more complex proofs, due to reference model

 Short
hortcu
cutt poss
possiible since
since this log
logiic trea
treatted most
ost instr
nstruc
uctition
on bits
bits as data
data
■ If Testbench included execution units, shortcut would not be possible
Ins
nstru
truct
ctio
ion
n Dis
Dispa
patch
tch:: Proo
Prooff Comp
Comple
lexi
xity
ty
 Philosophy: “don’t be precise where unnecessary for a given testbench”
is very powerful for enabling proofs
■ Instr Dispatch
spatch requires
requires prec
preciise Instr Tag modeling due to flushes; does
not care about decoded instr 
■ Some downstream Execution Units don’t care about Instr Tag; require

precise instr code

 However, this occasionally runs contrary to “reusable properties”

■ E.g., “patterns which cannot be driven are not Dispatched” check

canno
cannott be reused
reused at high
higher
er leve
levell, where
where overcon
overconstr
straint
aints
s are not
not present
present
 Ins
nstru
truct
ctio
ion
n Dis
Dispa
patch
tch:: Proo
Prooff Comp
Comple
lexi
xity
ty
 Semi-Formal Verification was main work-horse in this verification effort
■ Wrung out dozens of bugs
● Corner-cases due to Flushes, Stalls, Bypass, …

■ For
For SFV
SFV, biasing
biasing of
of random
random stim
stimulus im
import
portan
antt to ena
enable
ble sim
sim to provi
provide
de
a reasonable sampling of state space
● Neede
eeded
d to bias
bias dow
down
n tran
transfers
sfers from
from IFU,
IFU, else Instr Buffer
uffer always
always full
full

 Param
arameterizing
eterizing size of
of Instr Buffer small
smaller,
er, setti
setting more deco
decode
ded
d instr 
bits constant helped enable proofs
 Ag
 A g en
endda
 Introduction to Formal Verification
 Case Studies from TI: Protocol & Control Logic Verification
 Case Studies from IBM: Formal Processor Verification
■ Complexities of High-End Processors
■  Al
 Algorithmic Methods for Reducing Verification Complexity
■ Processor Verification Case Studies

● Sequential Equivalence Checking (SEC)

● Instruction Dispatch Case Study
● Instruction Fetch-Hang Case Study
● Floating-Point Unit Verification
● Load-Store Verification

 Verification Closure: Coverage Analysis & Integration with Simulation

Ins
nstr
truc
ucti
tion
on Fetc
tchh Case Stu
Study
dy
 Motivated by an encountered deadlock:
■ Instruction Fetch Unit stopped fetching instructions!
InitFetch

 .      s
       ’
     r        h      r
       t      c        t
Instr       s       t      s
     n      e      n
Fetch        I        F        I Instr 
ICache Buffer 
State
Machine

Mispr
Mispred
edic
ictt Flus
Flush
h Instructions

Branch To
Instr 
Execution Execution
Pipeline
Unit Units
Fet c h -Han g Cas e St
St u d y
 Suspe
uspected:
cted: Ins
Insttr Fetch St
State Mac
Machine
hine (IF
(IFSM) can
can enter
enter ilillegal
egal hang state

 First tried to isolate IFSM in a Testbench

■ Despite simple previous Figure, formidable to specify accurate driver

due to numerous ugly timing-critical interfaces

■ With unde
underconst
rconstrrained
ained Test
Testbenbench,
ch, checked
checked whether
whether IFS
IFSMM could enter a
state whe
where
re it did not
not init
initiiate Instr
Instr Fetch afte
fter InitFe
InitFetch
tch com
command
and

■ Discov
scovered
ered a hang
hang state – yet
yet could
could not
not readi
readily extr
extrap
apolate
olate the
the tiny
tiny
counterexample to one of the entire IFU+IDU
● Exhibited input timings thought to be illegal
● Yet designer was able to discern a scenario which, if producible, could lead

to deadlock
Fet c h -Han g Cas e St
St u d y
 Given extrapolated scenario, next attempted to produce that scenario on
larger IFU+IDU components
■ Interfaces at this level were very easy to drive

● Ab
 Abstracted the ICache to contain a small program

■ However, VERY large+complex Testbench

● Could not get nearly deep enough to expose condition which could reach
hang state

 Used 2 strategies to get a clean trace of failure:

■ Tried to define the property as an earlier-to-occur scenario

■ Constrained the bounded search to extrapolated scenario

 Fet c h -Han g Cas e St
St u d y
 Extrapolated scenario:
■ Stream A is being executed, encounters a branch to B (to be taken)
■ Instructi
Instructionons
s in-li
in-line from
from A are sti
stilll bei
being fetched
fetched to Instr Buffer at
time of branch resolution
■ Somehow the in-line instructions are not immediately invalidated
■ Fetch to B is delayed until exactly the point that the in-line
instr
nstruc
uctitio
ons are
are dispatch
dispatched
ed out
out of thethe Instr Buffer (Init
(InitF
Fetch)
etch)
● This can put the IFSM into the dangerous hang state
■ Som
omehehow
ow a Mi
Mispred
sprediict Flush doe
does
s not
not ge
gett tririgg
ggered
ered (to
(to squa
squash
sh the
  e in-line instructions) to break IFSM out of the hang state
  m
   i
   t

 Difficult to discern how to complete scenario to end up in deadlock

 Fet c h -Han g Cas e St
St u d y
 Reacha
chabilit
ility
y of hang state on full Test
estbenc
benchh possibl
possible
e wi
with BM
BMC
■ However, normal execution always kicked IFSM out of hang state

 But trace provided useful insight: an in-line instruction may avoid

invalidation if fetched during 1-clock window where branch is dispatched
■ This information, plus the timing at which activity occurred during the

BMC trace, was used to constrain a deeper BMC check

Strea
tream
m A hits branch to B,
B, Branch resolve
resolv ed Hang state
Hang s tate
predicted
predicted “ not taken”
taken” “taken” entered

time

Inli
nline
ne A fetch concur
con current
rent w/ Inli
nline
ne A dispa
disp atch Hang not
branch-to-B dispatch concurrent
concur rent w/ B fetch
fetch broken?
Fet c h -Han g Cas e St
St u d y
 Const
onstraine
rainedd B MC run exposed the
BM th e dea
deadlo
dlock
ck sit
situa
uatio
tion!
n!
 A
 Ad
ddress B exactly same address as in-line instructions from A
from A which
spurious
spuriouslly mad
made
e it through
through Instr Buffe
uffer 
■ Other conditions required, e.g. no spuriously dispatched branches

 However, removing constraints to check for alternate fail conditions (and

validity of fix) became intractable even for BMC
■ Tried manual abstractions of Testbench to cope with complexity

● Repl
eplaced
aced Inst
nstr Buffer
uffer wi
with small
smaller
er tim
timing-accu
ng-accurate
rate abstr
abstracti
action
on
● Still intractable due to depth, size of Fetch logic

■ Realized we needed purely abstract model to approach a proof 

Fet c h -Han g Cas e St
St u d y
 Built a protocol-style model of entire system, merely comprising timing
information and handling of relevant operations

 Validated (bounded) cycle accuracy vs. actual HDL using SEC

 Easily reproduced failures on unconstrained protocol model

■ Then verified HW fix: closing one-clock timing window

■  Al
 Also verified SW fix: strategic no-op injection
● Clearly wanted to inject as few as possible for optimality
● Modeled by adding constraints over instruction stream being executed

upon abstract model

● Re-running with constraints yielded proof 
 Ag
 A g en
endda
 Introduction to Formal Verification
 Case Studies from TI: Protocol & Control Logic Verification
 Case Studies from IBM: Formal Processor Verification
■ Complexities of High-End Processors
■  Al
 Algorithmic Methods for Reducing Verification Complexity
■ Processor Verification Case Studies

● Sequential Equivalence Checking (SEC)

● Instruction Dispatch Case Study
● Instruction Fetch-Hang Case Study
● Floating-Point Unit Verification
● Load-Store Verification

 Verification Closure: Coverage Analysis & Integration with Simulation

 FPU
FPU Case
Case Stud
Stu d y
 Floating point number format: M * BE
■ M: Mantissa e.g. 3.14159
■ B: Base, here B=2
■ E: Exponent, represented relative to predefined bias
●  Ac
 Actual exponent value = bias + E

 A normalized FP number has Mantissa of form 1.?????

■  As
 Aside from zero representation

 Fused multiply-add op: A*B + C for floating point numbers A,B,C

■ C referred to as addend
■  A*B referred to as product

 Guard bits rounding modes sticky bits used to control rounding errors
FPU
FPU Case
Case Stud
Stu d y
 Highly-reusable methodology developed for FPU verification

 Checks numerical correctness of FPU datapath

■ Example bugs:
● If two nearly equal numbers subtracted (causing cancellation), the wrong
exponent is returned
● If result is near underflow, the wrong guard-bit is chosen

 Focused upon a single instruction issued in an empty FPU

■ Inter-instruction dependencies independently checked, conservatively
flagged as error 
FPU “ Nume
umeric
rica
al Corr
Correect
ctne
ness”
ss”
 Uses a simple IEEE-compliant reference FPU in HDL
■ Uses high-level HDL constructs: + - loops to count number of zeros
■ Imp: 15000 lines VHD
VHDL; Ref-
ef-FPU
FPU: <700 lines

 Formally compare Ref-FPU vs. real FPU

164 bit
2e_addend 2e_prod
mantissa addend ..00
mantissa product …00
+/- a+b
cnt lea
leadin
ding
g 0’s co
copy
py and
and round
round
Leading zero’s can happen, e.g.,
1.101011
 – 1.10100
1.1010011
S Exp Frac Final IEEE result = 0.000010
FPU Co mp
mpll ex i t y Issues
Iss ues
 Certain portions of FPU intractable for formal methods
■ E.g., align
alignm
ment-
ent-sh
shiifter,
fter, multipli
ultiplier 
er 

 Needed methods to cope with this complexity:

■ Black-box multiplier from cone-of-influence
● Verified independently using “standard” techniques
● Multi
ultipliers
pliers are fairl
fairly
y regu
regullar, in
in contras
contrastt to
to rest of
of FPU

■ Case-splitting
● Restrict operands  each subproblem solved very fast
● Utilize batch runs  subproblems verified in parallel

■  Ap
 Apply automatic model reduction techniques
● Redundancy removal, retiming, phase abstraction…
● These render a combinational problem for each case
 FPU Ca
Cas e-Sp l i t t i n g
 Four distinct cases distinguished in Ref-FPU
■ Based on difference between product, addend exponent
δ   = e prod  − ec where e prod  ( = e a + eb − bias ) is the  product exponent
and  ec is the addend  exponent

 Case splitting strategy via constraining internal Ref-FPU signals

■ Verifi
erifica
catition
on algos implicit
plicitlly propa
propaga
gate
te these
these cons
constr
traints
aints to real
real FPU
FPU
■  Al
 Allows each case to cover large, difficult-to-enumerate set of operands

C δ   := ( e a + eb − bias = ec + δ  )

 Disjunction of cases easily provable as a tautology, ensuring

completeness
FPU Ca
Cas e-Sp l i t t i n g
FPU No r mali
malizzati
tioo n Sh
Sh i f t Ca
Cas e-s pl
plii t s
 Normalization shifter is used to yield a normalresult
■ Depends upon # number of leading zeros of intermediate result

 Define a secondary case-split on normalization shift

■ Constraint defined directly on shift-amount signal (sha) of Ref-FPU
■ S ha is 7-bit signal (double-precision) to cover all possible shift amounts

C sha := ( sha =  X  ) for all 106  possible shift amounts;

C sha / rest  := ( sha > 106 ) to cover the remaining cases (trivially discharged  )
FPU Res u l t s
 Development of methodology required nontrivial trial-and-error to ensure
tractability of each proof 
■  An
 And some tool tuning…

 Resulting methodology is highly portable

■ ~1 week effort to port to new FPUs

 Numerous bugs flushed out by this process

■ In one case, an incorrect result was flushed out after billions of
simulati
simulation
on patt
patterns
erns
 Ag
 A g en
endda
 Introduction to Formal Verification
 Case Studies from TI: Protocol & Control Logic Verification
 Case Studies from IBM: Formal Processor Verification
■ Complexities of High-End Processors
■  Al
 Algorithmic Methods for Reducing Verification Complexity
■ Processor Verification Case Studies

● Sequential Equivalence Checking (SEC)

● Instruction Dispatch Case Study
● Instruction Fetch-Hang Case Study
● Floating-Point Unit Verification
● Load-Store Verification

 Verification Closure: Coverage Analysis & Integration with Simulation

L o ad -Sto
torr e Un i t Cas e Stu
tuddy
 Numerous properties to check of LSU and Memory Infrastructure:
■ Multiprocessor cache coherency properly maintained
■ Correc
rrectne
tness
ss of asso
associati
ciativit
vity
y pol
policy
■ Proper address-data correlation and content maintained
■ Parity and data errors properly reported
■ Data prefet
prefetch
chiing stays within
thin prope
properr page
page lim
limits
■ …

 In this case study we introduce several Testbench modeling tricks that

can be used for such checks
 Cac h e Co
Co h er en c e Ca
Cas e St
St u d y
 Cache coherence protocol requires masters to obtain a clean snoop
response before initiating a write
■ Obtain Exclusive snoop to write, clean snoop to read

■ Otherwise data consistency will break down

 Manda
andatory
tory for
for drive
driverr to adhe
adhere
re to
to protoco
protocoll, else wi
will spurious
spuriouslly break
break logi
logic

 Ad
 Adhering to protocol requires either:
■ Building reference model for each interface, indicating what coherence
state it has for each valid address
● Safe, but dramatically increases Testbench size!

■ Using internal cache state to decide legal responses

● Not safe: if cache is flawed (or has timing windows due to pipelining), driver
may miss bugs or trigger spurious fails
Cac h e Co
Co h er en c e Ca
Cas e St
St u d y
 Trick: check coherence only for one randomly-selected address
■ Reference model becomes very small

 Al
 Allow arbitrary activity to be driven to other addresses
■ Will generate illegal stimuli, but cache should still behave properly for
checked address

 Other tricks:
■ Parameterize RAM! Caches often are VERY large
■ Can limit # addresses that can be written to, but need to take care that
exercise sectoring, N-way associativity, …
 As
 A s s o c i at
atii v i t y Cas
Casee Stu
St u d y
 N-way associative caches may map M>N addresses to N locations
■ When loading N+1 ’th address, need to cast a line out
N+1’th
■ Victim line often chosen using Least-Recently Used (LRU) algo

 Verify: newly-accessed entry not cast out until every other entry accessed

 Randomly choose an entry i to monitor; create a N-1 wide bitvector 

■ When entry i accessed, zero the bitvector 

■ When entry j != i accessed, set bit j

■ If entry i is cast
cast out,
out, chec
check
k that
that bitve
bitvector
ctor is all 1’s

 Weaker
eaker pseudo
pseudo-LR
-LRUU may onl
only guar
guarant
antee:
ee: no cast
castou
outt unt
until J accesses
■ Zero count upon access of entry i

■ Increment count upon access of j != i

■  As
 Assert counter never increments beyond J
 Ad
 A d d r es
esss -Dat
-Dataa Con
Co n s i s t en
enccy
 Many portions of LSU need to nontrivially align data and address
■ Data prefetch, load miss queues: delay between address and data
entering logic
●Many timing windows capable of breaking logic
■ Cache needs to properly assemble sectors of data for writes to memory
■  Ad
 Address translator logic maps virtual to real addresses

 Can either build reference model tracking what should be transmitted

(remembering input stimuli)

 Or – play the trick

trick used
used on Instr Dispatch
spatch exam
example
ple
■ Encode information into data
  Ad
 A d d r es
esss -Dat
-Dataa Con
Co n s i s t en
enccy
 Drive data as a function of addr 
■ Validate that outgoing addr-data pairs adhere to encoded rule

■ Should trap any improper association and staging of data

 Encode atomicity requirements onto data

■ Tag each cache line sector with specific code, validate upon write

■ Tag each word of quad-word stores with specific code, validate that

stores occur atomically and in order 

 Encode a parity bit onto driven data slices

■ Can even randomize odd vs. even parity

■ Should trap any illegal data sampling

 Drive poisoned data values if known that they should not be transmitted
 Pari
rity
ty / Err
rror
or Detecti
tection
on Cor
orrectn
rectne
ess
 Error code schemes are based upon algorithms:
■ Properly diagnose <I bit error code errors, <J data bit errors
■ Properly correct <K bit data errors

 Often use a reference model based upon error code algorithm

■ Build a Testbench for each type of injected error 
● Single-bit data, double-bit data, single-bit error code, …
■ Case-split on re
reac
actition
on type
type
● Compare logic reaction against expected outcome

 Used to find error detection bugs; improve error detection algorithms

■ Quantify % N-bit errors detected using symbolic enumeration
■ Study undetected cases to tighten algorithm
Pre
refe
fetch
tch Corre
orrectne
ctness
ss
 Prefetch
refetch logi
ogic is a perf
perform
orman
ance
ce-enh
-enhan
ancing
cing feat
feature
■ Guess addresses likely to be accessed; pull into cache before needed
■ Often use a dynamic scheme of detecting access sequences:
● Start by fetching one cache line
● If conti
continu
nued
ed access
accesses
es to pref
prefetched
etched strea
stream
m, start
start fetching
fetching multi
ultiple line
lines
s

 Howev
owever,
er, faulty
faulty prefetch
prefetch logi
ogic can
can brea
break
k functi
function
onali
ality
ty
■ Generati
eneration
on of ill
illegal
egal prefetch addres
addresse
ses
s  checkstop
■ May be responsible for address-data propagation

■  An
 And bad prefetching can easily hurt performance
Pre
refe
fetch
tch Corre
orrectne
ctness
ss
 Genera
enerattion of ilillegal
egal prefetch addre
address
sses
es  checkstop
■ Most
ost prefetchi
prefetching is requir
required
ed not
not to cross add
address
ress barri
barriers
ers
● E.g. must be done to same page as actually-accessed addr 
■ Can restri
restrict
ct address
address stream bei
being gene
generat
rated
ed,, or mon
moniitor addr
addr strea
stream
m,
and
and vali
valida
datte that prefetch
prefetch requests
requests stay wi
within same
same pag
page e

 Al
 Also wish to verify that prefetch initiates prefetches when it should, does
not when it shouldn’t
■ Often done using a reference model or set of properties to encode

spe
specific
cific prefetch
prefetching
ing algori
lgorithm
thm
 Ag
 A g en
endda
 Introduction to Formal Verification
 Case Studies from TI: Protocol & Control Logic Verification
 Case Studies from IBM: Formal Processor Verification
 Verification Closure: Coverage Analysis & Integration with Simulation
■ Design Intent Verification

■ Verification as Coverage Analysis

■ Design Intent Coverage and Specification Refinement
■ Reasoning about Specifications
■ Have I Written Enough Properties?
■ Property Directed Simulation Games
■ The Integrated Picture
Desi
sign
gn Int
nteent Veri
rifi
fica
cati
tion
on

D
Design
esi gn IInte
sign nt
nte
nt eent
nt Design
Verification
Specification
Specification
Design
Design
Implementation
Implementation

D
Design
esi gn IInte
sign nt
nte
nt eent
nt
Modeling
Modeling

Design Intent
Verification

This ste
s tep
p is be
becoming
comingg ve
becomin very
ry
important
impor tant in pra
pr actice  – why?
Wha
hatt is
i s de
d esi
sign
gn int
inteent ve
veri
rific
ficaati
tion
on?
?
 Design Intent
■  A set of abstract high-level global safety and liveness requirements
 Design Intent Modeling
■ Developing a set of high-level component models that mimic the
expected behavior of the components
 Design Intent Verification
■ The models taken together should satisfy the design intent
Traditional: With
ithout
out intent
intent verif
verifica
icatio
tionn

Comp
Comp--11 Comp
Comp--11
Specs
Specs Design
Design

Comp
Comp--22 Comp
Comp--22
Desig
Designnn Intent
ent Specs
Spe Specs
Specs Design
Design
Desig
De sign
Design Int
Intent Specs
cs
Specs Integrated
Integrated
(English)
(English) Design
Design
Comp
Comp--kk Comp
Comp--kk
Specs
Specs Design
Design
Sub
Su b - system
Design
Design of 
Tech
Tech.. Specs
the parts
(English)

Design Verification

This ve
v erif
rification
ication ta
t ask is large
l arge a
la nd compl
and co mple
ex
Emerging: Wit
ithh inte
in tent
nt ve
v eri
rifi
fica
cati
tion
on

Comp
Comp--11 Comp
Comp--11
Specs
Specs Design
Design
Desig
Design
Desig
De nnIntent
sign IntentSpecs
Specs
Spe cs Integrated
Integrated
(English)
(English) Design
Design
Comp
Comp--kk Comp
Comp--kk
Specs
Specs Design
Design
Sub - system
Design
Design of 
Tech. Specs
the parts
(English)

Comp
Comp--11 Verif .
Desig
De sign
nn Intent F--Model
Specs
Design
Design
Desig Intent of the
(Formal Specs)
(Formal Specs) parts
parts
Comp-
Comp
Comp-
Comp --kk
F--Model
Specs
Sub -system Integrated
Design
Design
Desig n Intent
sign Design
Models
Verification
- Verification
Can
Can be done
don e before
befor
beforee implementation
im plementation
Int
nteent Veri
rifi
fica
cati
tion
on – Everywhere?
 Digital design verification
■ Micro-architectural properties (SVA / PSL)
■ RTL properties (SVA / PSL), RTL blocks
 Mixed-signal design verification
■ Integrated power mgmt chips (LDOs, buck converters, battery charger)
■ Modeling options include VerilogA, VerilogAMS
 Embedded systems
■ Safety critical properties
■ Modeling options include Matlab, UML statecharts
Desi
sign
gn Int
nteent Veri
rifi
fica
cati
tion
on – Why?
 Practical considerations
■ Components are out-sourced, and integrated into the design at the end
(sometimes as black-boxes)
 Helps in developing the component specs before implementation

 Helps in developing acceptance tests for the components and verifying

the integrated design
Dimensions

Discrete Models

Continuous
Contin uous M
ontinuous odels
Models
Models
Boolean
Boolean Logic
Logi c
Hybrid Models
Hybrid Models System
Temporal
mporal Logi
Temporal
Te c
L ogic
Logic
Type Formal FSM
specs Equations
Satisfiability
Core Hybrid
ybri d Automata
Hybrid
Synthesizability Problems
Formal coverage

Model Checkin
Checkingg
 Ag
 A g en
endda
 Introduction to Formal Verification
 Case Studies from TI: Protocol & Control Logic Verification
 Case Studies from IBM: Formal Processor Verification
 Verification Closure: Coverage Analysis & Integration with Simulation
■ Design Intent Verification

■ Verification as Coverage Analysis

■ Design Intent Coverage and Specification Refinement
■ Reasoning about Specifications
■ Have I Written Enough Properties?
■ Property Directed Simulation Games
■ The Integrated Picture
Veri
rifi
fica
cati
tion
on is
i s all
all about
abou t cove
co vera
rage
ge

Valid Invalid

The role of a specification

specifi cation is to
partit
partition
ion the set
set of possib le behaviors
behaviors
into valid
valid and invalid
i nvalid behaviors
behaviors

The role of ve
v erif
rification
ication is
i s to check
c heck
whether the valid
valid beha
behavio
viorsrs w.
w .r.t the
specification covers all behaviors of  o f 
the implementatio
implementation n

Implementation
Implementation

In othe
oth er words
wo rds,, the set
set of be
b ehaviors not exhibit
xhi bite
ed by the
t he
impleme
imp lementation
ntation covers
implementation co vers
cove rs the inva
inv alid behaviors
invalid behaviors of the
t he specification
specifi cation
specifica tion
Veri
rifi
fica
cati
tion
on as co
cove
vera
rage
ge

Implementation
Implementation

If:
If :
• is a function
funct ion that
t hat captu
captures
res exactly
exactly th
thee set of all
all behavio
behaviors
rs
of the
t he given impl
impleementation, and
and
• is the
t he specification,
specification,

Then
hen the verific
verificaation proble
probl em is to che
ch eck for
fo r the
th e validity
validity of 
 A b u g i s a wi
w i t n es
esss f o r 
 Verification Methodologies
as a witness for R ∧¬ A 
 A bug is a trace that acts as

 Simulation and dynamic ABV: Enumerate traces treating R as an executable

black box and search for ¬ A on each trace

 Model checking: Represent R as a state machine, represent ¬ A as another

automaton and check whether the product of the two machines is e mpty

 Design intent coverage: Represent R as a collection of properties that are

guaranteed by the components, and check the satisfiability of R ∧¬ A 
Com
ompo
posi
siti
tion
onaal Ve
Veri
rifi
fica
cati
tion
on & Cov
Coveera
rage
ge

: Specific
Specificaatio n

Properties

Verification

1 2 k

: Implementatio
Implementationn

Model checking:
checking : is a set set of logic blocks or st ate machines
machines
Simulation: is a set set of
o f execut
executaable black boxes
bo xes
Design
sig n inte
int ent cove
cov erage:
rage: is a set set of loc
locaal properties
pr operties over the
th e is
Two key
key pr
probl
oble
ems

: Specific
Specificaatio n
What should
shou ld be our appro ach whe
wh en
Properties consists
consis ts of state machines
machines for some
compone
compon ents, properties for some
Verification compon
com pone ents and executable
executa
execut able black
boxes for tthe
he rest?
rest?

1 2 k

: Impl ementatio
mentationn

Problem -1: Developin

veloping g a unified
unif ied model for
fo r covera
co verage
ge analysis
analysis
Problem -2: Prop
ropeerty slic
slicing
ing / Specific
Specificaatio
tionn refinement
refinement
Mod
odeel ch
cheeck
ckin
ing
g as co
cove
vera
rage
ge ana
analy
lysi
siss

module GrayCounter(a1, a2, r s t )

rst
a1 input r s t ;
r eg a1, a2;
always
alway s @ (posedge c l k )
begin
a2 a1 <= a2 & ~r s t ;
a2 <= ~a1
~a1 & ~r s t ;
end
en d
clk endmodule

Sample
mpl e property
pro
propperty
propeerty : If the
th e counter is not rese
reset,
t, then the next
next va
v alue of 
the count
counteer di
differs
ffers from
fr om th
thee present va
value
lue by e
exactly
xactly one
on e bit
bit..

G( r s t ( a1 Xa1 ) ( a2 Xa2 ))
Mod
odeel ch
cheeck
ckin
ing
g as co
cove
vera
rage
ge ana
analy
lysi
siss

module GrayCounter(a1, a2, r s t )

rst
a1 input r s t ;
r eg a1, a2;
always
alw ays @ (posedge c l k )
begin
a2 a1 <= a2 & ~r s t ;
a2 <= ~a1
~a1 & ~r s t ;
end
en d
clk endmodule

G( (a2 rst Xa1) ( a1 rst Xa2) )

G( r s t ( a1 Xa1 ) ( a2 Xa2 ))

Model checking on modu modulele GrayCounter  is equivale

qui valent
nt
to checking th e validity
validity of or checking the
satisfiability o f
 Ag
 A g en
endda
 Introduction to Formal Verification
 Case Studies from TI: Protocol & Control Logic Verification
 Case Studies from IBM: Formal Processor Verification
 Verification Closure: Coverage Analysis & Integration with Simulation
■ Design Intent Verification

■ Verification as Coverage Analysis

■ Design Intent Coverage and Specification Refinement
■ Reasoning about Specifications
■ Have I Written Enough Properties?
■ Property Directed Simulation Games
■ The Integrated Picture
Pri
rior
orit
ityy Cach
chee Ac
Acce
cess
ss

The block
The blo ck a
block rbitrates
rbi trates between
arbitrates between
r1 and
and r2 to assert g1 or g2,
g 2,
and between r3 and and r4 to
assert g3 or g4

The lilines
nes d1/d2
d1/d2 indic
ind ica
ate whethe
wheth er the
t he page request
requested
requesteed
ed by
b y the
tth
hee
high
hig h / low pri
priori
ority
ty de
d evic
vicee is available
available in the cache
cache
• If the
th ere is a cache hit then d1 is asserted
asserted within
wit hin two cycles
cyc les
• If the
th ere is a cache
cache miss then d1 is asserted
asserted after fetch
fetching
ing the page
from memory
[Source: A Ro
Roa
admap for
for Fo
Form
rma
al Prop
Property
rty Verifica
rificattion
ion, Sprin
Springe
ger,
r, 2006]
 A r c h i t ec
 Ar ectt u r al Pr
Proo p er
ertt y

M
M11 and
and M 2 have high
M2 hi
higgher 
er 
higher 
her 
priori
pri ority
ty over M3
M3 and
and M4

 A p age
ag e req
r equu est
es t ed b y M1/M2 is
i s eit
ei t h er s erv
er v ed w i t h i n t w o c y c l es
(cach
(cachee hit)
hi t) or served
s erved when th thee page is readyready.. In the latter case , no
page should
sho uld be b e served
served to the
t he low priori
pri ority
ty devices in between.
between.

G[ r1 V r2 ⇒ XX[ d1 V X( ¬d2 U d1 ) ] ]
One possib
poss ible
le arch
archite
itect
cture
ure

G[ r1
r 1 V r2 XX[ d1
XX[
V X( d2 U d1 )]]

What does one do if i f we

we
cannot
ca nnot veverify
rify the property
directly
dire
dir ectl
ctlyy due
d ue to capacity
capa cityy
capacit
limitations?

Write properties
properties on individual
indiv idual blocks that together
together prove t h e
archite
rchit ectural property
Deve
velo
lopin
pingg blo
b lock
ck spe
specs
cs

Target :
 A1:
 A 1: G[ r1
r 1 V r2 XX[[ d
XX d11
V X( d2 d2 U d1 )]]

 Ar
 A r b i t er:
er :
R1: G[ r1 V r2 Xa ]
R2: G[ z3 V z4 Xb ]

Cache Block
Bl ock:: Mask:
R3: G[ a X[
X[ w U d1 ]] R5: G[
G[ r3 w z3 ]
R4: G[ b Xd 2]
d2 R6: G[ r4 w zz4
4]
Desi
sign
gn Int
nteent Cov
oveerage

 Ar
 A r c h . Spec
Sp ecss:
 A1:
 A 1: G[ r1 r 1 V rr2
2 XX[[ d
XX d11
V X( d2 d2 U d1 )]]

RTL
RTL Specs:
Specs
Spe cs::
R1: G[ r1 V r2 Xa ]
R2: G[ z3 V z4 Xb ]
R3: G[ a X[
X[ w U d1 ]]
R4: G[ b X d2 ]
R5: G[ r3 w z3 ]
Is it pos
possib
sible
le to have an impleme
im plementation
ntation R6: G[ r4 w z4 ]
that
th at satisfi
satis fies
es R1,
R
R11,, …,
…, R6
R6,
R6,, but
bu t refut
refutes
es A1?
r efutes A 1?

… if not,
n ot, then all
all invalid
in valid behavio
behaviors
rs are not covered
c overed by the
RTL
RTL specs,
sp ecs, and we needneed to add more
mo re RTL
RTL properti
pro perties
es
 In t h i s c as e, we have a g ap ! !

 A 1: G[ r1
 A1: r 1 V r2 XX[ d1
d 1 V X(
X( d2 U d1 ) ]]
Gap: G[ r1
r 1 V r2
r2 XX[ ( d 1 Xd1) V d1 V X( d2 U d1 ) ]]
[Source: A Ro
Roa
admap for
for Fo
Form
rma
al Prop
Property
rty Verifica
rificattion
ion, Sprin
Springe
ger,
r, 2006]
The Cor
orre
rect
ct Arc
A rchi
hite
tect
ctur
ure
e

Target :
 A1:
 A 1: G[ r1
r 1 V r2 XX[
XX[ d1
d1
V X( d 2 U d1 )]]
d2

 Ar
 A r b i t er:
er :
R1: G[ r1 V r2 Xa ]
R2: G[ r3 V r4 Xb ]

Cache Block
Bl ock:: Mask:
R3: G[ a X[
X[ w U d1 ]] R5: G[ b w z]
R4: G[ z X d2 ]
… th
this
is ti
time
me R1,…
R1,…,R
,R5
5 covers
cov ers A1
A1
Int
nteent Cov
oveerage Pro
Probl
ble
em: Formally…

 Given:

■  Architectural specification A , consisting of a set of temporal

properties over a set  AP  A  of Boolean signals

■ RTL specification R , consisting of a set of temporal properties

over a set AP R  of Boolean signals, such that
⊆ AP R 
 AP  A  ⊆

[Primary Coverage Goal] Does R cover  A  ?

[Specification Refinement] Refinement of  A 

 A to derive a property that captures
the behaviors not covered by R .
Specif
pecific
ica
ati
tion
on R
Reefi
fineme
nement
nt

 Consider the coverage of  A  by R :

 A : G( (¬r 2 ) ⇒ X g1)

R : G( (r 1  ∧ ¬ r 2) ⇒ X g1 )

∧  ¬

  A ∨ ¬
¬R  can be represented as:

(a) G( r 1 ∨ r 2 ∨ X g1 )

(b) G( ( ¬X g1 ∧ ¬r 1 ) ⇒ r 2 )
∧  ¬

(c) G( (¬ r 1 ∧ ¬r 2 ) ⇒ X g1 )

∧  ¬

(c) is visually closest to  A  –

 – our goal is to show the coverage gap as (c).
Sp ec . Ref i n em ent by Relaxati
laxatio on
l ax at io
ion

 Arch. Spec: G((r 2 ∧ z) ⇒ X(( g2 ∧ ¬g1) U ¬r 2))

RTL Specs: G((r 1 ∧ ¬r 2) ⇔ X g1)
GF(¬r 2)

The RTL specs do not cover the architectural specs, but we

can decompose the architectural specs as:

 Ax: G((r 2 ∧ z) ⇒ X(g2 U ¬r 2))

 Ay: G((r 2 ∧ z) ⇒ X(¬g1 U ¬r 2))

■  Ay is covered by the RTL specs

■  Ax represents the coverage gap more accurately
Th e Sp ec Mat c h er Tool
Too l

■ Compares temporal specifications (in LTL)

■ Reports the coverage gap in terms of structure
preserving properties.
■ Two key algorithms:
● Push_Terms: Computes the bounded temporal terms in
the gap, A ∨ ¬ ¬R , and pushes these terms into the syntactic
structure of the architectural properties.
● Relax_ArchSpec: Systematic weakening of architectural
properties having unbounded temporal operators (such as
G, F, and U), and isolating the coverage gap.

[Source: A Ro
Roa
admap for
for Fo
Form
rma
al Prop
Property
rty Verifica
rificattion
ion, Sprin
Springe
ger,
r, 2006]
Th e In t eg r ated
t ed Flow
Flo
Fl ow
   t System-level
  n
  e    t  Ar
 A r c h Spec
Sp ecs
s test pla
pl an
  m   n
  e   e Intent
  n
   i   m
   f   e Coverage
  e
  r   n
   i
   f
  n   e
  g   r Block-level
   i  .
  s   c test plans
  e   e
   D   p Intent
   S
Coverage    V
   +   B
  n   A
  o   c
   t    i
   i
  a   m
   l
  u   a
  n
   i   y
  m
Unit    S   D
ni t leve
levell specs

Unit level Validatio

Validationn
(Sim
(Simululatio
ationn + FPV
FPV))
In t ent Cov
nte
nt oveerrage
ag e an
andd Mo
nd Mo d el Ch
ode
od Ch ec ki
kin
hecking
hecki n
ngg

: Specif
Specif ication  Ad
 A d v ant
an t ages
ag es :
• Scalable
Properties • Can
an be don
donee top down,
do
dowwn,
n, before
before
befor e impleme
impl ementatio
ntation
implementatio n

Verification Disadvantages :
• Requires more
mor e user
user invol
iinvo
nvolveme
vement
nt
lvement
lveme
1 2 k

: Properties for M 1 … Mk
: Specific
Specificaation
tio n
Pure Design
Design Intent
Int ent Coverage
ntent
Properties

Verification
 Ad
 A d v ant
an t ages
ag es :
• Less user invol veme
vement
nt 1 2 k

Disadvantages :
: State
State m/c for M1 … Mk
• Capacity limit
limitaation
tionss
Pur
uree Mod
Model
el Checki
Checking
ng
Betw een Int
Betwe nteent Cov
oveerage and
and Mod
odeel Check
heckin
ing
hecking
hecki g
ng

: Specif
Specification
ication
Properties  Ap
 A p p r o ach
ac h -1:
•Find th
thee gap, T,
T, between
between anand
d
Verification
the properties
properties of Mi +1 … Mk
1 2 k •Model che
ch eck T on M1 … Mi

: State
State m/c for M1 … Mi and
an d
properties for M i +1 … Mk  Ap
 A p p r o ach
ac h -2:
•Tra
Trans
nslate
late M1 … Mi into th e prop
propeerty
Hybr
ybrid
id Desig
signn Intent Coverage
domain
• Use pure desig
designn intent
in tent cove
cov erage

[Source: Das
Das et. al. What
What lies
l ies between Desig
Designn Intent Coverage and
and
Model Checkin
Check ing?
g? DATE
DA TE 2006
2006]]
 Ag
 A g en
endda
 Introduction to Formal Verification
 Case Studies from TI: Protocol & Control Logic Verification
 Case Studies from IBM: Formal Processor Verification
 Verification Closure: Coverage Analysis & Integration with Simulation
■ Design Intent Verification

■ Verification as Coverage Analysis

■ Design Intent Coverage and Specification Refinement
■ Reasoning about Specifications
■ Have I Written Enough Properties?
■ Property Directed Simulation Games
■ The Integrated Picture
Con
onsi
sist
steenc
ncyy and Com
ompl
ple
ete
tene
ness
ss
 Verification: R ⇒  A

 Consistency issues:
■ Verification is vacuous when R is unsatisfiable
■ Verification is vacuous when A is valid

 Completeness issues:
■ Verification closure depends on the completeness of  A  –
 – have I
written enough properties?
Form
ormaal Cons
Consis
iste
tenc
ncy
y Ana
A naly
lysis
sis – Why?
 A
 Arre my properties correct?
■ Debugging formal specifications can be quite hard
■ Codi
oding errors
errors – new
new langu
anguag
ages,
es, alien
alien seman
semanttics
■ Logical errors
 A
 Am
m I checking the right property on the right design?
■  A typical BUS protocol consists of:
● Properties over individual components: master, slave, arbiter 
● Global properties

■ In the absence of proper assume constraints, checking global properties

on indi
ndividual
vidual com
compone
ponents
nts can
can lead to real
realizabil
zabiliity probl
problems.
ems.
 Reasoning about formal specifications
■ Logical implication
■ Coverage analysis
Veri
rific
fica
ati
tion
on is
i s Logic
Lo gica
al Cons
Consis
iste
tenc
ncy
y
 Verification is mostly about checking logical implication

 Model checking:
■ Does
oes the
the produc
productt of the
the com
compo
pone
nent
nt state
state mach
machiines
nes log
logiically
cally im
imply each
each
of the formal properties?
 Design intent coverage:
■ Do the properties of the component modules together imply the
architectural properties of the design?
 Logical implication ≡ Satis
tisfia
fiability
ility / Valid
lidity
ity

Satisfiabi
Satisfiabilit
lityy and Realiz
Realizabi
abilit
lityy for
forms
ms the basis
basis of reasonin
reasoningg about
specifications
Unsatisfiable Specification
 When the master is not in the IDLE or WAIT states, the request line, req, should be
kept high
 The master lowers the request line, req, sometime

`define
defi ne IDLE
IDLE 3’b000
`define
`defi ne WAIT 3’b
3’b00
000
0

property ReqHighDuringTransfer ;
@ (posedge c l k )
(state
(st atee != ‘IDLE || state
(stat stat e != ‘WAIT) |-> r eq ;
endproperty

property ReqIsSometimesLow ;
@ (posedge c l k )
##[0:
##[0:$]
$] ! r eq ;
endproperty
The Cor
orrect
rect Specif
Specific
ica
ati
tion
on
 When the master is not in the IDLE or WAIT states, the rrequest
equest line, req, should be
kept high
 The master lowers the request line, req, sometime

`define
`defi ne IDLE 3’b
3’b00
000
0
`define
`defi ne WAIT 3’b
3’b00
000
0

property ReqHighDuringTransfer ;
@ (posedge c l k )
(state
(stat e != ‘IDLE
‘IDLE & & state
st ate != ‘WAIT) |-> r eq ;
endproperty

property ReqIsSometimesLow ;
@ (posedge c l k )
##[0:
##[0:$]
$] ! r eq ;
endproperty
Vacuity

 When the gnt signal is high, the master should not be in the IDLE or WAIT
states

property UseBusWhenGranted;
@ (posedge c l k )
g n t |-> (st
(state
ate != ‘IDLE || state
stat e != ‘WAIT) ;
st ate
endproperty

 Same mistake – this time the property will always be true

 This gives us a false sense of security !!
Realizability o
off Open
Ope
Openn Syst
Sys ttem
Systemem Specs
Spe cs
Specs

 Whenever the high-priority request, hreq, arrives, the grant, hgnt, is given for
one cycle

property HighPriorityGrant ;
hreq hgnt
@ (posedge c l k )  Ar
 A r b i t er 
hreq |-> ##1 hgnt ##1 ! hgnt ;
endproperty

 The property cannot be satisfied if we have hreq in consecutive cycles

 The property is satisfiable -- consider all traces where hreq is not asserted in
consecutive cycles
 The property is unrealizable because hreq is an input signal
■ It will become realizable under the assumption that hreq never arrives in
consecutive cycles
[Ref: Pnueli, Rosner, ACM POPL,
POPL , 1989]
Realizability = Satisfiability Inputs?
 Not quite. Consider the following properties:
■ Each request is eventually granted
■ The request line is lowered one cycle after the grant

property Gnt
Gn t ;
@ (posedge c l k )
r eq |-> ##[1:
##[1:$]
$] g n t ;
##[1:$] r eq gnt
 Ar
 A r b i t er 
endproperty
property LowReqAfterGnt ;
@ (posedge c l k )
g n t |-> ##1 ! r eq ;
endproperty

 The second property can be satisfied if we know the future inputs !!

[Ref: Pnueli, Rosner, ACM
 A CM POPL, 1989]
Receptiveness

 Consider the property:

 A request is always lowered

lowered after the grant is asserted.

property LowReqAfterGnt ;
r eq gnt
@ (posedge c l k )  Ar
 A r b i t er 
g n t |-> ##1 ! r eq ;
endproperty

 The arbiter can realize this property by never asserting g1.

■ This is an un -receptive property – why?

  A module must have the freedom of choosing

choosing its outputs as long as it does
not refute the property
[Ref: Dill, MIT Press,
Press , 1989]
1989]
 Ag
 A g en
endda
 Introduction to Formal Verification
 Case Studies from TI: Protocol & Control Logic Verification
 Case Studies from IBM: Formal Processor Verification
 Verification Closure: Coverage Analysis & Integration with Simulation
■ Design Intent Verification

■ Verification as Coverage Analysis

■ Design Intent Coverage and Specification Refinement
■ Reasoning about Specifications
■ Have I Written Enough Properties?
■ Property Directed Simulation Games
■ The Integrated Picture
Have I wr
writ
itte
tenn enough
enou gh pr
prop
ope
ert
rtie
ies?
s?

 Completeness can be assured against some functional coverage goa l

 Paradox:
■ If I had a formal definition of the coverage goal, then that its elf could become the
formal specification!!

 Solution:
■ Evaluate the formal property specification with some structural coverage metric
■ The structural coverage metric should be such that:
● Low coverage indicates gaps in the specification, and I need mor e

properties
● High coverage does not necessarily mean that I have enough properties
Typ
ypees of
o f Coverage
Coverage Appro
App roa
ach
chees
 Mutation-based Approaches
■  A given implementation is used as the reference
 Fault-based Approaches
■  A given fault model is used as the reference
 Design Intent Coverage
■  A higher level specification is used as the reference
Mutation -b ased Cove
Cov errage
age

Specification :
00 State: (g1,
(g 1,g2)
g2)
(g1,g2) P1: g1 is never asserted in consecutive cycles
property NoConsecutiveG1;
NoConsecutiveG1;
NoConsecutive G1;
@ (posedge clk) g1 |-> ##1 !g1 ;
10 01
endproperty
 Ab
 A b s t r act
ac t FSM of
of a P2: g2 is never asserted in consecutive cycles
Round -robin Arbiter 

 Ap
 A p p r o ach
ac h es :
Falsity coverage: Mutate the FS
FSM and
and check whethe
w hetherr the truth
tr uth of
any property
pr operty cha
ch anges.
Vacuity coverage: Mut
utate
ate th
thee FS
FSM
M and check
ch eck whether
wh ether any
any pro
p roperty
perty
becomes vacuous.

[Ref: Hos
Hoskote
kote et.al.
t.al. DAC 1999, Chockler
Chockler et.al.
t.al. CAV 2001]
Mutation -base
basedd Fals
lsit
ity
y Cov
oveerage

s0
00 State: (g1,
(g 1,g2)
g2)
(g1,g2) Specification :
P1: g1 is never asserted in consecutive cycles
s1 s2 P2: g2 is never asserted in consecutive cycles
10 01

 Ab
 A b s t r act
ac t FSM of
of a
Round -robin Arbiter 

s0 s0
10 P1 fail
fails.
s. 00
The value of g1
g 1 at
at s0
s 0 is covere
cov ered
d
co vered
covered
s1 s2 s1 s2
10 01 P2 fail
fails.
s. 11 01
The value of g2
g 2 at
at s1
s 1 is covere
cov ered
d
co vered
covered
Mutant -1 Mutant -2
(g1 fli
flipped
pped at s0) (g2 fli
flipped
pped at s1)
Mutation -base
basedd Vacu
cuit
ity
y Cov
oveerage

Specification :
s0
00 State: (g1,
(g 1,g2)
g2)
(g1,g2) P1: g1 is never
never asserted
asserted in conse
cons ecutive
cut ive
ve cycle
co nsecutive
consecuti cycl es
cycles
property NoConsecutiveG1;
NoConsecutiveG
NoConsecutiveG1;1;
s1 s2
@ (posedge clk) g1 |-> ##1 !g1 ;
10 01 endproperty
 Ab
 A b s t r act
ac t FSM of
of a P2: g2 is never
never asserted
asserted in conse
cons ecutive
cut ive
ve cycle
co nsecutive
consecuti cycl es
cycles
Round -robin Arbiter 

Falsi
Falsity
ty coverage
s0 No property
pr
prooperty
perty fails.
00 The value of g1g 1 at s1 is no
nott covered
ccov
overed
covere
ered
d
s1 s2
00 01 Vacuity coverage
P1
P1 is satisf ied vacuo
va cuously
vacuou usly.
sly..
cuously.
Mutant -3 The value of g1 g 1 at
at s1
s 1 is covere
cov ered
d
co vered
covered
(g1 fli
flipped
pped at s1)
What doe
do es mut
m uta
ati
tion
on co
covera
verage
ge mea
mean?
 If mutations on many parts of the implementation do not affect the truth
of the
the prope
propert
rtiies,
es, the
then
n it is like
likelly that
that our
our spec
speciifica
ficatition
on does
does not
not cover
cover
those behaviors that are exhibited by those parts of the implementation
■ Extensions to transition coverage, path coverage, etc

■ Extensions to simulation coverage metrics, such as code coverage

(mutations on the HDL code) and circuit coverage (toggle coverage on

latches and signals)
 Does this mean that 100% coverage ⇒ we have written enough
properties?
Hig
ighh Cov
Coveera
rage
ge is no
nott goo
g ood
goodd eno
enough
nough

Gray Counter: Correct!! Gray Counter: Incorrect!!

module GrayCounter ( x1, x2 ) module GrayCounter ( x1, x2 )

r eg x1, x2; r eg x1, x2;
always
alw ays @ (posedge c l k ) always
alway s @ (posedge c l k )
begin begin
x1 <= x2; x1 <= x1;
x2 <= ~x1; x2 <= ~x2;
end
en d end
en d
endmodule endmodule

00 01 00 01

10 11 10 11

Property P: The next

next value
v alue of th e cou
t he
the counter
nter
count er dif
difffers
ers from
differs fr om the
th e present
t he present
value in exactly
exactly one
o
onne e bit (100
(100%% state coverage,
co verage,
coverag e, but
bu t …)
Faul
ultt -b ased Co
Co verage A naly
nalyssis
 Core Idea: Inject a fault into the specification and test whether the
spe
specification
cification rem
remains sati
satisfiab
sfiablle / reali
realiza
zab
ble
 Fault Model
■ Stuck-at faults on one or more signals
■ Possible counter-example scenarios
 Good for a first-cut check on the first formal specification
■ Low coverage means we need more properties
■ High coverage does not necessarily mean we have sufficient properties

[Ref: A Ro
Roa
admap for
for Fo
Form
rma
al Prop
Property
rty Verif
Verifica
icattion
ion, Sprin
Springe
ger,
r, 2006]
St uc
uckk -at Faul
Faultt Cove
Cov erage
 Output Fault Coverage:
■  A stuck-at fault on a non-input y is covered if there exists some scenario
where the specification forces y to take the opposite value
 Input Fault Coverage:
■  A stuck-at fault on a input x is covered if we cannot realize the
specification without reading that input x

[Ref: Das
Das, et.al.,
t.al., VLSI 2005]
Exampl
xamplee: Out
utpu
putt Faul
ultt Cov
oveerage
Memory arbiter: mem-arbiter(input r 1, r 2 ; output g1, g2)

r1 g1
Priority of g1: G ((r1 ∧ r2)  X g1)
 Ar
 A r b i t er  g2
r2
Mutex: G (¬g1 ∨  ¬
¬g2)

 g1 s-a-0 is directly covered (input: r1=r2=1)

 g2 s-a-1 is indirectly covered – it implies g1 s-a-0, which is directly covered

 g2 s-a-0 is not covered – we can have a valid implementation that never asserts g2 !!

■ add a property for the scenario where g2 has to be high

G((¬r1 ∧ r2)  X g2)

■ now, g2 s-a-0 is directly covered and g1 s -a-1 indirectly covered

Exampl
xamplee: Inp
nput
ut Faul
ultt Cov
oveerage
Memory arbiter: mem-arbiter(input r 1, r 2 ; output g1, g2)

r1 g1
Priority of g1: G ((r1 ∧ r2)  X g1)
No starvation: G ((¬r1 ∧ r2)  X g2)  Ar
 A r b i t er  g2
r2
Mutex: G (¬g1 ∨  ¬
¬g2)

 r1 s-a-0 and s-a-1 are covered

 r2 s-a-0 is covered

 r2 s-a-1 is not covered – we can realize the specs without reading r2

(always assume that it is high)
■ Modified specs (substituting r2=1) covers original specs:
G (r1  X g1), G (¬r1  X g2), G (¬g1 ∨  ¬ ¬g2)
■  Add a property for some scenario where
whe
wherre
e r2 is low
G (¬r2  X (¬g2))
■ Substituting r2=1 fails to cover the above property. Hence r2 s a 1 is covered.
 Ag
 A g en
endda
 Introduction to Formal Verification
 Case Studies from TI: Protocol & Control Logic Verification
 Case Studies from IBM: Formal Processor Verification
 Verification Closure: Coverage Analysis & Integration with Simulation
■ Design Intent Verification

■ Verification as Coverage Analysis

■ Design Intent Coverage and Specification Refinement
■ Reasoning about Specifications
■ Have I Written Enough Properties?
■ Property Directed Simulation Games
■ The Integrated Picture
Th e In t eg r at ed F l o w (Rec ap )
Flo
Flow

   t System-level
  n
  e    t  Ar
 A r c h Spec
Sp ecs
s test pla
pl an
  m   n
  e   e Intent
  n
   i   m
   f   e Coverage
  e
  r   n
   i
   f
  n   e
  g   r Block-level
   i  .
  s   c test plans
  e   e
   D   p Intent
   S
Coverage    V
   +   B
  n   A
  o   c
   t    i
   i
  a   m
   l
  u   a
  n
   i   y
  m
Uni
nitt level specs    S   D

Unit level Va
Valilidatio
dationn
(Sim
(Simululatio
ationn + FPV
FPV))
Dyn
ynaami
micc Pro
roperty
perty Veri
rifi
ficatio
cationn (DPV)

[Source: A Ro
Roa
admap for Fo
Form
rma
al Prop
Property
rty Verifica
rificattion
ion, Sprin
Springe
ger,
r, 2006]
The not
notion
ion of con
conte
text
xt
 Two types of properties:
■ Inv
nvari
arian
ants
ts – prop
properti
erties
es that mus
mustt hold alway
always
s
● Example: An
 An arbiter must never assert two grants at the same time (mutex)
■ Context sensitive properties
● Example: In a burst mode transfer, addresses wrap around the 2KB
address boundaries

 To verify the second property we must reach a burst transfer 

■ In all other scenarios, the property is vacuous
 As
 A s s er
ertt i o n Co
Covv er
erag
age
e
 As
 Assertion coverage attempts to determine whether the simulation has
covered scenarios where the assertion was checked non-vacuously
■ How do we check the vacuity?

■ Many definitions:

● Implication vacuity
● Expli
xplici
citt specifi
specifica
cattion of coverage
coverage go
goals
als – suc
such
h as
as cover
cover properti
properties
es in SV
SVA
● Gaming definitions
Imp
mplic
lica
mpli ati
tion
cati on va
vacui
cuity
ty
 Property: If the request r is asserted, then the grant g must be asserted
in the next two cycles, unless r is lowered in between.

property
pro perty P;
P;
@ (posedge c l k )
r |-> ##1
##1 (g or (!g &&
& & !r)
! r) or ((!g
((!g && r) ##1
##1 g) ;
endproperty

 Whenever r is asserted, implication vacuity will report non -vacuous

interpretation

 If r is asserted and the DUT does not assert g in the next cycle, then we
should drive r again to check the remaining part of the property
■ Otherwise, the real intent of the property is not checked
Property -dr
driv
ive
en test gene
genera
rati
tion
on?
?
 It’s a game …

Player -1: The Test Bench (TB)

■ Drives the input signals in each round

Player -2: The Design Under Test (DUT)

■ Produces the output signals in each round

 We play for the test bench …

 Two types of games – same players, but the winning conditions are different
■ Vacuity games

■ Realizability games

[Ref: Baner
nerjee, et.al.
et.al.,, DAC 2006]
Vac u i t y Gam
Gam es

 Winning condition defined in terms of a formal property, P

 In any round of the game, the inputs written by Player -1 (TB) is vacuous iff P
is satisfied under these inputs regardless of the values of the outputs in that
round
■ Player -1 (T
(Test
est bench
b ench))
● It loses if in any round it writes vacuous
vacuous inputs
iin
nputs
puts
● It wins if
i f in any round it writes non -vacuous input
in puts
s and
inputs and yet the
property
pro perty is satisfied or refuted

■ Player -2 (DUT) loses

lo ses when
wh en Player 
Player -1 wins
wi ns,, and wins
win s when
wh en Player 
Player -1
loses

 The values written in a round re-defines the property for the next round

[Ref: Baner
nerjee, et.al.
t.al.,, DAC 2006]
Realizability Games
 Problem with unreceptive specifications
■ In some round of the game, the property for the next round may become
unrealizable, but satisfiable

■ This means that Player -1 (TB) has a winning strategy from that round
r ound – if it uses
that strategy then the property for some future round will become unsatisfiable

■ This strategy is to be demonstrated by a realizability game

 Winning conditions (P is unrealizable)

■ Player -1 (Test bench)
● Wins in a round if the property is refuted (becomes unsat)
● Loses if the property for the next round becomes realizable

■ Player -2 (DUT) loses when Player -1 wins, and wins otherwise

[Ref: Baner
nerjee, et.al.
t.al.,, DAC 2006]
Example – t i c -t ac -t o e

X ? X X : Environment
Enviro nment
0 : DUT

0 X

? 0

• Open
n Game
Open
Ope Game at
at start
s tart:: DUT
DUT has a strate
str ategy
gy to wi
winn
 Specific
pecificaatio
tionn is rea
realiza
lizable
ble
• DUT make
UT makess a mistake
mis take:: Occupi
Occu pie
es row
r ow 2, column
colu mn 1
co lumn
 Specific
pecificaatio
tionn is
i s unrea
u nrealiza
lizable
ble
• Int
nteellllig
igeent Test -bench
bench:: for
f orces
ces DUT
DUT to de
d efea
featt
defeat
defea
 Cruc
rucial
ial Move:
Move: Occu
Occupyi
pying
ng centra
centr al squa
squ are
Int
nteegr
graati
tion
on int
i nto
o Te
Test Env
Envir
iron
onme
ment
nt

 As
 A ssssert
 As
 A er
ertttiioonn Mon
er Mo
Mon
Mo nniittoor r 
++
Game
Ga me
me Oracle
Game
Ga Oracl
Oraclee
Oracle

Test
Test
DUT
DUT Bench
Bench
 Ag
 A g en
endda
 Introduction to Formal Verification
 Case Studies from TI: Protocol & Control Logic Verification
 Case Studies from IBM: Formal Processor Verification
 Verification Closure: Coverage Analysis & Integration with Simulation
■ Design Intent Verification

■ Verification as Coverage Analysis

■ Design Intent Coverage and Specification Refinement
■ Reasoning about Specifications
■ Have I Written Enough Properties?
■ Property Directed Simulation Games
■ The Integrated Picture
The Desi
sign
gn Int
nteent Veri
rifi
ficati
cation
on Flo
loww

Design
Desi gn Intent
(Formal Properties) Test
Generator  Integrated
+ Design
Uncovered
Uncovered behavior
behaviorss
 As
 A s s er
ertt i o n
Monitor 
Coverage
 An
 A n al
alyy zer 
Simulation
Simul ation + DP
DPV Platfo
Platform
rm

Sub
ub-syst
-system
em Specs
Specs Sub-system
(Formal) Modules

Sub-syst
ub-systeem Acc
Acce eptance Testi
Testing
ng
(simulatio
(simu lation
n + FPV
FPV)
References
1. Banerjee
Banerj ee,, A., B.Pal,
B.Pal, S.Das,
S.Das, A.Kumar
A.Ku mar,, P.Da
P.Dasg
sgup
upta,
ta, Test
Test Generatio
Generationn Games
Games from
fr om
orm al Specificatio ns, DAC 2006.
Forma 2006.

2. Basu,
Basu , P.,
P., S.Da
S.Das,
s, A.Banerj
A .Banerjee
ee,, P.Da
P.Dasg
sgup
upta,
ta, P.P.Chakrabart
P.P.Chakrabart i, C.R.Mohan
C.R.Mohan,, L.Fix,
L.Fix ,
R.Armoni, Design
sig n Intent Coverage
Coverage – A New
New Paradigm
Paradigm f or Formal Property
Verification. IEEE
IEEE Trans.
Trans . on CAD, Oct 2006.

3. Chockler, H., O.Kupferman, R.P.Kurshan, M.Y.Vardi, A p r act

ac t i c al app
ap p r o ach
ac h t o
coverage in mo del checking . CAV 2001.
2001.

4. Chockler, H., O.Kupferman, M.Y.Vardi, Coverage

overage metrics
metric s for temporal
t emporal logi c model
checking. TACAS
TACA S 2001,
2001, LNCS
L NCS 2031.
2031.

5. Chockler, H., O.Kupferman, M.Y.Vardi, Coverage

overage Metrics
Metrics for Forma
orm al Verif
Verif ication.
icatio n.
LNCS 2860, 2003.

6. Das,
Das, S., P.Basu,
P.Basu, A.Banerj
A .Banerjee
ee,, P.Da
P.Dasg
sgup
upta,
ta, P.P.
P.P.Chakr
Chakrabart
abarti,
i, C.R.Mohan
C.R.Mohan,, L.Fix,
L.Fix ,
R.Armoni, Formal Veri
Verific
fication
ation Coverage:
overage: Comput ing the covera
co verage
ge gap between
between
temporal specifications, ICCAD 2004.
temporal

7. Das,
Das, S., A.Banerjee
A.Banerj ee,, P.Basu
P.Basu,, P.Da
P.Dasg
sgup
upta,
ta, P.P.
P.P.Chakr
Chakrabart
abarti,
i, C.R.Mohan
C.R.Mohan,, L.Fix,
L.Fix ,
Formal method s for
fo r analyzing
analyzing the
t he completeness
completeness of an asserti
assertion
on su ite against
against a
References
8. Das,
Das, S., P.Basu
P.Basu,, P.Da
P.Dasg
sgup
upta,
ta, P.P.C
P.P.Chakr
hakrabart
abarti,
i, What lies between
between design intent
int ent
coverage and model checkin g? DATE 2006.
2006.
9. Dasg
Dasgup
upta,
ta, P., A Road
Ro adm
m ap f o r For
Fo r m al Pro
Pr o p ert
er t y Veri at i o n , Springer 2006.
Ver i f i c ati
10. Dill, D.L., Trace Theory
Theory forf or Automatic
Aut omatic Hierarch ierarchical
ical Verif
Verificatio
icatio n of Speed-
Speed-
independent ircui ts, ACM
independent c ircuits,  A CM Dis
Di s t i n g u i s h ed Dis
Di s s ert
er t ati
at i o n s , MIT Pres
Pr esss , 1989.
11. Hoskote, Y., T.Kam, P.H.Ho, X.Zao, Coverage estimation fo r symboli
symb olic
c model
checking, DAC 1999.
1999.
12. Pnueli, A., R.Rosner, On the
th e synthesis
syn thesis of reactive modul e, ACM
o f a reactive  A CM Sym
Sy m p o s i u m o n
Princip
rin ciples
les of Programmin
rog ramming
g Language
L anguages,
s, 198
1989 9.