IP/SOC Verification

IP Verification
Tian-Sheuan Chang
 Outline
• Verification challenges
• Verification process
• Verification tools
• RTL logic simulation
IP Verification

• RTL formal verification

• Verifiable RTL – good stuff
• Verifiable RTL – bad stuff
Tian-Sheuan Chang

• Testbench design
• SOC verification

1 Copyright ©2003 All rights reserved

 Verification Challenges (1)
• Verification goals is 100% correct
– Mission impossible
• Macro-level testbenches and test suite must be
reusable
– For next redesigned macro
IP Verification

– For integration team

• Verified in standalone as well as in final applications
• Testbench must be compatible with the system level
Tian-Sheuan Chang

verification tools

2 Copyright ©2003 All rights reserved

 Verification Challenge (2)
• Design Productivity has risen tenfold since 1990
– Gain by synthesis tools contributed to this challenge
• Only able to verify approximately 100 gates/day
Hardware Design Productivity

160
IP Verification

140
120
Gates Per Day

100
80
Tian-Sheuan Chang

60
40
20
0
80

82

84

86

88

90

92

94

96

98
19

19

19

19

19

19

19

19

19

19
Year

3 Copyright ©2003 All rights reserved

 Verification Challenge (3)

IC/ASIC Designs Having One or More Re-spins by Type of Flaw

Logic or Functional 67%
Analog Circuit 35%
Noise 29%
Slow Path 28%
Clocking 25%
IP Verification

Yield 23%
Mixed-Signal Interface 21%
IR Drops 20%
Race Condition 17%
Tian-Sheuan Chang

Power 17%
Firmware 13%
Other 4%

0% 10% 20% 30% 40% 50% 60% 70%

Source: Collett International Research (Apr02)

4 Copyright ©2003 All rights reserved

 Re-spins are EXPENSIVE
Costs $10.7M
($K) 10,000
Proto & Validation
8,000 Support SW

Physical Design
6,000
Verification
IP Verification

$4.7M
4,000 Design Planning

IP Dev and Qual

2,000
Tian-Sheuan Chang

Spec

0
Original Re-spin

Plus a) lost revenue, b) opportunity costs

Source: International Business Strategies, 2002
5 Copyright ©2003 All rights reserved
 Verification and Design Reuse
• Reuse is about trust
• The key to design reuse is gaining that trust
• Verification for Reuse
– Complete functional verification
– All possible configurations
IP Verification

– All possible uses

Tian-Sheuan Chang

6 Copyright ©2003 All rights reserved

 Outline
• Verification challenges
• Verification process
• Verification tools
• RTL logic simulation
IP Verification

• RTL formal verification

• Verifiable RTL – good stuff
• Verifiable RTL – bad stuff
Tian-Sheuan Chang

• Testbench design
• SOC verification

7 Copyright ©2003 All rights reserved

 Boosting Productivity throughout
the Verification Flow
Functional
Specification

RTL Code Assertions

Test Plan
Lint Checking
IP Verification

Simulation
Test Bench with Assertion
Checking
Tian-Sheuan Chang

Functional Coverage

Code Coverage

Verified RTL Code

8 Copyright ©2003 All rights reserved

 Verification Plan
• Part of early design cycle
• Verification takes over 70% of development time
• Contents
– Test strategy for subblock and top level
– Simulation environment including a block diagram
IP Verification

– Test bench components – BFM, bus monitors

– Required verification tools
– List of specific tests for the key features
Tian-Sheuan Chang

– Target code coverage

– Regression test environment and regression procedure
– Criteria when the verification process is completed

9 Copyright ©2003 All rights reserved

 Role of Verification Plan
• Specifying the specification
• Defining First-Time Success
– Ensures all essential features are appropriately
verified
– Which features must be exercised under what
IP Verification

conditions and what is the expected response

• Define features priority
• How many testbenches must be written
Tian-Sheuan Chang

• How complex they need to be

• How they depend on each other

10 Copyright ©2003 All rights reserved

 Benefit of Verification Plan
• Force designers to think through the very time-
consuming process before performing them
• Peer review allows a pro-active assessment of the entire
scope
• Focus efforts first for area of most needed and greatest
payoff
IP Verification

• Minimize the redundant effort

• Tracked and managed more effectively
• Enable verification tests and testbench early
• Enable a separated verification team in parallel to reduce
Tian-Sheuan Chang

design cycle

11 Copyright ©2003 All rights reserved

 From Specification to Feature
• Component-Level Features
– Unit, reusable, ASIC level
– Do not involve system-level interaction with other
component
• System-Level Features
IP Verification

– A subset of an ASIC, a few ASICs, an board design

– Minimize the features verified at this level
– Limited to connectivity, flow-control and inter-operability
Tian-Sheuan Chang

12 Copyright ©2003 All rights reserved

 From Feature to Testcase
• Prioritize
– Must-have – verify all possible configuration & usage
– Should-have – verify basic functionality
– Nice-to-have – verify only as time allow
• Group into testcases
IP Verification

– Configuration, verification strategy

– Testcase: labeled, objective description(list of features)
• Design for verification
Tian-Sheuan Chang

– Identify “hard-to-verify” features

13 Copyright ©2003 All rights reserved

 From Testcase to Testbench
• Testcase naturally fall into groups
– Configuration of the design
– Abstraction level for the stimulus and response
– Verify closely-related features
• Testbench
IP Verification

– One testcase per testbench

– Grouping several testcases into a single testbench
• Verifying testbenches
Tian-Sheuan Chang

– Review by other verification engineer

– Simulation output log

14 Copyright ©2003 All rights reserved

 Verification Strategies
• Three phases
– Subblocks
• Exhaustive functionality verification
• Ensure no syntax errors in the RTL code
• Basic functionality is operational
• Method: simulation, code coverage, TB automation
– Macro
IP Verification

• Interface verification between subblocks

• Backward compatible (regression test suite)
• Method: simulation, code coverage, TB automation, hardware
accelerator
Tian-Sheuan Chang

– Prototyping
• Real prototype runs real application software
• Method: emulator, FPGA, ASIC test chip
• Bottom up approaches
– Locality
– Easier and faster to catch bugs at the lower level
15 Copyright ©2003 All rights reserved
 Types of Verification Tests
• Compliance testing
– For standard based design
• Corner case testing
• Random verification
– Inputs are subjected to valid individual operations
– Prediction of the expected outputs is more complicated
IP Verification

– Create the condition you have not thought

– Hit corner cases
• Assertion-based verification (Property checking)
• Real code testing
Tian-Sheuan Chang

– Avoid misunderstand specification

• Regression testing
– Verify that bug fixing won’t create new bugs
– Run on regular basis

16 Copyright ©2003 All rights reserved

 Taxonomy
• Functional Verification • Timing Verification
– Dynamic – Dynamic timing analysis
• Simulation based • Simulation based
• Require input vectors • Require input vectors
• No 100% guarantee
• No 100% guarantee
• Used in gate-level simulation
– Formal/static • Useful for timing verification of
• Property power-up sequences and
IP Verification

– Mathematical proof timing exception path, e.g.

asynchronous logic, multi-cycle
– No input vectors paths, false paths,
– 100% guarantee – Static timing analysis
• Classifications • Exhaustive search
– Equivalence checking
Tian-Sheuan Chang

• No input vectors
– Model checking • 100% guarantee
– Semi-/dynamic formal • No simulation required
• Simulation based • Fastest approach
• Check assertions during • Sometimes pessimistic due to
incorrect timing exceptions
simulations

17 Copyright ©2003 All rights reserved

 Classification of Verification

Levels of Design Abstraction

Catch the
Specs.

Behavioral RTL Netlist

IP Verification

Functional
Specs. …
Tian-Sheuan Chang

Validation Intent Equivalence Equivalence

Verification Verification Verification

18 Copyright ©2003 All rights reserved

 Functional Verification Methodology

• RTL remains the golden model throughout the

course of functional verification
• Apply extensive functional verification on RTL
– Simulation, code coverage, functional coverage,
property checking, assertion-based checking
IP Verification

• Use equivalence checking to keep it golden for

successive design transformations
Tian-Sheuan Chang

Synthesis
Efficient and Effective Verification
RTL or Netlist RTL or Netlist
Equivalence
Checking
19 Copyright ©2003 All rights reserved
 Outline
• Verification challenges
• Verification process
• Verification tools
• RTL logic simulation
IP Verification

• RTL formal verification

• Verifiable RTL – good stuff
• Verifiable RTL – bad stuff
Tian-Sheuan Chang

• Testbench design
• SOC verification

20 Copyright ©2003 All rights reserved

 Verification Tools (1/2)
• Simulation
– Event driven: good debug environment
– Cycle based: fast simulation time
• Code coverage
– No. of executed lines / total lines
– Coverage on RTL structure
IP Verification

– Verification Navigator, CoverMeter

• Hardware verification languages
– A language providing power constructs for generating
Tian-Sheuan Chang

stimulus and checking response

– Aid creating verification IP and reusable testbenches
– Vera, e, System Verilog, TestBuilder

21 Copyright ©2003 All rights reserved

 Verification Tools (2/2)
• Functional coverage
– Coverage on functionality
• Formal property checking
– Verplex BlackTie, 0-In Search/Confirm
• Verification IP (VIPs)
IP Verification

– Bus functional model (BFM) and bus monitors for

standard protocols
• Hardware modeling
Tian-Sheuan Chang

• Emulation
• Prototyping
– FPGA
– ASIC test chip
22 Copyright ©2003 All rights reserved
 Inspection as Verification
• Fastest, cheapest and most effective to detect
and remove bugs
• How
– Design (specification, architecture) review
– Code (implementation) review
IP Verification

• Line-by-line fashion
• At the subblock level
• Reviewer should fully understand the implementation
Tian-Sheuan Chang

• Purpose is to help drive quality and not for performance

assessment
• Lint tool help spot defects w/o simulation
– VN-Check, nLint, LEDA

23 Copyright ©2003 All rights reserved

 Adversarial Testing
• Designer
– Focus on proving the design is right
• Verification team
– Prove the design is broken
– Keep with the latest tools and methodologies
IP Verification

• The combination of the two gives the best results

Tian-Sheuan Chang

24 Copyright ©2003 All rights reserved

 Limited Production
• Even after robust verification and prototyping, it’s
still not guaranteed to be bug free
• A limited production for new macro is necessary
– 1 to 4 customers
– Small volume
IP Verification

– Reduce the risk of supporting problems

Tian-Sheuan Chang

25 Copyright ©2003 All rights reserved

 Coverage
• A metric identifies important:
– Structures in a design representation
– e.g. HDL lines, FSM states, paths in netlist
• Classes of behavior
– Transactions, event sequences
IP Verification

• Maximize the probability of simulating and

detecting bugs, at minimum cost (in time, labor,
and computation ) [ Dill ICCAD 99]
• Difficult to formally prove that a coverage metric
Tian-Sheuan Chang

provides a good proxy for bugs

• Goal
– Comprehensive validation without redundant effort
26 Copyright ©2003 All rights reserved
 Coverage Metric Classifications
• Ad-hoc metrics
– Bug detection frequency
– Length of simulation after last bug
– Total number of simulation cycles
• Code coverage
IP Verification

– Line coverage
– Branch coverage
– Path coverage
Tian-Sheuan Chang

– Expression Coverage
– Toggle Coverage
• Functional coverage
27 Copyright ©2003 All rights reserved
 Coverage (1/2)
• Hardware code coverage
– Statement, branch, condition, path, toggle, triggering, FSM
– Recommended 100% statement, branch and condition
– 100% code coverage does not mean 100% functional coverage
– Optimize regression suite runs
• Redundancy removal
IP Verification

• Minimize regression test suites

– Quantitative stopping criterion
– Verify more but simulate less
Tian-Sheuan Chang

• Functional coverage
– A user-defined metric that reflects the degree to which functional
features have been exercised during the verification process.

28 Copyright ©2003 All rights reserved

 Coverage (2/2)
• The “Coverage First” Paradigm
– Identify areas that were sufficiently exercised, and
therefore need not be exercised any further
– Replace the need to write a lot of deterministic,
delicately crafted test, by showing that these scenarios
IP Verification

were already encountered

• Functional Coverage
– You can achieve 100% code coverage, and still miss
key areas where bugs can be hiding.
Tian-Sheuan Chang

– It can eliminate the need to write many of the most time

consuming and hard to write tests.

29 Copyright ©2003 All rights reserved

 Code Coverage Process

Original Pre- Instrumented

Testbenches
Model Processor Model
IP Verification

Simulation engine
Tian-Sheuan Chang

Coverage Report Metrics

Metrics Generator Database

30 Copyright ©2003 All rights reserved

 Code Coverage Flow

RTL Code

Testbench Coverage Analysis

IP Verification

Code Coverage Test Suite

Report Optimization Report

No
Tian-Sheuan Chang

Modify Testbench Meet ?

Yes

High Quality Testbench

31 Copyright ©2003 All rights reserved

 Drawbacks of Code Coverage
• No qualitative insight into functional correctness
• Limited to measuring what is controllable
• Activating an erroneous statement does not mean
the bug will manifest itself to an observable output
– Like testing problems
IP Verification

– Cases found where 90% line coverage only achieved

54% observability coverage [Devadas et al. ICCAD 96]
Tian-Sheuan Chang

32 Copyright ©2003 All rights reserved

 Problems with Existing Coverage Tools

Propagation
Controllability
IP Verification

Justification X vs.
Enumeration
Observability
Tian-Sheuan Chang

100% code coverage does not imply

100% functional coverage !
33 Copyright ©2003 All rights reserved
 Increase Observability
• Black-box testing vs. white-box testing
• Event-Monitors and Assertion Checkers
– Halt simulation (if desired)
– Simplifies Debugging
– Increases test stimuli observability
IP Verification

– Measure functional coverage (using a line cover tool)

– Enables formal and semi-formal techniques
– Capture and validate design assumptions and
Tian-Sheuan Chang

constraints

Assertion-based Verification

34 Copyright ©2003 All rights reserved

 Power of Assertion (1/3)
• DEC Alpha 21164 project [Kantrowitz et al.,DAC 1996]
Assertion Checkers 34%
Cache Coherency Checkers 9%
Reference Model Comparison
Register File Trace Compare 8%
Memory State Compare 7%
IP Verification

End-of-Run State Compare 6%

PC Trace Compare 4%
Self-Checking Test 11%
Manual Inspection of Simulation Output 7%
Tian-Sheuan Chang

Simulation hang 6%
Other 8%
Simulation hang 6%
Other 8%

35 Copyright ©2003 All rights reserved

 Power of Assertion (2/3)
• DEC Alpha 21264 project [Taylor et al.,DAC 1998]

Assertion Checker 25%

Register Miscompare 22%
Simulation "No Progress” 15%
IP Verification

PC Miscompare 14%
Memory State Miscompare 8%
Manual Inspection 6%
Tian-Sheuan Chang

Self-Checking Test 5%
Cache Coherency Check 3%
SAVES Check 2%

36 Copyright ©2003 All rights reserved

 Power of Assertion (3/3)
More evidences

• 17% of bugs were identified by assertions on Cyrix

M3 (p1) project [1998]
• 50% of bugs were identified by assertions on Cyrix
IP Verification

M3 (p2) project [1998]

• 85% of all bugs were found using OVL assertions
on HP [2000]
Tian-Sheuan Chang

• 400 bugs (Intel) were found from formal proofs of

assertions [2001]

37 Copyright ©2003 All rights reserved

 Assertion Types
• Invariant
– assert_never(ck,event1, expression, event2)
– assert_always(ck,event1, expression, event2)
• Liveness Event 1 Event 2

– assert_eventually(...)
IP Verification

P P P P P P P P

– assert_eventually_always(...) 0 Invariant time

• Other
Event 1 Event 2
– assert_one_hot(...)
Tian-Sheuan Chang

– event_monitor(...) P

0 Liveness time

38 Copyright ©2003 All rights reserved

 Open Verification Library (OVL)
• Free download from www.verificationlib.org
– Verilog, VHDL and PSL flavors

assert_always assert_odd_parity
assert_change assert_one_hot
IP Verification

assert_decrement assert_proposition
assert_delta assert_range
assert_even_parity assert_time
assert_increment assert_transition
Tian-Sheuan Chang

assert_handshake assert_unchange
assert_never assert_win_change
assert_no_overflow assert_win_unchange
assert_no_transition assert_window
assert_no_underflow assert_zero_one_hot

39 Copyright ©2003 All rights reserved

 Assertion-based Verification
• Assertion
– Design assumption and properties
• “input should range from 0 to 240”
• “after req raises, gnt is expected within 10 clock cycles”
– Break the simulation when assertion fails
– Both the spatial and temporal relationship can be asserted
– Help designers to locate bugs at right place and time
IP Verification

• Approaches
– Library based
• Open verification library. www.verificationlib.org
– Language based
Tian-Sheuan Chang

• PSL (Sugar), System Verilog DAS (OVA)

• On average, 1 line in assertion language = 50 lines in Verilog
• Concept extended to functional monitors and functional
coverage
40 Copyright ©2003 All rights reserved
 Improving Verification with
Assertions
• New Designs:
– Capture requirements and assumptions while writing
HDL
– Use assertions to validate signal assumptions
throughout the design process, e.g., block -> system
IP Verification

transition
• IP / Design Reuse
– Assertions validate correct stimulation of IP within
system
Tian-Sheuan Chang

• Travel with IP
• Provide immediate feedback to IP users
• Reduce support calls to IP vendors
• Document behavior and expectations
41 Copyright ©2003 All rights reserved
 Benefits of Assertion-Based
Verification
• Reduces debugging time
– Assertions can continuously monitor internal signals in
the design, catching violations early in the design
process
• Documents design
IP Verification

– Assertions can be used to capture designer’s intent

• Monitors I/O
– Assertions can be used to verify protocols
Tian-Sheuan Chang

• Improves design quality

– Enables comparing the design specification with the circuit -
throughout the design process
– Assertions can be thought of as a “partial specification” for your
design
42 Copyright ©2003 All rights reserved
 Outline
• Verification challenges
• Verification process
• Verification tools
• RTL logic simulation
IP Verification

• RTL formal verification

• Verifiable RTL – good stuff
• Verifiable RTL – bad stuff
Tian-Sheuan Chang

• Testbench design
• SOC verification

43 Copyright ©2003 All rights reserved

 Fast Simulation
• How to make simulation more productive ?
– Make simulation more efficient
• Coding style, faster workstation, hardware accelerator
– Make simulation more effective
• Code coverage, functional coverage, ABV
IP Verification

Fast Simulation Principle

Tian-Sheuan Chang

A design project must include tailored RTL to

achieve the fastest simulation possible.

44 Copyright ©2003 All rights reserved

 RTL Logic Simulation
• Noble goal - eliminate all design errors before silicon
• Realistic goal - achieve self test on first silicon

Project simulation phases

• Debugging
– Full accessibility, fast turnaround time
IP Verification

• Performance profiling
– To accelerate the simulation
•Log files over networking, large log files
•Bad memory allocated policy
Tian-Sheuan Chang

• Regression
– Efficiency is the king
•Cycle-based, 2-state simulation
• Recreating hardware problems
– Simulation debug & regression
45 Copyright ©2003 All rights reserved
 Choosing Simulation Tools

Subblock module test stage

Subblock 1 Subblock 2 ● Interpreted, event-driven simulator
(VSS, Verilog-XL, VCS)

Block-level Integration stage

Macro 1 ● Compiled, event-driven simulator or
cycle-based simulator
IP Verification

Chip-level Integration stage

● Cycle-based simulator start
Macro 1 Macro 2
● Modules can migrate to emulation
when relatively bug-free
CHIP ● Testbench migrates to emulation last
Tian-Sheuan Chang

Software testing stage

Testbench/ ● Emulation
Real Software CHIP ● Chip and testbench are in the emulator
for max performance

46 Copyright ©2003 All rights reserved

 Difference in Different Modes

Debugging Regression
Phase Phase
Verilog Std Vendor Cycle-based
Compilation Model Model
Signal Full Limited
IP Verification

Accessibility
Waveform Frequent Seldom
Viewing
Logging Full Limited
Tian-Sheuan Chang

Output
PLI Debug Debug
C/C++ mode ON mode OFF

47 Copyright ©2003 All rights reserved

 Visit Minimization
• Visit buses instead of bits
• Bypass evaluation visits to intermediate logic not in an
active path
– Use condition like if()
• Eliminate event visits by using cycle-based evaluation
IP Verification

Visit Minimization Principle

Tian-Sheuan Chang

For best simulation (and any EDA tool) performance,

minimize the frequency and minimize granularity of visits.

48 Copyright ©2003 All rights reserved

 2-State Simulation
2-state methods in place of X
• Zero-initialization finds bugs that X can’t
– Due to X-state optimism
– For more robust power-up verification
• Random initialization should do better
IP Verification

– Capability to regenerate the specific random sequence

– Keep the random seed
• Transform Z’s at tri-state boundaries to random
Tian-Sheuan Chang

2-state values
• 2-state simulates faster than 4 state

49 Copyright ©2003 All rights reserved

 Outline
• Verification challenges
• Verification process
• Verification tools
• RTL logic simulation
IP Verification

• RTL formal verification

• Verifiable RTL – good stuff
• Verifiable RTL – bad stuff
Tian-Sheuan Chang

• Testbench design
• SOC verification

50 Copyright ©2003 All rights reserved

 RTL Formal Verification
• Increasingly complex systems require more time
to verify functionality
• Process of verifying design transformations
should be automated
• Orthogonal Verification Principle
IP Verification

– Separate verification of circuit equality vs. circuit

functionality
• Coding techniques to facilitate formal verification
Tian-Sheuan Chang

51 Copyright ©2003 All rights reserved

 Equivalence Checking
• Checking after
– Synthesis
– Scan chain insertion
– Clock-tree synthesis
– Manual modification
IP Verification

– Place and route

– ECO
• Equivalence checking for large designs
Tian-Sheuan Chang

– Tough due to exponential-of-input size nature

– Logic cone partitioning is required

52 Copyright ©2003 All rights reserved

 Cutpoint
• Internal cross-design signal equivalence pairs are
referred as cutpoint
• Partition large cones of logic into smaller cones
for the proof
IP Verification

Cutpoints
Tian-Sheuan Chang

53 Copyright ©2003 All rights reserved

 Functional complexity Isolation
// Not so good Cutpoints
assign c_indx = (((coord_x * coord_y) & indx_mask) + indx_offset);

// Better Cutpoints
mult_16x15 mult1 (coord_x, coord_y, mult_prod);
assign c_indx = ((mult1_prod & indx_mask) + indx_offset);
IP Verification

Cutpoint Identification Principle

Tian-Sheuan Chang

A single design decision pertaining to functional complexity

must be isolated and localized within a module to facilitate
equivalence checking cutpoint identification

54 Copyright ©2003 All rights reserved

 Test Expression Observability (1/2)

Test Expression Observability Principle

Complex test expressions within a Verilog case or if
IP Verification

statement must be factored into a variable assignment.

Tian-Sheuan Chang

55 Copyright ©2003 All rights reserved

 Test Expression Observability (2/2)

// Not so good
case ((a & b | c ^ d) || mem[idx])
4'b0100: c_nxt_st = r_nxt_st << 1;
4'b1000: c_nxt_st = r_nxt_st >> 1;
default: c_nxt_st = r_nxt_st;
endcase;
IP Verification

//Good
c_nxt_st_test = (a & b | c ^ d) || mem[idx];
Tian-Sheuan Chang

case (c_nxt_st_test)
4'b0100: c_nxt_st = r_nxt_st << 1;
4'b1000: c_nxt_st = r_nxt_st >> 1;
default: c_nxt_st = r_nxt_st;
endcase;

56 Copyright ©2003 All rights reserved

 Outline
• Verification challenges
• Verification process
• Verification tools
• RTL logic simulation
IP Verification

• RTL formal verification

• Verifiable RTL – good stuff
• Verifiable RTL – bad stuff
Tian-Sheuan Chang

• Testbench design
• SOC verification

57 Copyright ©2003 All rights reserved

 Why Verifiable RTL

• A lot of guideline for reuse and synthesis exists

• Lack of RTL coding guidelines to optimize the
verification process
• This vacuum becomes a problem as:
– Design complexity increases
IP Verification

– Advance verification processes are considered

• Cycle-based simulation, 2-state simulation, property checking,
equivalence checking, emulation
• Verifiable RTL style consists of
Tian-Sheuan Chang

– A verifiable subset of Verilog

– A set of RTL coding guidelines
– A set of fundamental principles

58 Copyright ©2003 All rights reserved

 RT-Level X-State Optimism (1/2)
• Optimism – State Machine
case (d)
2'b00 : e = 2'b01;
2'b01 : e = 2'b11;
2'b10 : e = 2'b10;
default : e = 2'b00;
IP Verification

endcase

• If d == 2’bXX, case statement always takes the

Tian-Sheuan Chang

default branch!
• Alternate branches never test during startup!

59 Copyright ©2003 All rights reserved

 RT-Level X-State Optimism (2/2)
• Accuracy Impractical

case (d)
2'b00 : e = 2'b01;
2'b0X : e = 2'bX1;
2'b01 : e = 2'b11;
IP Verification

2’bX0 : e = 2’bXX;
2'bX1 : e = 2'bXX;
2'b11 : e = 2'b00;
Tian-Sheuan Chang

2'b1X : e = 2'bX0;
2'b10 : e = 2'b10;
2'bXX : e = 2'bXX;
endcase

60 Copyright ©2003 All rights reserved

 X? In Real World

module mux (a,b,s,q); There are no X’s

in the real circuit!
output q;
reg a, b, q;
a
reg [1:0] s;
always @(a or b or s) s[1]
q
IP Verification

begin
case (s)//synopsys full_case
b
2’b11: q = 1'bz;
2’b01: q = a;
Tian-Sheuan Chang

2’b10: q = b; s[0]
endcase
end If s[1] = 0 and s[2] = 0
endmodule we might be SMOKING!
Semantic Mismatch
61 Copyright ©2003 All rights reserved
 How Slow Can Your Simulation Go?

for (i=0; i<64; i=i+1) begin

bit5 = (i > 31);
bit4 = (i > 15) && (i < 32) || (i > 47);
bit3 = (i > 7) && (i < 16) || (i > 23) && (i < 32) || (i > 39) && (i < 48) || (i > 55);
bit2 = (i > 3) && (i < 8) || (i > 11) && (i < 16) || (i > 19) && (i < 24) || (i > 27) && (i < 32) ||
(i > 35) && (i < 40) || (i > 43) && (i < 48) || (i > 51 ) && (i < 56) || (i > 59);
bit1 = (i == 2) || (i == 3) || (i == 6) || (i == 7) || (i == 10) || (i == 11) || (i == 14) || (i == 15) ||
(i == 18) || (i == 19) || (i == 22) || (i == 23) || (i == 26) || (i == 27) || (i == 30) ||
(i == 31) || (i == 34) || (i == 35) || (i == 38) || (i == 39) || (i == 42) || (i == 43) ||
IP Verification

(i == 46) || (i == 47) || (i == 50) || (i == 51) || (i == 54) || (i == 55) || (i == 58) ||

(i == 59) || (i == 62) || (i == 63);
bit0 = (i == 1) || (i == 3) || (i == 5) || (i == 7) || (i == 9) || (i == 11) || (i == 13) || (i == 15) ||
(i == 17) || (i == 19) || (i == 21) || (i == 23) || (i == 25) || (i == 27) || (i == 29) ||
(i == 31) || (i == 33) || (i == 35) || (i == 37) || (i == 39) || (i == 41) || (i == 43) ||
(i == 45) || (i == 47) || (i == 49) || (i == 51) || (i == 53) || (i == 55) || (i == 57) ||
Tian-Sheuan Chang

(i == 59) || (i == 61) || (i == 63);

tmp [i] = pd [i] && (bit5 ~^ cell[5]) && (bit4 ~^ cell[4]) && (bit3 ~^ cell[3]) && (bit2 ~^
cell[2]) && (bit1 ~^ cell[1]) && (bit0 ~^ cell[0]);
end // for
hit = | tmp ;

62 Copyright ©2003 All rights reserved

 Better Ways for Speed

parallel mask fashion for more parallelism -- 1000x faster

tmp= pd & (~(64'hffffffff00000000 ^ {64{cell[5]}}))

& (~(64'hffff0000ffff0000 ^ {64{cell[4]}}))
& (~(64'hff00ff00ff00ff00 ^ {64{cell[3]}}))
& (~(64'hf0f0f0f0f0f0f0f0 ^ {64{cell[2]}}))
IP Verification

& (~(64'hcccccccccccccccc ^ {64{cell[1]}}))

& (~(64'haaaaaaaaaaaaaaaa ^ {64{cell[0]}}));
hit = | tmp;
Tian-Sheuan Chang

bit-indexing for more and more parallelism -- 3000x faster

hit = pd[cell];

63 Copyright ©2003 All rights reserved

 Verifiable Subset
• Two ways of constructed a design
– to make it so simple that there are obviously no deficiencies
– to make it so complicated that there are no obvious deficiencies
• However, synthesizer vendors tend to enlarge the synthesizable
subset
• Where there are 2/3/4 ways to express the same thing in RTL
IP Verification

– PICK Simple ONE, Simple wins in verification

Verifiable Subset Principle

Tian-Sheuan Chang

A design project must select a simple HDL verifiable subset,

which serves all verification tools within the design flow as well as
providing an uncomplicated mechanism for conveying clear
functional intent between designers

64 Copyright ©2003 All rights reserved

 Verifiable Verilog Keyword
• Verifiable subset is a subset of synthesizable subset
• 27 out of the 102 Verilog-1995 keywords
• “for” looping construct could be used for extra exception

always else initial parameter

IP Verification

assign end inout posedge

begin endcase input reg
case endfunction module tri
Tian-Sheuan Chang

casex endmodule negedge tri0

default function or tri1
if output wire

65 Copyright ©2003 All rights reserved

 Unsupported Operators

operator example function

- -a unary minus
IP Verification

* a*b multiply
/ a/b divide
=== a===b equality (0/1/X/Z)
Tian-Sheuan Chang

!== a!==b inequality (0/1/X/Z)

66 Copyright ©2003 All rights reserved

 Asynchronous Principle

• Asynchronous - not addressed by RTL verification. Requires:

– Protocol verification - Petri net modeling
– Failure rate analysis - Circuit analysis
IP Verification

Asynchronous Principle
A design project must minimize and isolate resynchronization
Tian-Sheuan Chang

logic between asynchronous clock domains.

67 Copyright ©2003 All rights reserved

 Combinational Feedback Principle
• Forms of Feedback
– Design errors
– False path
– Apparent
IP Verification

Combinational Feedback Principle

Designers must not use any form of combinational logic
Tian-Sheuan Chang

feedback (real, false-path, apparent)

in their Verilog.

68 Copyright ©2003 All rights reserved

 False Path

s module m (s, a, b, y, z);

input s;
input a, b;
a output y, z;
IP Verification

y
wire s, a, b;
wire y, z;
assign y = s ? a : z;
assign z = s ? y : b;
Tian-Sheuan Chang

z endmodule
b

69 Copyright ©2003 All rights reserved

 Apparent Feedback

module m (a, d); module m (a, d); module m (a, d);

input a; input a; input a;
output d; output d; output d;
reg b, d; reg b, c, d; wire b, c, d;
wire c; always @(a) assign b = a;
always @(a or c) assign d = c;
IP Verification

begin
begin b = a; assign c = b;
b = a; c = b; Endmodule
d = c; d = c; //order
end end //independent
Tian-Sheuan Chang

assign c = b; endmodule
endmodule //order dependent Fix 2

Fix 1

70 Copyright ©2003 All rights reserved

 Verifiable case/casex
• case/casex practices supporting verifiable RTL
– Fully specified case/casex statements
– Consistent test signal and constant widths
IP Verification
Tian-Sheuan Chang

71 Copyright ©2003 All rights reserved

 Fully specified case/casex
• Pros
– Faster boolean equivalence checking
• No don’t care conditions
– RTL - Gate-level simulation alignment
– Improved RTL simulation:
• Performance (no X state)
IP Verification

• Verification - startup state, fault simulation

– RTL Manufacturing test simulation
• Cons
Tian-Sheuan Chang

– Worse synthesis result (not always true)

• Alternative solution exists
– Loss of simplicity
• Alterative solution has more complicated coding style

72 Copyright ©2003 All rights reserved

 case/casex – Verification vs
Synthesis
module one_hot(c_hot,c_code);
input [7:0] c_hot;
output [2:0] c_code;
reg [2:0] c_code;
always @ (c_hot) begin
case (c_hot) // synthesis full_case, for synthesis?
8’b10000000: c_code = 3’b000;
8’b01000000: c_code = 3’b001;
IP Verification

8’b00100000: c_code = 3’b010;

8’b00010000: c_code = 3’b011;
8’b00001000: c_code = 3’b100;
8’b00000100: c_code = 3’b101;
8’b00000010: c_code = 3’b110;
Tian-Sheuan Chang

8’b00000001: c_code = 3’b111;

default: c_code = 3’b000; // or for verification
endcase
end // always (c_hot)
endmodule // one_hot
If default case is used, 3X gates are generated

73 Copyright ©2003 All rights reserved

 Partially-Specified to Fully-
Specified
• For smaller case statements
– minimization savings not worth loss of verifiability
• For larger case statements –
– use alternative (fully specified) coding style
IP Verification
Tian-Sheuan Chang

74 Copyright ©2003 All rights reserved

 case/casex – Alternative for One-
Hot
module one_hot(c_hot,c_code);
input [7:0] c_hot;
output [2:0] c_code;
reg [2:0] c_code;
reg [2:0] c_code0,c_code1,c_code2,c_code3;
reg [2:0] c_code4,c_code5,c_code6;
always @ (c_hot) begin
IP Verification

c_code6 = (c_hot [6]) ? 3’b001 : 3’b000;

c_code5 = (c_hot [5]) ? 3’b010 : 3’b000;
c_code4 = (c_hot [4]) ? 3’b011 : 3’b000;
c_code3 = (c_hot [3]) ? 3’b100 : 3’b000;
c_code2 = (c_hot [2]) ? 3’b101 : 3’b000;
Tian-Sheuan Chang

c_code1 = (c_hot [1]) ? 3’b110 : 3’b000;

c_code0 = (c_hot [0]) ? 3’b111 : 3’b000;
c_code = c_code0 | c_code1 | c_code2 | c_code3 |
c_code4 | c_code5 | c_code6;
end // always (c_hot)
endmodule // one_hot

75 Copyright ©2003 All rights reserved

 Outline
• Verification challenges
• Verification process
• Verification tools
• RTL logic simulation
IP Verification

• RTL formal verification

• Verifiable RTL – good stuff
• Verifiable RTL – bad stuff
Tian-Sheuan Chang

• Testbench design
• SOC verification

76 Copyright ©2003 All rights reserved

 X-state Pessimism

X-state Pessimism - arithmetic

reg [15:0] a,b,c;

…
begin
IP Verification

b = 16’b0000000000000000;
c = 16’b000000000000X000;
a = b + c;
Tian-Sheuan Chang

$display(" a = %b",a);
end

a = 16’bXXXXXXXXXXXXXXXX

77 Copyright ©2003 All rights reserved

 X-state Optimism - case Statement
reg [1:0] d,e;
…
begin
case (d)
2’b00 : e = 2’b01;
2’b01 : e = 2’b11;
2’b10 : e = 2’b10;
IP Verification

default : e = 2’b00;
endcase
$display(" e = %b",e);
end
Tian-Sheuan Chang

• If d contains an X then e = 2’b00

• RTL simulation will miss verifying alternate branches
(especially at the start-up sequences)
78 Copyright ©2003 All rights reserved
 Accuracy impractical
• Simulation performance.
• Labor content.
– Added X-state tests
– branch to boolean conversion
• Complex verification
IP Verification

• Completeness
• Synthesis
Tian-Sheuan Chang

79 Copyright ©2003 All rights reserved

 Prohibit X for “don’t care’s”
...
case (select)
2’b01 : mux = b;
2’b10 : mux = c;
IP Verification

default : mux = 2’bX;

endcase
Tian-Sheuan Chang

80 Copyright ©2003 All rights reserved

 X in “don’t care’s”
• Mask errors which can’t be found at RT-level
simulation
• Slows RT-level simulation
• Slows RTL-to-gate equivalence checking
• Causes semantic mismatches between RTL and
IP Verification

gate-level simulation.
Tian-Sheuan Chang

81 Copyright ©2003 All rights reserved

 Visit Minimization
• Criminals to degrade simulation performance
– referencing bits instead of buses
– Run-time configuration tests
– loops.
IP Verification
Tian-Sheuan Chang

82 Copyright ©2003 All rights reserved

 Bits v.s. Bus

BAD: Explicit bit visits

c_ecc_out_1 = c_in [10] ^
c_in[11] ^ c_in[12] ^
c_in[13] ^ c_in[14] ^ c_ecc_out_1 =
IP Verification

c_in[15] ^ c_in[16] ^ ^ (c_in & 40’h003ffff893);

c_in[17] ^ c_in[18] ^
c_in[19] ^ c_in[20] ^
c_in[21] ^ c_in[22] ^
c_in[23] ^ c_in[24] ^ GOOD: Parallel
Tian-Sheuan Chang

c_in[25] ^ c_in[26] ^ value evaluation

c_in[27]^ c_in[28] ^
c_in[32] ^ c_in[35] ^
c_in[38] ^ c_in[39];

83 Copyright ©2003 All rights reserved

 Run-Time Configuration
module fifo(
...
parameter WIDTH = 13;
parameter DEPTH = 32;
parameter ENCODE = 0;
...
function [31:0] encoder;
IP Verification

input [WIDTH-1:0] indata;

begin
if (ENCODE != 0) begin
< calculate encode value based on indata >
end
Tian-Sheuan Chang

else
encoder = indata;
End

Use conditional compilation directives `if, `else, `elseif,

`endif instead
84 Copyright ©2003 All rights reserved
 For-Loops

BAD: Individual bit

visits, loop overhead

input [‘N-1:0] a; input [‘N-1:0] a;

output [‘N-1:0] b; output [‘N-1:0] b;
integer i; assign b = ~a;
IP Verification

reg [‘N-1:0] b;
always @ (a) begin
for (i=0; i<=‘N-1; i=i+1)
b[i] = ~a[i];
GOOD: Parallel
value evaluation
Tian-Sheuan Chang

end

85 Copyright ©2003 All rights reserved

 For-Loop: Bus Reversal

BAD: Better:
For-loop Concatenation
input [15:0] a;
IP Verification

input [15:0] a;
output [15:0] b; output [0:15] b;
integer i; assign b = { a[0], a[1],
reg [15:0] b; a[2], a[3], a[4], a[5],
always @ (a) begin a[6], a[7], a[8], a[9],
Tian-Sheuan Chang

for (i=0; i<=15; a[10], a[11], a[12],

i=i+1) a[13], a[14], a[15] };
b[15 - i]] =
~a[i];
end

86 Copyright ©2003 All rights reserved

 For Loop
• Simulate slow
– from 10X to > 1000X slower than non-for loop versions.
• Synthesizes slow
• Memory clear
– only legitimate for loop use in chip design.
IP Verification

• Avoid using the for loop whenever possible

Tian-Sheuan Chang

87 Copyright ©2003 All rights reserved

 Faithful Semantics
• Bad coding style - unequal design information
– HDL simulator information not used in synthesis
– Synthesis switches not used by simulator.
• X state
IP Verification

Faithful Semantics Principle

Tian-Sheuan Chang

An RTL coding style and set of tool directives must be

selected that insures semantic consistency between
simulation, synthesis and formal verification tools.

88 Copyright ©2003 All rights reserved

 Full_case & Parallel_case
• Fully-specify case/casex
– Do not use full_case and parallel_case
• Eliminate case-item constant overlaps
• Find alternative coding if necessary
• Implement RTL priority encoder as multiplexer
IP Verification
Tian-Sheuan Chang

89 Copyright ©2003 All rights reserved

 Verilog Initial Blocks
• Explicitly creates RTL-gate differences.
• Better - place in testbench
• Best - encapsulate within storage element (FF’s
memories) library modules
IP Verification
Tian-Sheuan Chang

90 Copyright ©2003 All rights reserved

 Careless Coding
• Incomplete sensitivity list
• Latch inference
• Incorrect procedural statement ordering
IP Verification
Tian-Sheuan Chang

91 Copyright ©2003 All rights reserved

 Timing Problems
• Project-wide policy
• #0; delays
• Non-blocking assignment delay
• Testbench delays
IP Verification
Tian-Sheuan Chang

92 Copyright ©2003 All rights reserved

 Race Condition
• Race • No race
//file a.v //file a.v
always @(posedge ck) always @(posedge ck)
begin begin
b = a; b <= a;
IP Verification

end end
//file b.v //file b.v
always @(posedge ck) always @(posedge ck)
Tian-Sheuan Chang

begin begin
c = b; c <= b;
end end

93 Copyright ©2003 All rights reserved

 Testbench Delays
• Testbench designers insert delays to offset timing
with respect to clock edges for:
– inserting control states
– observing states
• Testbench timing often less disciplined than chip
IP Verification

timing
Tian-Sheuan Chang

94 Copyright ©2003 All rights reserved

 User-Defined Primitive (UDP)
• Not RTL!
• Often preclude use of new RTL verification tools
• Sequential UDP’s present special challenges
IP Verification

Just say no
Tian-Sheuan Chang

95 Copyright ©2003 All rights reserved

 Summary
• Code your RTL for synthesis and verification as
well

• Verifiable RTL coding styles

– Prevent you from pitfalls in the verification process
IP Verification

– Make you curse verification-related tools less

– Increase the verification performance
– Provide better verification outcome
Tian-Sheuan Chang

– Give you more robust design

96 Copyright ©2003 All rights reserved

 Outline
• Verification challenges
• Verification process
• Verification tools
• RTL logic simulation
IP Verification

• RTL formal verification

• Verifiable RTL – good stuff
• Verifiable RTL – bad stuff
Tian-Sheuan Chang

• Testbench design
• SOC verification

97 Copyright ©2003 All rights reserved

 Testbench Design (1)
• The testbench design differs depending on the
function of the macro
– microprocessor macro, test program,
– bus-interface macro, use bus functional models and
bus monitors
IP Verification

• Subblock testbench

Output Interface
Input Interface
Tian-Sheuan Chang

Input Output
Transaction Transaction
Generator Checker

98 Copyright ©2003 All rights reserved

 Testbench Design (2)
• Transaction-based stimulus generation and response
checking
– Legal set of input
– Corner case and random test
• Auto or semi-auto stimulus generation is preferred
• Automatic response checking is a must
– Self-checking is recommended
IP Verification

– Detect problems as early as possible

• Reusable testbench

Devce under test

Tian-Sheuan Chang

Automatic
stimulus response
checking
Referene model
(C/C++,HDL/HVL,
Hardware modeler)

99 Copyright ©2003 All rights reserved

 Testbench Authoring
• An effective testbench
– Concurrency
– Encapsulation and abstraction
– Self-checking
– Automatic test stimulus generation
– Reusable components
IP Verification

• Testbench authoring tools

– Partitioning the responsibility among TVMs and tests
– Specifying cause and effect relationships among transactions
Tian-Sheuan Chang

– Specifying complex concurrency using inter-transaction

synchronization
– Specifying localized constraints in the attributes of transaction

100 Copyright ©2003 All rights reserved

 Macro Testbench
Application
Software

Drivers

HW/SW cosim
Environment
IP Verification

Translator
Tian-Sheuan Chang

Application
PCI Bus
Bus
Monitor
Monitor

PCI Bus Application Bus

PCI Macro
Functional Model Functional Model

101 Copyright ©2003 All rights reserved

 Bus Functional Model
PCI
AGP Design
USB RAM Under Test
IEEE1394
CPU
Random Definition: Simulation model
Logic
Ethernet allowing designers to verify
Bus Interface
SCSI-2 Logic compliance to a particular
EISA specification prior to prototyping:
VME
? 1. Model the bus transactions on the
System Bus
bus, each read and write
IP Verification

transaction is specified by the test

developer
2. Monitors bus activity for protocol
compliance
Tian-Sheuan Chang

Rest of the system bus monitor

Bus Functional Model

102 Copyright ©2003 All rights reserved

 Benefits
• Test for interoperability during simulation
• Verify compliance prior to fabrication
• Generate test vectors more efficiently
• Learn a new bus faster
IP Verification

• BFM is written in RTL, C/C++, or testbench

automation tools System Bus
– Flexibility
Tian-Sheuan Chang

– Visibility into model operation

Test of the system bus monitor

103 Copyright ©2003 All rights reserved

 Verification Suite Design
• Once built the testbench , we can develop a set of
tests to verify the correct behavior of the macro
• Test sets
– functional testing
– corner case testing
IP Verification

– code coverage
– random testing
Tian-Sheuan Chang

104 Copyright ©2003 All rights reserved

 Transaction-based Verification
IP Verification
Tian-Sheuan Chang

105 Copyright ©2003 All rights reserved

 Efficient Simulating Debug (1)
• Transaction viewing
– The abstract information about a transaction is
displayed.
• Cause and effect
– The relationships among transactions are displayed.
IP Verification

• Error transactions
– An error detected during simulation is recorded.
• Concurrency
Tian-Sheuan Chang

– Out-of-order/pipelined transactions are displayed

106 Copyright ©2003 All rights reserved

 Efficient Simulating Debug (2)
IP Verification
Tian-Sheuan Chang

107 Copyright ©2003 All rights reserved

 Behavioral Models
• Describe the black-box functionality of a design, required
for all IPs
• Benefits
– Audit of the specification
– Development and debug of testbench in parallel with RTL coding
– System verification can start earlier
– It can be used as an secure evaluation and integration tool by your
IP Verification

customer
– Faster to write, debug, simulate and time-to-market
• Cost
– Require additional resource to write the behavior model
Tian-Sheuan Chang

– Maintenance requires additional efforts

• BFM is required particularly for interface Ips
• ISA Model is required for processor IPs
• Commercial available for standard based IP
– Verification IP, e.g. PCI, IEEE 1394, USB
108 Copyright ©2003 All rights reserved
 Verification IP
• A package including well-designed and well
verified BFM/monitor for a specific
protocol/interface
– AMBA, Ethernet, SONET, UTOPIA, PCI, USB, UART,
CAN, ..
IP Verification

– Avoid re-invent-the-wheel
– Accelerate the verification
Tian-Sheuan Chang

109 Copyright ©2003 All rights reserved

 Verify AMBA System

AHB Master 1 AHB Master 2

APB Slave 1
AHB Master AHB Master
Protocol Checker Protocol Checker
AHB Master AHB Master APB Slave
Coverage Mointor Coverage Mointor Coverage Mointor

Bridge
IP Verification

AHB Slave AHB Slave APB Slave

Protocol Checker Protocol Checker Coverage Mointor
AHB Slave AHB Slave
Coverage Mointor Coverage Mointor
APB Slave 2

AHB Slave 1 AHB Slave 2

Tian-Sheuan Chang

hclk
htrans NSEQ SEQ SEQ SEQ
haddr A A+4 A+8 A+8 AHB Master Protocol Transfer Response Summary
hwrite -------------------------------------
hwdata DA DA+4 DA+4 DA+8
violation: Response Type | Count
hresp OKAY OKAY OKAY OKAY Address was changed -------------------------------------
hready
while in transfer OKAY | 104
ERROR | 0
extension (hready low). RETRY | 5
Ref. Spec. section 3.4 SPLIT | 13
-------------------------------------
110 Copyright ©2003 All rights reserved
 Verification Support
• Protocol Checker
– Monitor the transactions on an interface and check for
any invalid operation
• Embedded in the test bench
• Embedded in the design
– Error and/or warning messing of bus protocol
IP Verification

• Expected results checker

– Embedded in the test bench
– Checks the results of a simulation against a previously
Tian-Sheuan Chang

specified, expected response file.

• Performance monitor
– Number of transfers, idle cycles...

111 Copyright ©2003 All rights reserved

 Simulation Management
• Pass or Fail ?
– Produce a message that the simulation was terminated
normally
• SDF Back-Annotation
– Very time-consuming
– Invoke the simulation once for multiple testcases
IP Verification

• Output File Management

– A copy of output message: verilog.log
– Dump waveform only for needed
Tian-Sheuan Chang

– Run multiple simulations in parallel

• Use “-l” option to change the name of the output log file
• Use script to help manage the configuration of a simulation and
the name of output file

112 Copyright ©2003 All rights reserved

 Regression
• A regression suite ensures that modifications to a
design remain backward compatible with
previously verified functionality
• Regressions are run at regular intervals
• Provide a fast mode
IP Verification

• Regression Management
– Simulation never terminate
– Put a time bomb in all simulations to prevent simulation
Tian-Sheuan Chang

running forever
– Success or failure of each testcase should be checked
after regression test

113 Copyright ©2003 All rights reserved

 Outline
• Verification challenges
• Verification process
• Verification tools
• RTL logic simulation
IP Verification

• RTL formal verification

• Verifiable RTL – good stuff
• Verifiable RTL – bad stuff
Tian-Sheuan Chang

• Testbench design
• IP Modeling
• SOC verification
114 Copyright ©2003 All rights reserved
 The Intent of Different Level of IP Model
• Design exploration at higher level
– Import of top-level constraint and block architecture
– Hierarchical, complete system refinement
– Less time for validating system requirement
– More design space of algorithm and system
architecture
IP Verification

• Simple and efficient verification and simulation

– Functional verification
– Timing simulation/verification
Tian-Sheuan Chang

– Separate internal and external (interface) verification

– Analysis: power and timing
• Verification support: e.g., monitor, checker...

115 Copyright ©2003 All rights reserved

 General Modeling Concepts
• Interface model
– Synonym: bus functional, interface behavioral
• Behavioral model
– Behavior = function with timing

Interface
– Abstract behavioral model out=AxB
M-Bus
IP Verification

– Detailed behavioral model Clock

Abstract
• Structural model Behavioral
Model
Tian-Sheuan Chang

CS
W_En
Interface

Interface

Interface

Interface
B B 20
S
out=AxB out=AxB Data_Bus
B B 16
Addr_Bus
Clock
Detailed
Interface Behavioral Structural
Behavioral
Model Model Model
Model
116 Copyright ©2003 All rights reserved
 Issues of IP Modeling
• Attributes
– What is the sufficient set of model attributes?
– How are these model attributes validated?
– How is the proper application of an abstract model
specified?
IP Verification

• Two important dimensions of time

– Model development time is labor intensive: model
reusability
Tian-Sheuan Chang

– Simulation time depends upon strategy chosen for

mixed domain simulations

117 Copyright ©2003 All rights reserved

 From Requirement to Delivery

Hierarchy Hierarchy
Refinemenet Validaton

Customer Product
Needs System Vaildation "Pattern" Deliver
System Function System Function
IP Verification

Architecture Architecture
Verification
Behavioral Behavioral

RTL Test
Tian-Sheuan Chang

Patern
Logical Netlist Logical Device
Layout
Wafer
Abstract Mask Real
Fab

118 Copyright ©2003 All rights reserved

 Example: Hierarchical Design
Refinement
Vertical refinement Horizontal refinement: Partition
F1 F2 F3

F4 F5 F6 F1 F2 F3

F4 F5 F6

CPU MEM Co-P

IP Verification

F1 F2 F3
In 1 In 2 Out 1
F4 F5 F6
Tian-Sheuan Chang

CPU MEM Co-P

In 1 In 2 Out 1

119 Copyright ©2003 All rights reserved

 Example: Manage Size and Run-
Time
Start at RTL

RTL Coding RTL Test Synthesis P&R

Integration Test

Start at behavioral level

IP Verification

RTL Coding RTL Test Synthesis P&R

Behavioral Level Coding

Tian-Sheuan Chang

Behavioral Level Test

Integration Test

120 Copyright ©2003 All rights reserved

 IP Modeling
General Modeling Concept
Primary Model Classes: Specialized Model Classes: Computational Model Classes:

Behavioral Model Performance Model Data Flow Graph Model

Functional Model Interface Model Other Models

Structural Model Hybrid Model

IP Verification

System Models Architecture Models Hardware Models Software Models

Executable Specification Token-based Performance Model Detailed-Behavioral Model Pseudo-Code
Mathematical-Equation
Abstract-Behavioral Model Register Transfer Level (RTL) Model High Level Language (HLL)
Model
Algorithm Model Data Flow Graph (DFG) Task Primitive Logic-Level Model Assembly code

Instruction Set Architecture (ISA) Model Gate-Level Model Micro-Code

Tian-Sheuan Chang

Switch-Level Model Object Code

Circuit-Level Model

Precision Axis
Temporal Precision Axis
Data Precision Axis
Functional Precision Axis
Structural Precision Axis
Software Programming Precision Axis
121 Copyright ©2003 All rights reserved
 CPU Model
• CPU model enable
– Estimate software performance
– Analyze system trade offs
• CPU model
– Bus functional model
IP Verification

I/O Bus
Compile to transactions Bus functional Events Hardware
Application code
host processor model simulator

– Instruction set simulator (ISS)

Tian-Sheuan Chang

• Instruction accurate
• Cycle accurate
– Virtual processor model (Cadence VCC technology)

122 Copyright ©2003 All rights reserved

 ARM Modeling (1/4)
Efficiency
Concept
System model
Instruction set simulators (ISS)
Co-verification model
Bus Interface model
IP Verification

Behavioral/RTL model
Design signoff models

Hardware modeling
Tian-Sheuan Chang

Gate Level netlist model

Silicon Accuracy

123 Copyright ©2003 All rights reserved

 ARM Modeling (2/4)
• System Model • Co-verification model
– Provision of customized – Each ARM processor core
Software Debugger/ARMulator contains a co-verification
packages, suitable for dataflow simulator component and a
simulation environments. bus interface model
component
– Cadence Signal Processing
– Co-verification simulator:
Worksystem (SPW) and
IP Verification

combines the properties of

Synopsys COSSAP Stream an advanced ISS with the
Driven Simulator bus cycle accurate pin
information capability
required to drive a hardware
Tian-Sheuan Chang

simulator
– CoWare N2C Design
System, Synopsys Eaglei, to
name a few.

124 Copyright ©2003 All rights reserved

 ARM Modeling (3/4)
• Bus interface models (BIM) • Design signoff models
– Run a list of bus transactions – Full architectural
to stimulate simulated functionality and full timing
hardware under test accurate simulation
– Accept process specific
– Allowing the designer to
timing and back annotated
concentrate on the hardware
timing
design without waiting for the
IP Verification

ARM control software to be – Used to ‘sign off’’ design

developed. before committing silicon
– Generated using ModelGen – Be compiled 'C' code which
enables protection of the
Tian-Sheuan Chang

inherent IP and superior

simulation execution speed
over pure HDL models
– Generated using ModelGen

125 Copyright ©2003 All rights reserved

 ARM Modeling (4/4)
• Hardware Modeling • Fault grading netlist
– Real chip-based products, – Full custom marcocells
based on real silicon yields models suitable for
– For logic and fault hardware accelerated fault
simulation grading, system simulation
– Synopsys ModelSource and emulation
– Emulator: IKOS, Mentor
IP Verification

hardware modeling systems

Graphics and Quickturn;
Simulation: IKOS
Tian-Sheuan Chang

126 Copyright ©2003 All rights reserved

 Intent of ModelGen
• Key requirements for ARM’s modeling
environment:
– Deliver highly secure models
– Minimize time spent creating, porting and re-verifying
models
IP Verification

– Support mixed-source languages—HDL, C and full

custom modeling
– Support multiple design and verification environments
Tian-Sheuan Chang

– Enable efficient simulation

– Provide a timing annotation solution that does not
compromise IP security

127 Copyright ©2003 All rights reserved

 “ModelGen” Timing Shell
• Overview:
– Black-box model
• Obscured IP
ModelGen Source
– User supplied timing (MGS)

(SDF)
IP Verification

– Single model
• Easily verifiable
– Exported State
Tian-Sheuan Chang

– Programmer model
• Nine-value Logic/Full
– Supports checkpointing

128 Copyright ©2003 All rights reserved

 Example of Model Generation Flow
IP Verification
Tian-Sheuan Chang

Synopsys VMC/VhMC based model generation flow

129 Copyright
VMC: Verilog Model Compiler, ©2003
VhMC: VHDL All Compiler
Model rights reserved
 Behavioral Model for A/MS
• Describes the functionality and performance of a VC block
without providing actual detailed implementation.
• Needed for system designers to determine the possibility
of implementing the system architecture
• It is a kind of abstract behavioral model
IP Verification

Bounds of actual behavioral

Actual behavioral
Signal Signal
Tian-Sheuan Chang

Frequency Frequency
Behavioral Model Block Detail Model
130 Copyright ©2003 All rights reserved
 Functional/Timing Digital Simulation
Model
• Used to tie in functional verification and timing simulation
with other parts of the system
• Describes the functionality and timing behavior of the
entire A/MS VC between its input and output pins.
• Pin accurate not meant to be synthesizable
IP Verification

• It is a kind of detailed-behavioral model

• Example of PLL: represent the timing relationship of
reference clock input vs. generate output clock.
– Model it by actually representing the structure of the PLL, or
Tian-Sheuan Chang

– Model it as just a delay value based on a simple calculation from

some parameters.

131 Copyright ©2003 All rights reserved

 Interface Model
• Describes the operation of a component with respect to its
surrounding environment.
• The external connective points (e.g ports or parameters),
functional and timing details of the interface are provided
to show how the component exchanges information with
its environment.
IP Verification

• Also named as bus functional model and interface

behavioral model
• For A/MS VC
Tian-Sheuan Chang

– Only the digital interface is described

– Analog inputs and outputs are not considered

132 Copyright ©2003 All rights reserved

 Peripheral Interconnect Model
• Specifies the interconnection RCs for the peripheral
interconnect between the physical I/O ports and the
internal gates of the VC
• Used to accurately calculate the interconnect delays and
output cell delays associated with the VC
• Used only for the digital interface of the A/MS VC
IP Verification

VC
Tian-Sheuan Chang

Block
VC Internal VC
Model

Peripheral Peripheral
Interconnection Interconnection

133 Copyright ©2003 All rights reserved

 Power Model
• Defines the power specification of the VC
• Should be capable of representing both dynamic
power and static power
– Dynamic power may be due to capacitive loading or
short-circuit currents
IP Verification

– Static power may be due to state-dependent static

currents
• Required for all types of power analysis: average,
Tian-Sheuan Chang

peak, RMS, etc.

• Abstract level
– Black/gray box, RTL source code and cell level

134 Copyright ©2003 All rights reserved

 Basic Power Analysis
Requirements
• Any power analysis should include effects caused
by the following conditions and events:
– Switching activity on input ports, output ports, and
internal nodes
– State conditions on I/O ports and optionally internal
IP Verification

nodes
– Modes of operations
– Environmental conditions such as supply voltage and
external capacitive or resistive loading.
Tian-Sheuan Chang

135 Copyright ©2003 All rights reserved

 Physical Modeling
• Physical block implementation of hard, soft and firm VCs.
• Two models for hard VCs
– Detailed model
• Description of the physical implementation of the VC at the polygon
level
• The preferred data format is GDSII 6.0.0
IP Verification

– Abstract model
• Contains enough information to enable floorplanning, placement, and
routing of the system level chip
– Footprint
– Interface pin/port list, shape(s), and usage
Tian-Sheuan Chang

– Routing obstructions within the VC

– Power and ground connections
– Signature
• The preferred data format is the MACRO section of VC LEF 5.1

136 Copyright ©2003 All rights reserved

 Outline
• Verification challenges
• Verification process
• Verification tools
• RTL logic simulation
IP Verification

• RTL formal verification

• Verifiable RTL – good stuff
• Verifiable RTL – bad stuff
Tian-Sheuan Chang

• Testbench design
• SOC verification

137 Copyright ©2003 All rights reserved

 System Verification
• It begins during system specification.
• Develop system-level behavioral model.
• Successful System-Level Verification
– Quality of the test plan
– Quality and abstraction level of the models and
IP Verification

testbenches used
– Quality and performance of the verification tools
– Robustness of the individual predesigned blocks
Tian-Sheuan Chang

138 Copyright ©2003 All rights reserved

 IP Modeling
General Modeling Concept
Primary Model Classes: Specialized Model Classes: Computational Model Classes:

Behavioral Model Performance Model Data Flow Graph Model

Functional Model Interface Model Other Models

Structural Model Hybrid Model

IP Verification

System Models Architecture Models Hardware Models Software Models

Executable Specification Token-based Performance Model Detailed-Behavioral Model Pseudo-Code
Mathematical-Equation
Abstract-Behavioral Model Register Transfer Level (RTL) Model High Level Language (HLL)
Model
Algorithm Model Data Flow Graph (DFG) Task Primitive Logic-Level Model Assembly code

Instruction Set Architecture (ISA) Model Gate-Level Model Micro-Code

Tian-Sheuan Chang

Switch-Level Model Object Code

Circuit-Level Model

Precision Axis
Temporal Precision Axis
Data Precision Axis
Functional Precision Axis
Structural Precision Axis
Software Programming Precision Axis
139 Copyright ©2003 All rights reserved
 SOC Verification
• System
– Validate through
• Prototype, real chip or FPGA
– Methodology
• High level model execution
• Hardware/software co-simulation
IP Verification

• Prototype or run software on sample chip

– Rapid prototyping is necessary for verification
• since RTL or gate-level simulation is the bottleneck when
developing a derivative design
Tian-Sheuan Chang

– The most appropriated rapid-prototyping device for

platform design consists of
• A hardwire hardware kernel (real chip)
• Slots of FPGA on the hardware kernel’s bus for configurations

140 Copyright ©2003 All rights reserved

 The Test Plan
• System-level verification strategy uses divide-and-
conquer approach based on the system hierarchy.
– Verify the leaf nodes.
– Verify the interfaces between blocks that are
functionally correct.
IP Verification

– Run a set of increasingly complex applications on the

full chips.
– Prototype the full chip and run a full set of application
software for final verification.
Tian-Sheuan Chang

– Decide when it is appropriate to release the chip to

production.

141 Copyright ©2003 All rights reserved

 Interface Verification
• Interface: address/data bus. Protocols
– permitted sequence of control and data signals
– use a bus transaction monitor to check the transaction
Block 1 Block 2
Bus
Block 1 RTL Block 2 RTL Transaction
IP Verification

Interface Interface Monitor

Tian-Sheuan Chang

Block 3 RTL Block 4 RTL

Interface Interface
Block 3 Block 4

• Use BFM to check the data read and write

142 Copyright ©2003 All rights reserved
 System Verification
using Interface Testing (1)
• Chip with Point-to-Point Interfaces

Block1 Block1 Block2 Block2

RTL Block1 RTL RTL Block2 RTL
Interface Interface Interface Interface
IP Verification

Block3 Block3 Block4 Block4

RTL Block3 RTL RTL Block4 RTL
Interface Interface Interface Interface
Tian-Sheuan Chang

Simple
Transaction
Checking

143 Copyright ©2003 All rights reserved

 System Verification
using Interface Testing (2)
• Chip with an On-Chip-Bus

Block 1 Block 2 Bus

Transaction
Block 1 RTL Block 2 RTL Monitor
IP Verification

Interface Interface

Bus
Tian-Sheuan Chang

Block 3 RTL Block 4 RTL

Interface Interface Insert
Functional Checkers
Models Block 3 Block 4

(BFMs)

144 Copyright ©2003 All rights reserved

 Functional Verification (1/2)
• Two basic approaches
– increase level of abstraction so that software simulators
running on workstations faster
– use specialized hardware for performing verification,
such as emulator or rapid prototyping
IP Verification

• Canonical SoC abstraction

– Full RTL model for IP cores
– behavior or ISA model for memory and processor
Tian-Sheuan Chang

– bus functional model and monitor to generate and

check the transactions between IPs
– generate real application code for the processor and
run it on the simulation model
145 Copyright ©2003 All rights reserved
 Functional Verification (2)
• A canonical SOC design
Peripherals

Memory
Processor Memory
IP Verification

Controller

System Bus
Tian-Sheuan Chang

I/O Data I/O

Interface Transformation Interface

146 Copyright ©2003 All rights reserved

 Functional Verification (3)

Monitor ISA Application Software/ Behavioral

Compiler
(RTL) drivers/RTOS Model

Processor Memory Controller Memory

C/C++ C/C++ C/C++ CHIP
RTL Interface RTL Interface
Other
IP Verification

Peripheral
(RTL)
Tian-Sheuan Chang

I/O Interface Data I/O Interface

(RTL) Transformation (RTL)
(RTL)

Communication Sequence Communication

Bus Functional Generator / Bus Functional
Model (RTL) Analyzer Model (RTL)

147
 Application-Based Verification
• Run actual applications on the system (a full
functional model).
– Major Challenge
• RTL Simulation is the bottleneck.
– Two approaches to address this problem
IP Verification

• Increase the level of abstraction of design.

• Use specialized hardware for performance verification
– Emulation
– Rapid Prototyping
Tian-Sheuan Chang

148 Copyright ©2003 All rights reserved

 Gate-Level Verification
• Correct functionality and timing
• Sign-Off Simulation
• Formal Verification
• Gate-Level simulation with Unit-Delay Timing
IP Verification

• Gate Level Simulation with Full Timing

Tian-Sheuan Chang

149 Copyright ©2003 All rights reserved

 Sign-Off Simulation
• Gate-level simulation, parallel test vectors, full scan
methodology
• RTL sign-off problems
– Simulation speed is too slow
– Parallel vectors with very low fault coverage
– Parallel vectors do not exercise all the critical timing paths
IP Verification

• Traditional addressed problems

– Verification that synthesis has generated a correct netlist
– Verification that the chip, when fabricated, will meet timing
– A manufacturing test
Tian-Sheuan Chang

• Different Approaches
– Formal Verification
– Static Timing Analysis
– Some Gate-level Simulation
– Full Scan plus BIST
150 Copyright ©2003 All rights reserved
 Rapid Prototyping
• FPGA prototyping
– Aptix (FPGAs + programmable routing chips
• Emulation-based testing
– FPGA-based or processor-based
– QuickTurn and Mentor Graphics
IP Verification

• Real silicon prototyping

– faster and easier to build an actual chip and debug it
– design features in the real silicon chip
Tian-Sheuan Chang

• good debug structure

• ability to selectively reset the individual IP blocks
• ability to selectively disable various IP blocks to prevent bugs
from affecting operations of the system

151 Copyright ©2003 All rights reserved

 Specialized Hardware
for System Verification
• System simulation through specialized hardware
systems for verification.
– Zycad, IKOS
• These accelerators map the standard, event-driven software
simulation algorithm onto specialized hardware.
• Parallel execution on multiple processors.
IP Verification

– Emulation Systems
• Non-synthesizable code, especially testbenches, must run the
host machine.
Tian-Sheuan Chang

• The partitioning of the circuits among numerous FPGAs.

• The use of FPGA makes controlling and observing individual
nodes in the circuit difficult.

152 Copyright ©2003 All rights reserved