IP Core Design

Lecture 9 IP Verification

Juinn-Dar Huang, Ph.D.

Assistant Professor
[email protected]

October 2004
 A USD$ 475 Million Bug
• Intel’s Pentium Bug in 1994
– 3.3 million transistors on chip
– 1 trillion (1012) cycles were simulated before tape-out
Juinn-Dar Huang

– a bug in floating-point division hardware was revealed

– fixed by adding 5 more transistors
• Intel offered replacements to users
– $475 Million USD penalty at Q4’94
[email protected]
IP Verification

copyright © 2004 1
 Verification Challenge (1/3)
• Designs grow as Moore’s Law predicted
• Design productivity has risen 10x since 1990 due
to synthesis
Juinn-Dar Huang

• Effort for verification doubles every 6-9 months

Hardware Design Productivity

160
140
[email protected]

120

Gates Per Day

The challenge of 100
80

functional verification
IP Verification

60
40

is getting bigger 20
0

80

82

84

86

88

90

92

94

96

98
19

19

19

19

19

19

19

19

19

19
Year

copyright © 2004 2
 Verification Challenge (2/3)
• 50% of chips require 1 or more re-spins
• 74% of re-spins are due to functional defects
Juinn-Dar Huang

Reasons of Re-Spins

Firmware 10%

IR Drop 11%

Power 14%

Mixed-Signal I/F 21%

Reason
Yield 23%
[email protected]

Race Condition 24%

Slow Path 31%

IP Verification

Clocking 31%

Noise 33%

Functional 74%

0% 20% 40% 60% 80%

%

Collett International 2000

copyright © 2004 3
 Verification Challenge (3/3)
• Average 1.5 functional bugs per 200 lines in RTL
– 150 bugs in a typical 20K-line design
• Average 1.5 man-day required to find a bug
Juinn-Dar Huang

– 225 man-day required for a 20K-line design

• Do you find all bugs before tape-out?

[email protected]

• How do you find bugs quicker and easier?

IP Verification

copyright © 2004 4
 Glossary
• Verification environment
– commonly referred as testbench (environment)
• Definition of testbench
Juinn-Dar Huang

– a verification environment containing a set of

components - such as bus functional models (BFMs),
bus monitors, memory models - and the interconnect of
such components with the design-under-test
[email protected]

• Verification (Test) suites (stimuli, patterns, vectors)

– test signals and the expected response under given
IP Verification

testbenches

copyright © 2004 5
 Verification Reuse
• Verification goal is defect-free
• The macro-level testbenches and suites should
be reusable
Juinn-Dar Huang

– the macro may be redesigned later

• The integration team must be able to reuse the
macro-level testbenches
[email protected]

– the macro must be verified both as a standalone unit

and in the context of the final application
IP Verification

• Testbenches must be compatible with system-

level verification tools
– testbenches may be used in system verification

copyright © 2004 6
 Verification Plan
• Verification plan is part of the design report
• Verification takes over 70% of development time
Juinn-Dar Huang

• Contents
– verification strategy for both subblock- and top-level
– testbench components - BFM, bus monitors, …
– required verification tools
[email protected]

– simulation environment including block diagrams

– key features needed to be verified in both levels
IP Verification

– regression test environment and procedure

– clear criteria to determine whether the verification is
successfully complete

copyright © 2004 7
 Benefits of Verification Plan
• Verification plan enables
– developing the testbench environment early
– developing the test suites early
Juinn-Dar Huang

– focusing the verification effort to meet the shipment

criteria
– forcing designers to think through the time-consuming
activities before performing them
[email protected]

– a separate verification support team creates a

verification environment in parallel with the primary
IP Verification

design task

copyright © 2004 8
 Verification Strategy
• Three major phases
– subblock verification
• thorough and exhaustive functionality verification
Juinn-Dar Huang

• simulation, code coverage, TB automation

– macro verification
• interface verification between subblocks
• simulation, hardware accelerator, TB automation, code coverage
[email protected]

– prototyping
• real prototype runs real software in the real application
IP Verification

• emulation, FPGA, ASIC test chip

• Bottom-up approach
– locality
– catching bugs is easier and faster in the lower level
copyright © 2004 9
 Types of Verification
• Compliance verification
• Corner case verification
• Assertion-based verification (ABV)
Juinn-Dar Huang

– either static(formal) or dynamic

• Random verification
– create scenarios that engineers do not anticipate
• Real code verification
[email protected]

– avoid misunderstanding the spec

IP Verification

• Regression verification
– ensure that fixing a bug will not introduce another bugs
– be performed on a regular basis
copyright © 2004 10
 Assertion
• An assertion
– is a statement about a design’s intended behavior
– is also named as property; used interchangeably
Juinn-Dar Huang

– usually does not map to the real HW (not synthesizable)

• An assertion is used
– to ensure consistency between the designer’s intention
and what he created
[email protected]

– to guard the design assumptions/properties

IP Verification

“Input should range from 0 to 240.”;

“After req raises, gnt is expected within 10 clock cycles.”

copyright © 2004 11
 Neither New nor Innovative Idea
• The assertion concept has been adopted to
software engineering for many many years
Juinn-Dar Huang

#include <assert.h>

char hex_to_char(unsigned int num)

{
assert(num <= 15);
[email protected]

if(num < 10)

return num + ‘0’;
else
IP Verification

return num - 10 + ‘A’;

}

“Writing Solid Code” or “如何撰寫 0 錯誤程式”

copyright © 2004 12
 Assertion-Based Verification
• Assertion
– to preserve design assumptions and properties
– break the simulation when assertion fails
Juinn-Dar Huang

– both spatial and temporal relationship can be asserted

– help designers to locate bugs at right place and time
• Open Verification Library (OVL)
– www.verificationlib.org
[email protected]

• Property Specification Language (PSL)

IP Verification

– derived from IBM’s Sugar, adopted by Accellera 2003

• Concept extended to function monitors and
functional coverage
copyright © 2004 13
 ABV Flow
• More thorough block-level
verification
– find bug easier Custom Existing Verification
– simulate effectively Assertions Libraries IPs
Juinn-Dar Huang

• Build SoC from more

robust IPs Design-Specific
– assertions can be reused Assertion Specifications
at system-level
– system-level verification Simulation
[email protected]

focuses on interfaces
among IPs Formal
IP Verification

TB Automation

Functional Coverage

copyright © 2004 14
 Taxonomy
Functional Verification

Dynamic Formal/Static
Juinn-Dar Huang

- simulation based - mathematical-proof

STA in timing
- input vectors required - no input vectors required
- no 100% guarantee - 100% guarantee verification
(false positive issue)

Model Checking Equivalence Checking

[email protected]

Semi-Formal - verify properties - pretty mature

- still simulation based formally
- check properties formally - capacity issue
IP Verification

at each simulation step - improvement

expected

copyright © 2004 15
 Verification Tools (1/2)
• Simulation
– even-driven: good debug environment
– cycle-based: fast simulation time
Juinn-Dar Huang

• Code coverage
– coverage on RTL structure
– Verification Navigator, CoverMeter
• Hardware Verification Language (HVL)
[email protected]

– a language providing powerful constructs for generating

IP Verification

stimulus and checking response

– OpenVera in Vera, e in Specman Elite, SystemVerilog,
(SCV in SystemC, former TestBuilder)

copyright © 2004 16
 Verification Tools (2/2)
• Functional coverage
– coverage on functionality
• Formal property checking
Juinn-Dar Huang

• Verification IPs (VIPs)

– BFMs and bus monitors for standard protocols
• Hardware modeling
[email protected]

• Hardware acceleration and emulation

• Prototyping
IP Verification

– FPGA
– test chip in silicon

copyright © 2004 17
 Verification IPs
• A package including well-designed and well-
verified BFM/monitor for a specific (interface)
protocol
Juinn-Dar Huang

– AMBA, Ethernet, SONET, UTOPIA, PCI, USB, I2C,

UART, CAN, …
– avoid reinvent-the-wheel
– accelerate the verification
[email protected]
IP Verification

copyright © 2004 18
 Code Coverage (1/2)
• Indicate how much of design has been exercised
• Point out what areas need additional verification
• Optimize regression suite runs
Juinn-Dar Huang

• Minimizes the use of valuable simulation resources

• Quantitative stopping criterion
• Verify more but simulate less
[email protected]
IP Verification

copyright © 2004 19
 Code Coverage (2/2)
• Types
– statement – branch
– condition – path
Juinn-Dar Huang

– toggle – triggering
– FSM

– 100% coverage in statement, branch, condition

[email protected]

– others can be used as secondary metrics

IP Verification

• Redundancy removal
• Minimize regression test suites
copyright © 2004 20
 Verification with Code Coverage
Juinn-Dar Huang
[email protected]
IP Verification

copyright © 2004 " Source: Verification Methodology Manual 21

 Bug Detection Curve
Juinn-Dar Huang
[email protected]
IP Verification

copyright © 2004 " Source: Verification Methodology Manual 22

 Inspection
• Finding bugs by careful inspection is faster than
that by simulation
• Inspection process
Juinn-Dar Huang

– design (specification, architecture) review

– code (implementation) review
• line-by-line fashion
• at the subblock-level
[email protected]

• Lint-like tool can help spot defects w/o simulation

IP Verification

copyright © 2004 23
 Adversarial Testing
• For original designers
– focus on proving that the design works correctly
• Separate verification team
Juinn-Dar Huang

– focus on trying to prove the design is broken

– keep up with the latest tools and methodologies
• The combination of the two gives the best results
[email protected]
IP Verification

copyright © 2004 24
 Limited Production
• Even after robust verification and prototyping, it’s
still not guaranteed to be bug-free
• A limited production for new macro is necessary
Juinn-Dar Huang

– 1 to 4 customers
– small volume
– reducing the risk of supporting problems
[email protected]
IP Verification

copyright © 2004 25
 Testbench Design
• Auto or semi-auto stimulus generation is preferred
• Automatic response checking is a must
• Powerful pattern generators and function monitors help
Juinn-Dar Huang

• Macro-level testbench
– will be shipped along with the macro so that the customer can
verify the macro in the system design (reusable testbench)

Input Log,
Commands Response Error Report,
[email protected]

at High-Level Pattern Checker, Functional

DUV
Abstraction Generator Function Coverage
Monitor Analysis
IP Verification

User PLI SV
Transactions
TestBuilder ACT
e/Specman 0-in
Vera …
copyright © 2004 26
 Example: USB Device Controller
AHB AHB AHB
correctness
Master Slave Bus transaction logging
BFM BFM Monitor coverage
AHB
Juinn-Dar Huang

correctness Slave Master

transaction logging
coverage USB Macro
[email protected]

UTMI I/F
USB
Monitor
IP Verification

USB Bus

Automatic
User USB Response
Commands BFM Checking

copyright © 2004 27
 Automatic Response Checking

Device Under
Verification
Juinn-Dar Huang

Automatic
Input
Response
Stimulus
Checking
Reference
Model
[email protected]

(C/C++,
HDL/HVL,
Hardware Modeler)
IP Verification

Typical use: datapath elements and processors

copyright © 2004 28
 Functional Models (1/2)
• Functional (Behavioral) model is required for all
IPs
– fast simulation and integration in system-level designs
Juinn-Dar Huang

– a very secure model for customer’s evaluation

• BFM model is required particularly for interface
IPs
– PCI, USB, IEEE1394, …
[email protected]

• ISA model is required for processor IPs

IP Verification

copyright © 2004 29
 Functional Models (2/2)
• Functional models of different abstractions
– provide tradeoffs between accuracy and simulation
speed
Juinn-Dar Huang

– meet various needs of the H/W and S/W teams in

various design phases
• Model security
– functional models usually developed by C/C++ or HDL
[email protected]

– protection should be applied if security is a concern

– compiled (encrypted) models are usually required
IP Verification

copyright © 2004 30
 Timing Verification
• STA is the fastest approach for timing verification
of synchronous designs
– avoiding any timing exceptions is extremely important
Juinn-Dar Huang

• Gate-level simulation
– most useful in verifying timing of asynchronous logic,
multi-cycle paths and false paths
– much slower run-time performance
[email protected]

– gate-level sign-off
IP Verification

– translate functional test patterns into tester patterns

copyright © 2004 31
 Case Study 1:
16-Bit Embedded DSP Core
 RTL Verification Environment

Assembly code
(*.dsp) Verification Environment
Coverage
Juinn-Dar Huang

measured
S/W Tools BDMA ISA by VN
I/F I/F
Execution file
(*.exe) BIST
PM PLL testing
(64Kx24)
[email protected]

DM Interrupt
IP Verification

Memory Image (64Kx16)

Core SPORT
files GPIO
(*.hexg) CM
(64Kx16) I/O

copyright © 2004 33
 Deterministic Verification
100+ deterministic verification suites

• Special functions
Juinn-Dar Huang

– MAC, ALU, SHIFTER, DAG, SPORTs, Timer and Interrupts

• Pipeline control
– data dependency, control dependency, and loop handling, …
• Common DSP algorithms
– FFT, IFFT, FIR, DEMEAN_BUFFER, …
[email protected]

• Real code
IP Verification

– DCT, PCM, modem, GSM codec, G.722, …

Completeness assured by code coverage tool

copyright © 2004 34
 Verification for External Interrupts
• One of the hard-to-verification cases
• Generating a complicated external interrupt
sequence is not easy
Juinn-Dar Huang

– scenarios such as nested interrupts, …

• Use an external interrupt generator
– self-timing
[email protected]

– highly configurable
– help create non- interrupts
IP Verification

trivial interrupt Interrupt

CPU Generator
sequences scenarios
(as an I/O device)

copyright © 2004 35
 Instruction Set Simulator (ISS)
• Implement in C
• Cycle-by-cycle accuracy
Juinn-Dar Huang

– all registers (internal registers, memory-mapped

register,…)
– important internal signals (program counter, stall
signals, pipeline control signals, …)
[email protected]

.. Hardware H.log
IP Verification

Verification Pattern Compare

ISS I.log

copyright © 2004 36
 FPGA Prototyping Board

SEGMENT
Juinn-Dar Huang

IO MEMORY
Unit
PF
AD1847 SPORTs Interface

BDMA
BDMA
DSP CORE MEMORY
[email protected]

Interface
Unit
ISA IDMA
Adatper Interface
IP Verification

BOOT ROM
Unit

PM DM CM

copyright © 2004 37
 Application for FPGA Prototyping
• Bass enhancement

AD1847
Juinn-Dar Huang

UDSP-FPGA
[email protected]
IP Verification

7-SEGMENT BOOT ROM

copyright © 2004 38
 ASIC Prototyping
• More difficult to debug
– lack of observability – limited external pins
– lack of controllability – free-run mode only
Juinn-Dar Huang
[email protected]
IP Verification

copyright © 2004 39
 40
MP3 Demo System

copyright © 2004
IP Verification
Juinn-Dar Huang [email protected]
 MP3 Player Demo System
Juinn-Dar Huang
[email protected]

CF Card
UDSP-
IP Verification

1600 Flash ROM

AC’97
Codec

copyright © 2004 41
 Development Kit
7-Seg. LEDs EPP Port
Juinn-Dar Huang

Connectors
[email protected]

Audio
I/F
Flash ROM
IP Verification

UDSP-1600

copyright © 2004 42
 Case Study 2:
32-Bit Embedded CPU Core
 Design Verification
• Rigorous RTL coding style
– assured by a linting tool
• Verifiable RTL coding style
Juinn-Dar Huang

– friendly to simulation, equivalence checker, …

• Deterministic verification
– basic feature verification
[email protected]

– corner case verification

– code coverage analysis
IP Verification

• Random code verification

• Sophisticated regression environment

copyright © 2004 44
 Deterministic Verification
• Data processing (ALU,shifter and multiplier)
• Memory access (load, store and swap)
• Branch and interrupt
Juinn-Dar Huang

• Pipeline data and control dependency

• Different CPU models
• Coprocessor interface
[email protected]

Completeness assured by code coverage tool

IP Verification

copyright © 2004 45
 Random Code Verification (1/2)

Massive constrained random patterns

Juinn-Dar Huang

Random seed Mismatch assertion

Golden
Model
Random Code Result
Generator Comparator
[email protected]

CPU RTL
IP Verification

Information dump
Constraint file

copyright © 2004 46
 Random Code Verification (2/2)
• Random code generator
– constrained random instructions generation
• instruction type
Juinn-Dar Huang

• ratio among types

– constrained random exception(interrupt) injection
• exception type
• probability for each exception
– capable to rebuild failure scenarios for debugging
[email protected]

• Result comparator
– automatically compare on the fly
IP Verification

– all output and register values are compared

– dump mismatch information for debugging
– Over 100 million cycles verified at least
copyright © 2004 47
Must-Do List for My Next CPU Project
 Must-Do List (1/2)
• Rigorous RTL coding and lint check
• Thorough verification plan in advance
Juinn-Dar Huang

• Deterministic verification
– predefined must-verify features
– corner-case verification
• Complete code coverage
[email protected]

• Constraint-driven random verification

– constraint-driven randomized stimulus generator
IP Verification

– use Vera or e
– functional coverage
• Untimed and Timed ISA models
copyright © 2004 49
 Must-Do List (2/2)
• Dynamic ABV
– use assertions intensively throughout the simulation
– use OVL and PSL
Juinn-Dar Huang

• Optional formal verification

– property checking (model checking)
– equivalence checking
• Optional HW accelerator
[email protected]

• FPGA prototyping
IP Verification

– choose an appropriate application

• ASIC-based demo system

copyright © 2004 50
 Untimed and Timed ISA Models (1/2)
• Untimed ISA model
– instruction-level accuracy
– no accurate timing information
Juinn-Dar Huang

– for behavioral simulation

• Timed ISA model
– cycle-accurate ISA model
– functionality + cycle timing information
[email protected]
IP Verification

copyright © 2004 51
 Untimed and Timed ISA Models (2/2)
• System C is a good candidate for above models
– native and trivial for system designers
– good encryption
Juinn-Dar Huang

– can co-simulate with HDL

– seamlessly integrate with GUI wrapper
• Timed model should be based on the untimed one
• ISS should be just a GUI wrapper + an ISA model
[email protected]
IP Verification

copyright © 2004 52
 RTL Verification Flow

Verification
Vectors
Juinn-Dar Huang

2 Simulation
Code Coverage
Functional Coverage
Dynamic ABV, … 3 Property
Checking

4 Synthesis Gate-Level
[email protected]

RTL Netlist
Equivalence
IP Verification

1 Linting 5 Checking

Rules
Database

copyright © 2004 53