Evolution of

Digital Verification

Walter Gude
Applications Engineering Consultant
[email protected]
Presented to the IEEE Long Island Section

Signal Processing Society & Photonics Society

on Tuesday October 8th, 2013

Agenda

Verification Overview

Assertion and Functional Coverage

Constrained Random
Requirements Tracing
Algorithmic TB InFact
Questa Formal
Questa Codelink
Questa VIP

Mentor Graphics Corp.

www.mentor.com

The Evolution of Digital Design & Verification

Gates
Whats
Next?

10M
Static Timing &
Equivalence Checking

1M

HDL Design &

Simulation

100K
Schematic Capture
& Logic Simulation

10K
Transistor-Level
Design
1960s

1970s

1980s

1990s

2000s

Mentor Graphics Corp.

www.mentor.com

Time

Creating HUGE Verification Challenges

E
Verification
Gap

Design
Gap

Ability to Fabricate
Dominated by feature size

Ability to Design
Dominated by interconnect

Ability to Verify
Dominated by behavior over
time and complexity of
concurrency

Mentor Graphics Corp.

Your Initials, Presentation Title, Month Year

www.mentor.com

Traditional Directed Test Flow.

Integrated Design and Verification Engineer

Integrated
test and
testbench

RTL

Pass
Pass
Fail at
Pass
fasil

Waveforms
Log files

Mainly directed tests

Ad hoc test planning

Code Coverage tools for advanced users
Manually inspected output files and waveforms

Problem: Creation of directed tests does NOT scale with increased complexity

HDL and some C/C++

Number of tests directly linked to number of engineers writing tests

Amount of testbench code becomes difficult to manage
NEED better way to create stimulus!

Problem: Code Coverage does not measure how well specifications are covered.
5
Mentor Graphics Corp.

WG - Feb 2013

www.mentor.com

When Is Verification Complete?

100%
90%
80%

Percent complete

70%
60%
50%
40%
30%
20%
10%
0%
time

Mentor Graphics Corp.

www.mentor.com

Advanced Verification
Functional
Specification
Design
Implementation

Verification Plan
Constraint
Solver

Simulation

Testbench
C-R Testbench
Implementation
N
Y

Assertion
Engine

Functional
Coverage

Sufficient
Coverage?

Done
Coverage-Driven
Verification

Debug

Bug
Found?

Y
Assertion Based
Verification

Mentor Graphics Corp.

www.mentor.com

SystemVerilog

Verilog 2001 base

Massive enhancements
Assertions
Functional
Coverage

Verilog
2001

Object
Oriented
Programming

Constrained
Randomization
Language
Enhancements
8
Mentor Graphics Corp.

www.mentor.com

The Verification Process

Functional
Specification
Design
Implementation

Verification Plan
Constraint
Solver

Simulation
ModelSim
Kernal

C-R Testbench
N
Y

Assertion
Engine

100%
Standards
based

Functional
Coverage

Sufficient
Coverage?

Done

Debug

Coverage-Driven Verification
Testbench Automation
Functional Coverage

Bug
Found?

Y
Assertion Based
Verification

Mentor Graphics Corp.

www.mentor.com

Questa Verification Platform

Best In Class Engines

Questa Sim
Questa inFact

Questa Formal

Questa PA

Questa CDC

Questa VIP

Functional

Static

Verification

Verification

Verification
Management

Verification
Methodology

Dynamic

HW/ SW
UVM
UVM Connect
UVM Express

Questa Codelink

Questa Verification
Management

Mentor Graphics Corp.

11

www.mentor.com

Questa Verification Platform

Best In Class Engines - Unified Front End Analysis & Compile

Questa Sim
Questa inFact

Questa Formal

Questa PA

Questa CDC

Questa VIP

Functional

Static

Verification

Verification

Verification
Management

Verification
Methodology

Dynamic

HW/ SW
UVM

Questa Codelink

UVM Connect
UVM Express

Questa
Verification
Management

Mentor Graphics Corp.

12

WG - Feb 2013

www.mentor.com

Questa Verification Platform

Comprehensive integrated SOC

verification platform
Best in class engines
Integrated
Comprehensive debug analysis

Industry Leading SOC Verification Solutions

Coverage Closure Solution
Low Power Verification Solution
Software Driven Verification Solution

Standards Leadership

Driving the evolution of IEEE

standards
Major donations to Accellera UVM
Accellera UCIS from Mentor UCDB
Mentor Graphics Corp.

13

WG - Feb 2013

www.mentor.com

What is an Assertion?
A concise description of [un]desired behavior

req
ack
Example intended behavior

After the request signal is asserted, the

acknowledge signal must come 1 to 3 cycles later

Mentor Graphics Corp.

14

www.mentor.com

SV Assertions
SV Assertion

property req_ack(req,ack);
@(posedge clk) $rose(req) |-> ##[1:3] $rose(ack);
endproperty
as_req_ack: assert property (req_ack(req1,ack1));
sample_inputs : process (clk)
always @(posedge req)
begin
ifbegin
rising_edge(clk) then
STROBE_REQ <= REQ;
repeat
(1) @(posedge clk);
STROBE_ACK
<= ACK;
end if;
fork: pos_pos
end process;
begin
protocol: process
variable CYCLE_CNT : Natural;
@(posedge ack)
begin
loop
$display("Assertion Success",$time);
wait until rising_edge(CLK);
disable
pos_pos;
exit when (STROBE_REQ
= '0')
and (REQ = '1');
end loop; end
CYCLE_CNT := 0;
begin
loop
wait until rising_edge(CLK);
repeat (2) @(posedge clk);
CYCLE_CNT := CYCLE_CNT + 1;
$display("Assertion
Failure",$time);
exit when ((STROBE_ACK
= '0') and (ACK = '1'))
or (CYCLE_CNT = 3);
end loop;
disable pos_pos;
if ((STROBE_ACK = '0') and (ACK = '1')) then
end
report "Assertion
success" severity Note;
else
join
report "Assertion failure" severity Error;
end
if; // always
end
end process protocol;

Verilog
VHDL

req

ack
Example intended behavior

HDL Assertion

Mentor Graphics Corp.

15

www.mentor.com

Assertions Need to be Everywhere

Assertions Enable Higher Quality Designs

Assertions provide
observability for
higher complexity
designs

Assertions describe
(un)desired behavior

Assertions
dramatically shorten
debug and repair
time

16

Reference Model

Bus
Monitor

Assertion
Checkers

Bus
Monitor

Assertions stay on
during block, chip
and system-level
tests

Finds bugs you

werent looking for

Mentor Graphics Corp.

www.mentor.com

Assertion
Checkers

Payoff Is High: Assertions Find Bugs

Assertion Monitors

34%

Cache Coherency Checkers

Register File Trace Compares
Memory State Compare
End-of-Run State Compare
PC Trace Compare
Self-Checking Test
Simulation Output Inspection
Simulation Hang
Other

9%
8%
7%
6%
4%
11%
7%
6%
8%

Kantrowitz and Noack [DAC 1996]

Assertion Monitors

25%

Register Miscompares
Simulation No Progress
PC Miscompare
Memory State Miscompare
Manual Inspection
Self Checking Test
Cache Coherency Check
SAVES Check

22%
15%
14%
8%
6%
5%
3%
2%

34% of all bugs found were identified by assertions on DEC

Alpha 21164 project
[Kantrowitz and Noack DAC 1996]

17% of all bugs found were identified by assertions on

Cyrix M3(p1) Project
[Kronik 98]
25% of all bugs found were identified by assertions on DEC
Alpha 21264 project
[Taylor et al. DAC 1998]

50% of all bugs were identified by assertions on

Cyrix M3(p2) Project
[Kronik 98]
85% of all bugs were found using over 4000 assertions
on HP Project
[Foster and Coelno HDLCon 2000]

10,000 assertions in Cisco project

[Sean Smith 2002]

Taylor et al. [DAC 1998]

Mentor Graphics Corp.

17

www.mentor.com

Formal
EnglishSpecification
Specification
Specification
Spec1: ----------------------------------------------------------------------------------------------------------------------

Weaknesses of natural language

specifications
Ambiguity

Including the ease of misinterpreting

Completeness

Spec2: ---------------------------------------------------------------------------------------------------------------------

Cannot be executed to verify

Completeness
Accuracy

:
:

SystemVerilog/PSL are a specification

languages
Unambiguous
Precise semantics

No emotional (connotation) baggage

Can be used to create executable

specification

Coverage of properties (functionality, tests)

Accuracy of specification

Mentor Graphics Corp.

18

www.mentor.com

Total Coverage Model

Specification

Functional
(specification-based)

Checks that all functions of

the design are tested
Created by verification team

Structural
(implementation-based)

Checks that all corner-cases

of the design are tested
Created by designers
Implementation

Mentor Graphics Corp.

19

www.mentor.com

SV Coverage Models
// States for a DRAM controller
enum {IDLE, MEM_ACC, SWITCH, RAS_CAS, OP_ACK, REF1, REF2} fsm_state;
covergroup dram_ctrl_fsm_states @(posedge clk);
// An implicit cover bin for all 7 values of fsm_state
covergroup dram_ctrl_fsm_transitions @(posedge clk);
c1: coverpoint fsm_state;
c1: coverpoint fsm_state {
endgroup
bins idle_bin = (IDLE => {REF1, MEM_ACC}) iff (!rst);
bins ref1_bin = (REF1 => REF2 => IDLE) iff (!rst);
// RAS_CAS can last one or two cycles depending on if
// memory access is a read or a write.
bins mem_acc_bin = (MEM_ACC => SWITCH =>
RAS_CAS[*1:2] => OP_ACK => IDLE)
iff (!rst);
bins rst2idle_bin = ({IDLE:REF2} => IDLE) iff (rst);
bins erroneous = default; // Catch undefined transitions
}
endgroup
Mentor Graphics Corp.

20

www.mentor.com

When is verification complete?

Verification is effectively
metric-less
Few designers know if
their strategy is
adequate or efficient
Sign-off criteria are ad
hoc and vary by
company
Code coverage is not a
functional verification
metric

If it isnt verified, its

broken
Mentor Graphics Corp.

21

www.mentor.com

When Is Verification Complete?

100%
90%
80%

Percent complete

70%
60%
50%
40%
30%
20%
10%
0%
time

Mentor Graphics Corp.

22

www.mentor.com

The Verification Process

Functional
Specification
Design
Implementation

Verification Plan
Simulation

C-R Testbench
N
Y

Sufficient
Coverage?

Done
Coverage-Driven
Verification

Debug

Bug
Found?

Y
Assertion Based
Verification

Mentor Graphics Corp.

23

www.mentor.com

Directed Tests
Specification
Spec1:------------------------------------------------------------------------Spec1:
-------------------------------------------------------------------------------------------------------------------------------------------------------------------

Design

Spec2:------------------------------------------------------------------------Spec2:
----------------------------------------------------------------------------------------------------------------------------------------------------------------Spec3:------------------------------------------------------------------------Spec3:
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:
:
Spec10,000: ---------------------------------------------------------------------------------------------------------------------------

Mentor Graphics Corp.

24

www.mentor.com

Functional Coverage and

Contrained Random Testing

In a directed test, the

coverage points are
coded in the test itself
Test writer must
code each specific
scenario to specify
intent explicitly
Must be able to
predict interesting
cases in order to
code them
Doesnt scale well

Test

Randomness inherent in CR allows unpredicted

scenarios to be exercised
Exposes corner cases the
designer may not have
considered
Testbench is an objective
description of intent against
which to check the design
Kind of like an independent
verification team

Functional Coverage tells

which features were
exercised

CR
Test

Mentor Graphics Corp.

25

www.mentor.com

SV-OOP vs. Verilog

Definition

class ether_packet extends packet;

src

dest

// Ethernet
bit[47:0]
bit[15:0]
bit [7:0]

len

payld[ ]

methods

Transmit
Checksum
Display

Clone
Verify
Build

Copy
Debug

In Verilog, a module is the only container that can

hold both the definition of the packet fields and all
the methods needed to manipulate those fields.
In SV, classes are infinitely better suited, because:
Objects are dynamic, not static
Create or destroy objects at will
Classes are inheritable, polymorphic etc.

Packet Fields
dest, src;
len;
payld [ ];

// onboard methods
function new(int i);
payld = new[i]; len = i;
endfunction : new
function void display;
$displayh("\t
src: ", src);
. . .
foreach (payld[i])
$display(payld[i] = %d",i,payld[i]);
endfunction : display
task transmit_frame();
. . .
function clone();
. . .
endclass : ether_packet
26
Mentor Graphics Corp.

26

www.mentor.com

Tools and Technology Arent Enough

EDA sells tools and materials

To build a bookcase, you need to

apply the tools and technologies
in useful ways

What we need are Advanced

Methodologies
27
Mentor Graphics Corp.

27

www.mentor.com

Unified Verification Methodology

UVM - Open-source framework

for reusable verification based
on SystemVerilog

SystemVerilog Class Library

Set of core building blocks for

effective, reusable verification
environments
Library of common services

UVM
VIP and Verification
Environment
Documentation,
(examples, code snippets
Class Library

SystemVerilog Language

Interoperability and reuse

IEEE 1800 Compliant

Simulator

Mentor Graphics Corp.

28

www.mentor.com

UVM-based Testbench Architecture

M
Testbench

Design Under Test

D
D

Transactions

Drivers - translate
transaction into signals

Monitors - translate signals

into transactions

Signals

Sequencers - generates
sequences of transactions

Analyzers - analyzes sequences

of transactions

Mentor Graphics Corp.

29

www.mentor.com

Coverage Data Overload

Testplan
VIP reduces
testbench
development time
Increases verification
efficiency

VIP
Verification Engineer

Project Manager
VIP
Coverage

T1
T2
...
T(n)

System

Testbench

Functional
Assertion

Coverage Files
Functional coverage
enables intelligent
verification
Increases quality

Constrained Random
doubles productivity of
verification engineers
Greatly reduces the need
for directed tests

Assertion
Library

TLM components enables

faster verification
performance
Enables re-use
Assertions improve
debug time by 50%
Find hard bugs
Increases quality

Code/FSM

Run Data

Giga/Tera Bytes of information

Thousands of tests data explosion lead to lack of process visibility

Mentor Graphics Corp.

30

Formal
Verification

www.mentor.com

30

Mentors Unique Advantage

Test
Functional Assertion
Code
Engine
Coverage Coverage

0-in
Formal

User
Coverage

Test
Specific

Plan

Native UCDB Generation

Test Associated Merging

HTML Export

Open Read And Write

API, C Functions & CLI

RD / WR API

UCDB Browser

RD / WR API

Test Ranking

Unified
Coverage
Database

Test Plan Tracking & Analysis

Mentor Graphics Corp.

31

www.mentor.com

Managing Verification Through Metrics

Questa Coverage from all engines
Veloce Code coverage
0-In Functional coverage
3rd Party Assertion coverage
User Defined metrics
Test Merging
Verification Plan Import
(XML)

UCDB

Verification plan import

Rank/Manage
results

Excel, Word, DocBook

Unique Coverage Labels
Track
coverage of
Types, Weights,
Goals

specific plan / tests

Mentor Graphics Corp.

32

www.mentor.com

When Is Verification Complete?

100%
90%
80%

Percent complete

70%
60%
50%
40%
30%
20%
10%
0%
time

Mentor Graphics Corp.

33

www.mentor.com

Agenda

Verification Overview

Assertion and Functional Coverage

Constrained Random
Requirements Tracing
Algorithmic TB InFact
Questa Formal
Questa Codelink
Questa VIP

Mentor Graphics Corp.

34

www.mentor.com

The ReqTracer Solution

INFRASTRUCTURE

INTEGRATION

Company
Standards

Requirements
Management

TRACEABILITY

DOCUMENT
XML
ASCII

Dynamic Tracing

Automated
Documentation

Mentor Graphics Corp.

35

www.mentor.com

ReqTracer in your Process

Functionality & Verification Results Meet Requirements

Capture

Tag

Requirements from many interfaces &

formats
Add documents to project

XML

Link relationships & dependencies

Dynamic linking

REQ_001
REQ_002

ASCII

REQ_003

Trace
Requirements design

REQ_004
REQ_005

implementation verification

Monitor/Analyze

Validate

Impact analysis on ECOs

Completeness of requirements
Extraneous code

REQ_001
REQ_002

Status of requirements against test plan

REQ_003
REQ_004

Report

REQ_005

Automated report generation

Simplifies design reviews/audits
Wide variety of formats

XML
ASCII

Mentor Graphics Corp.

36

www.mentor.com

Capture

Projects

Add documents to project

Associate documents define coverage relationships
Design covers requirements
Verification results verify test plan

Bugzilla

User
Requirements

Acceptance Tests

Design
Spec

Integration Tests

Design RTL

Testbench RTL
Verification
Results

Mentor Graphics Corp.

37

www.mentor.com

Tag

Configure ReqTracer to
capture existing
requirements

Tag new requirements

REQ_001
REQ_002
REQ_003
REQ_004
REQ_005
Design
Specification

Mentor Graphics Corp.

38

www.mentor.com

Trace
COVERS

COVERS
Hardware

Design

Requirements

Specification

40%

Can you prove all

requirements are
implemented and
tested?

Certification
Authority

RTL

60%

Design

What shall I
work on
next?

Hardware
Designers

What is this
code for?

System
Architect

Mentor Graphics Corp.

39

www.mentor.com

Monitor/Analyze
IMPACT
CHANGE

COVERS

IMPACT
IMPACT
IMPACT
IMPACT

COVERS
Hardware

Design

Requirements

Specification

20%

40%

Design

Project
Manager

Quality
Manager

Mentor Graphics Corp.

40

60%

Which tests
need updating
now the design
has changed?

Can we make
a change and
still release
on time?

How risky
would it be if I
changed this?

System
Architect

RTL

www.mentor.com

Validate

Ensure links
Requirements RTL
source
Requirements test plan
Test plan testbench
Test plan verification
results

View Coverage of the

tests
Determine Pass or Fail

Does verification test result

satisfies the test plan?

Mentor Graphics Corp.

41

www.mentor.com

Summary
Requirements tracing is best as an active
process throughout the project
development

XML

REQ_001
REQ_002

ReqTracer

ASCII

REQ_003

Aggregates requirements from multiple

sources
Traces requirements through the HW design
process
Provides direct ECO impact analysis
Automates report generation
Interfaces with HDL Designer

REQ_004
REQ_005

REQ_001
REQ_002
REQ_003

HDL code, version management &

documentation

REQ_004
REQ_005

Interfaces with Questa

Validation of test plan

XML

Satisfies DO-254 needs for certification

ASCII

Mentor Graphics Corp.

42

www.mentor.com

Agenda

Verification Overview

Assertion and Functional Coverage

Constrained Random
Requirements Tracing
Algorithmic TB (InFact)
Questa Formal
Questa Codelink
Questa VIP

Rule Checking

Precision

Mentor Graphics Corp.

43

www.mentor.com

Three Generations of Functional Verification

Testbench Evolution

Directed Tests

Quality - Engineer Directed

Quantity - Engineer Limited

Good - But Not Scalable

10X

10X

Higher
Coverage

Faster
Coverage

Constrained Random Tests

Quantity - Time & CPU Limited

Quality - Redundant

Better - But Not Sufficient

Intelligent Tests

Quantity - High

Quality - High

Best - Do More With Less

Directed Tests
1,000s

Random Tests
1,000,000s

Mentor Graphics Corp.

44

www.mentor.com

Intelligent Tests
100,000s

Intelligent Testbench Automation

Using Questa inFact to Verify an AMBA AHB Bus Fabric

100
100%
90
80

Coverage (%)

70
60

inFact

50

51%

40
30
20

CRT

10

10%

Simulation Time After Initialization (ms)

Mentor Graphics Corp.

45

www.mentor.com

9800

9500

9200

8900

8600

8300

8000

7700

7400

7100

6800

6500

6200

5900

5600

5300

5000

4700

4400

4100

3800

3500

3200

2900

2600

2300

2100

1800

1500

1200

900

600

300

Applying iTBA

Incremental Change Results in >>10X Improvement

Re-Use Existing Testbench

No Change to Verification Language

No Change to Verification Methodology

No Change to Verification IP Blocks

Incremental Changes

Top Level Testbench Only

Describe Test Plan Functionality in a Graph

Replace Existing Sequence Generator

inFact Simulation Results

Graph Controls VIP Blocks

Removes, Reduces, or Keeps Redundancy

Targets Coverage, Not Just Measures It

Top Level Testbench

Score
board

Test
Gen

Verification IP Block
Test
Driver
Gen
Verification
IP Block
Test
Score
Driver
Monitor
Gen
Verification
board IP Block
Test
Score
Gen
board
Score
board

Driver
Monitor

Monitor

Mentor Graphics Corp.

46

www.mentor.com

DUT

Simplicity of iTBA

Top Layer is Described in a Standard Graph

Simplicity of Graphs

Start

These are standard - nothing new here

Used since 1960s for compiler & software test

Easy to write, read, and control - visual

Contain highly compressed information

init
wait_rdy

Rw_opts setup_wr
setup_rd

rw_2
Rw_size
rw_4

Origin of Graphs

Pre-date Verilog, VHDL, etc.

John Backus - IBM Fellow in 1963

Also invented Fortran and Algol

rw_1

ack
Stop

Start = init repeat ( wait_rdy Rw_opts Rw_size ack ) ;

Rw_opts = setup_rd | setup_wr ;
Rw_size =

rw_1 | rw_2 | rw_4 ;

Mentor Graphics Corp.

47

www.mentor.com

Power of iTBA

Questa inFact Adapts and Learns During Simulation

Power of Algorithms

Original graph based test applications were limited to deterministic systems

Used by many computer companies to test software & compilers

But VLSI hardware systems are non-deterministic

How to automatically generate deterministic stimulus for non-deterministic

systems?
Start

Uniqueness of Questa inFact

Extend graph technology to VLSI design verification

Apply breakthroughs in automata based math

Close coverage >>10X faster

init
wait_rdy

Rw_opts setup_wr
setup_rd

rw_2
Rw_size
rw_4
ack
Stop
Mentor Graphics Corp.

48

www.mentor.com

rw_1

Creating Graphs
Easy to Use

Interactive Development Environment

Graphical User Interface

Automatically Generates Graphs from Text

Integrate With Questa Simulation

49

Compile Testbenches

Link Graphs to Waveforms

Unmatched Debugging

Your Initials, Presentation Title, Month Year

Mentor Graphics Corp.

www.mentor.com

inFact AXI Master

Rule Code, Expanded Graph, and Collapsed Graph

Collapsed
View

==

Mentor Graphics Corp.

50

www.mentor.com

Intelligent Testbench Automation

Verify all functionality called out in test plan

Reduce or remove unwanted redundancy

Confirm test plan coverage with UCDB

Leave Time to Expand Coverage

Add tests to cover more functionality

Target test generation to desired areas

Hunt for bugs so they dont escape into mfg

iTBA
CRT
DT
Less Time

More Coverage

Achieve Target Coverage in Less

Time

Functional Coverage

Achieve Target Coverage >>10X Faster

Project TImeline

Mentor Graphics Corp.

51

www.mentor.com

Agenda

Verification Overview

Assertion and Functional Coverage

Constrained Random
Requirements Tracing
Algorithmic TB (InFact)
Questa Formal
Questa Codelink
Questa VIP

Mentor Graphics Corp.

52

www.mentor.com

Questa Verification Platform

Best In Class Engines

Questa Sim
Questa inFact

Questa Formal

Questa PA

Questa CDC

Questa VIP

Functional

Static

Verification

Verification

Verification
Management

Verification
Methodology

Dynamic

HW/ SW
UVM
UVM Connect
UVM Express

Questa Codelink

Questa Verification
Management

Mentor Graphics Corp.

53

www.mentor.com

Questa Verification Platform

Best In Class Engines - Unified Front End Analysis & Compile

Questa Sim
Questa inFact

Questa Formal

Questa PA

Questa CDC

Questa VIP

Functional

Static

Verification

Verification

Verification
Management

Verification
Methodology

Dynamic

HW/ SW
UVM

Questa Codelink

UVM Connect
UVM Express

Questa
Verification
Management

Mentor Graphics Corp.

54

WG - Feb 2013

www.mentor.com

Questa Verification Platform

Comprehensive integrated SOC

verification platform
Best in class engines
Integrated
Comprehensive debug analysis

Industry Leading SOC Verification Solutions

Coverage Closure Solution
Low Power Verification Solution
Software Driven Verification Solution

Standards Leadership

Driving the evolution of IEEE

standards
Major donations to Accellera UVM
Accellera UCIS from Mentor UCDB
Mentor Graphics Corp.

55

WG - Feb 2013

www.mentor.com

Simulation Alone Is Insufficient

MYTH:

If we only had a faster simulator, if we

only had a better coverage model,
things would have been better.

Simulation algorithms,
no matter how good,
have inherent
limitations

assert always (A==B) <-> E ;

A [31:0]
E
B [31:0]

0-In formal
verification directly
addresses these
limitations

264 vectors * 1 vector every s

= 584,941 years

Mentor Graphics Corp.

56

www.mentor.com

Simulation vs. Formal

Simulation

Lets see what happens if I apply this stimulus?

Probabilistic whether or not stimulus exposes
interesting behavior

Formal

Simulation

What stimulus do I have to apply to make this

happen?
Use mathematics to solve for the stimulus

Formal

Mentor Graphics Corp.

57

www.mentor.com

Static Formal Verification

Static Formal

Takes a single start-state

Performs exhaustive
verification of an
assertion(s)
Provides proofs

Design Under Test

0in_prove

Provides
counterexamples
0in_confirm

Can be used early in the

design cycles
Does not require a
testbench
Ensures critical
functionality is correct

=
FIFO

Bus Arbiter

Overflow error

Corner cases proven

Mentor Graphics Corp.

58

www.mentor.com

Static Formal
Verification
Dynamic
Formal
Verification

Design State Space

Overcomes inherent limitation of formal techniques

Combines formal verification and simulation to

cover more of the design

Unified coverage ties techniques together

Mentor Graphics Corp.

59

www.mentor.com

Unified Coverage Data Base

Test
Functional Assertion
Code
Engine
Coverage Coverage

0-in
Formal

User
Coverage

Test
Specific

Plan

Native UCDB Generation

Test Associated Merging

HTML Export

Database

Open Read And Write

API, C Functions & CLI

RD / WR API

UCDB Browser

RD / WR API

Test Ranking

Test Plan Tracking & Analysis

Mentor Graphics Corp.

60

www.mentor.com

Agenda

Verification Overview

Assertion and Functional Coverage

Constrained Random
Requirements Tracing
Algorithmic TB (InFact)
Questa Formal
Questa CDC

Questa Codelink
Questa VIP

Mentor Graphics Corp.

61

www.mentor.com

Domains Require Special

Attention
Clock Domain Crossing signal

Tx

Logic
A

Rx

Clock domain A

Clock domain B

Signals that cross

asynchronous clock
domains (CDC signals)
WILL violate setup and
hold conditions

Clock A
Tx
Clock B

Setup/hold window

Mentor Graphics Corp.

62

www.mentor.com

Clock Domain Crossing (CDC) Signals

Cause Metastability

When setup/hold conditions are violated, the

output of a storage element becomes
unpredictable
Setup/hold window

MTBF

Domain A
Domain B

CLK

CLK
Q

1
fclk fin td

fclk = Clock Frequency

fin = Input Signal Frequency
td = Duration of critical time window

This effect is called metastability

Metastability is UNAVOIDABLE in designs with

multiple asynchronous clocks
Mentor Graphics Corp.

63

www.mentor.com

Designers Use Synchronizers to

Isolate Metastability
Q

Clock A

Clock B

Metastability window

Rx
Tx
i -1

i +1

i +2

i -1

ii

i +1
i +1

i +2
i +2

i +3

When metastability occurs, the delay through a

synchronizer becomes unpredictable
Mentor Graphics Corp.

64

www.mentor.com

Complete Anatomy of

CDC Bugs

1. Missing or incorrect synchronizer

2. Incorrectly implemented CDC protocol
3. Design does not account for nondeterministic delay
through synchronizers (a.k.a. reconvergence error)
Mentor Graphics Corp.

65

www.mentor.com

The Benchmark CDC Verification

Solution

1. Provides complete structural CDC analysis

2. Provides complete protocol verification to ensure
correct transfer of data across synchronizers
3. Supports metastability injection in simulation to
enable detection of reconvergence errors
Mentor Graphics Corp.

66

www.mentor.com

Step 1. Structural CDC Analysis

Block B

Block A

test_clk

a_clk

/
config A

Mode control

b_clk
config B

Identify all primary asynchronous clocks

Identify the clock distribution/control strategy
Derived clocks, clock dividers
Clock gating, on/off schemes

Mentor Graphics Corp.

67

www.mentor.com

Provides Automatic Protocol

Checks

CDC protocols ensure

that data is predictably
transferred between two
clock domains

The 0-In CDC

verification solution
automatically generates
assertions to capture
these protocols

Protocol: When a transmitter's data select signal

crosses a clock domain and drives the select input
of a data multiplexer in the receiver, it must be held
stable long enough for the signal to be sampled
reliably by the receiver and the data must remain
stable while the data select signal asserts.

Assertion

Only when CDC protocols are represented as assertions can

they be used both in simulation and in formal verification
Mentor Graphics Corp.

68

www.mentor.com

Performs Static Reconvergence

Analysis

69

Automatically
identifies potential
reconvergence
problems in logic

Generates
metastability
injection assertions
to be used in
simulation

Supports both
combinational and
sequential
reconvergence

Synchronizer 1

Potential
Reconvergence Problem

Synchronizer 2

Mentor Graphics Corp.

www.mentor.com

Easy to Use Reporting and Debugging

Provides a results overview window and details for all CDC

issues

Uses generated schematics (with user control) where

appropriate

Mentor Graphics Corp.

70

www.mentor.com

Step 2. CDC Protocol Verification

Static CDC Analysis
CDC
Analysis

RTL

Protocol
Assertions

Protocol Verification

Simulation

Formal
Verification Simulate assertions using

Unified Coverage Data Base

existing testbench
0-In Formal verification
Measure coverage
Create regression tests

Mentor Graphics Corp.

71

www.mentor.com

Verifying Protocol Assertions in

Simulation

All CDC protocol violations are shown in CDC

analysis window
Debug is performed in simulation environment
Highlights exactly where a
protocol violation occurs

Mentor Graphics Corp.

72

www.mentor.com

Coverage Metrics for Protocol

Assertions
Using assertions to capture CDC protocols enables
coverage metrics to be collected

Coverage metrics:

Ensure that all CDC paths

are exercised
Ensure that all protocol
corner cases are stressed
Enable regression test
development for full
coverage
Unified Coverage Data Base

Mentor Graphics Corp.

73

www.mentor.com

Step 3. Reconvergence
Verification
Static CDC Analysis
RTL

CDC
Analysis

Simulation

Metastability
Assertions

Reconvergence Verification

Unified Coverage Data Base

Simulate assertions using

existing testbench
Measure coverage
Create regression tests

Mentor Graphics Corp.

74

www.mentor.com

Metastability Injection Assertions

Metastability happens

i -1

i +1

i +2

i +3

Metastability injector randomly

subtracts
a cycle
from
the delay
adds a cycle
to the
delay

Metastability injection assertions randomly modify the delays

through synchronizers (+1 or -1 cycle) when metastability
conditions are present in silicon

Mentor Graphics Corp.

75

www.mentor.com

Questa CDC Automatically Adds

Metastability Injection Assertions

Only adds metastability injection assertions to

those synchronizers that might cause
reconvergence issues

Random delay (+1 or -1 cycle) is only inserted

when metastability is possible

Metastability injection assertions automatically

collect coverage data

Automatic metastability injection in simulation is the

only way to effectively verify your CDC
reconvergence issues
Mentor Graphics Corp.

76

www.mentor.com

Metrics for Metastability

Injection Assertions
Using assertions for metastability injection enables
coverage metrics to be collected

Coverage metrics:

Ensure that all CDC paths

are exercised
Ensure that all protocol
corner cases are stressed
Enable regression test
development for full
coverage
Unified Coverage Data Base

Mentor Graphics Corp.

77

www.mentor.com

Questa CDC Verification Delivers

Structural CDC analysis

Automatically recognizes a large set of synchronizers

Comprehensive modal analysis

Protocol verification

Automatic generation of CDC protocol assertions

These can either be proven with formal analysis or
verified through simulation

Reconvergence verification

Complete structural analysis to identify potential

reconvergence issues
Automatic metastability injection in simulation to
verify the design correctly handles reconvergence

Mentor Graphics Corp.

78

www.mentor.com

Agenda

Verification Overview

Assertion and Functional Coverage

Constrained Random
Requirements Tracing
Algorithmic TB (InFact)
Questa CDC
Questa Formal
Questa Codelink
Questa VIP

Mentor Graphics Corp.

79

www.mentor.com

Manually Linking Software with Simulation

Must convert software binary to memory image file

No source-level debug

Slow, ~ 1 to 10 instructions/sec

Integrated
Debug
Environment

CoverageDriven
Verification

software
Developer

Verification
Engineer

Testbench
Automation

Mentor Graphics Corp.

80

www.mentor.com

Standards:
VHDL, Verilog,
SystemVerilog,
PSL, SystemC

System-Level
Design

AssertionBased
Verification

Codelink Automates Software Testbenches

Loads software binary

Full source-level debug

Significantly faster than full-functional model

Integrated
Debug
Environment

CoverageDriven
Verification

Codelink
software
Developer

Testbench
Automation

Mentor Graphics Corp.

81

www.mentor.com

Standards:
VHDL, Verilog,
SystemVerilog,
PSL, SystemC

System-Level
Design

AssertionBased
Verification

Agenda

Verification Overview

Assertion and Functional Coverage

Constrained Random
Requirements Tracing
Algorithmic TB (InFact)
Questa CDC
Questa Formal
Questa VIP

Mentor Graphics Corp.

82

www.mentor.com

What is Verification IP?

SoC Design Trends

More platform based designs

Increasing reuse of Design IP
Built around standard interfaces

Complex verification environment

reuse a necessity

Verification IP

Re-usable testbench building blocks

Compliant with standard interfaces and protocols
Built using standard languages and methodologies

Mentor Graphics Corp.

83

JSP, Questa Verification IP, September 2011

www.mentor.com

Questa Verification IP

Comprehensive verification IP

Complete protocol coverage and

checking
Test suite with compliance tests

Leverage latest verification

techniques

Verification planning, constrained

random, functional coverage
Developed for OVM and UVM

Supports popular and leading edge

SoC standards
PCIe, USB, AMBA, ethernet,

Mentor Graphics Corp.

84

www.mentor.com

Questa Verification IP Features

Complete protocol test sequences and coverage

OVM Testbench With RTL Design

Test Sequence

SCORE
BOARD

Coverage

AXI Agent
AXI SLAVE
VIP Component

DUT

User component
User design
Mentor Graphics Corp.

85

JSP, Questa Verification IP, September 2011

www.mentor.com

Questa Verification IP Example

Coverage results for SPI Controller

Questa Verification IP provides protocol Test Plan, Test Sequences

and Coverage

User adds DUT specific Test Plan, Test Sequences and Coverage

Questa Verification Management combines the results

Mentor Graphics Corp.

86

JSP, Questa Verification IP, September 2011

www.mentor.com

Questa Verification IP Features

Protocol stack debug

Quickly understand and analyze bus activity

Mentor Graphics Corp.

87

www.mentor.com

Questa Verification IP Features

Protocol stack debug

Highlighting links transaction and signal activity

Highlighting shows
when signals are valid

Mentor Graphics Corp.

88

www.mentor.com

Questa Verification IP Features

Full support for UVM and OVM

Connect and reuse standard UVM or OVM components

Agents, TLM ports, sequences, sequence items, config
TEST
ENV
Test Config
Scoreboard

Host
interface
Agent

PCIe IP
DUT

Test
Sequence
Sequence
item

Host IF

PIPE

Virtual
Sequencer

Mentor Graphics Corp.

89

www.mentor.com

PCIe
PIPE
Agent

Protocol support

Supports popular and leading edge SoC standards

AMBA

APB3, AHB, AXI, AXI4, AXI4-lite, AXI4-stream, AXI-LP

PCI Express

1.1, 2.0, 3.0

USB

2.0, 3.0, OTG, UTMI, PIPE

Ethernet

10/100, 1G, 10G, 40G, 100G

SPI 4.2
DDR2, DDR3
OCP 2.2
HDMI
I2C, I2S
SPI, UART

Mentor Graphics Corp.

90

www.mentor.com

Questa Verification Platform

Best In Class Engines

Questa Sim
Questa inFact

Questa Formal

Questa PA

Questa CDC

Questa VIP

Functional

Static

Verification

Verification

Verification
Management

Verification
Methodology

Dynamic

HW/ SW
UVM
UVM Connect
UVM Express

Questa Codelink

Questa Verification
Management

Mentor Graphics Corp.

91

www.mentor.com

Questa Verification Platform

Best In Class Engines - Unified Front End Analysis & Compile

Questa Sim
Questa inFact

Questa Formal

Questa PA

Questa CDC

Questa VIP

Functional

Static

Verification

Verification

Verification
Management

Verification
Methodology

Dynamic

HW/ SW
UVM

Questa Codelink

UVM Connect
UVM Express

Questa
Verification
Management

Mentor Graphics Corp.

92

WG - Feb 2013

www.mentor.com

Questa Verification Platform

Comprehensive integrated SOC

verification platform
Best in class engines
Integrated
Comprehensive debug analysis

Industry Leading SOC Verification Solutions

Coverage Closure Solution
Low Power Verification Solution
Software Driven Verification Solution

Standards Leadership

Driving the evolution of IEEE

standards
Major donations to Accellera UVM
Accellera UCIS from Mentor UCDB
Mentor Graphics Corp.

93

WG - Feb 2013

www.mentor.com

Agenda

Verification Overview

Assertion and Functional Coverage

Constrained Random
Requirements Tracing
Algorithmic TB (InFact)
Questa CDC
Questa Formal
Questa VIP

Rule Checking

Precision

Mentor Graphics Corp.

94

www.mentor.com

A RTL Reuse Method

Previous Design
Unknown Quality
Unfamiliar Design

Reused
RTL

Step 1
Design
Integrity

Step 2
Quality
Assessment

Automate Analysis
of RTL Integrity

Assess RTL to
Design Standards

Step 3
Design
Visualization

Visualize Behavior
and Structure

New
Functionality

Quality Quantified
Fully Understood

Reused
RTL
New
Functionality

Share Knowledge

Mentor Graphics Corp.

95

Ready to Reuse

www.mentor.com

Do I Have all the Files?

Design Files

Verilog
VHDL
C & C++
Memory Content
Constraint Files

Design Support Files

Tool Scripts
Design Documents

Tool Output Files

Netlists
Reports
Projects

Design Files: describe the design to the tools in a flow

Design Support Files: help you understand & automate

aspects of the design flow

Tool Output Files: tool-generated files for use as input or

that provide information

Mentor Graphics Corp.

96

www.mentor.com

Missing Files are Found

Missing Files are

Highlighted in the Hierarchy

Mentor Graphics Corp.

97

www.mentor.com

A RTL Reuse Method

Previous Design
Unknown Quality
Unfamiliar Design

Reused
RTL

Step 1
Design
Integrity

Automate Analysis
of RTL Integrity

Step 2
Quality
Assessment

Ready to Reuse
Quality Quantified
Fully Understood

Reused
RTL

Assess RTL to
Design Standards

New
Functionality

New
Functionality

Mentor Graphics Corp.

98

www.mentor.com

Take Emotion Out of the Decision

How do you
tell people
their baby is
ugly?

Mentor Graphics Corp.

99

www.mentor.com

Objective Standards are Key

100%

Code
Quality
Automated
RTL Quality Assessment

HDL Files

RMM 3.0
Rules

Xilinx
Rules

Altera
Rules

Essential
Rules

User
Definable

Design Rule Standards

Mentor Graphics Corp.

100

www.mentor.com

0%

Code Scoring

All rules have can have a score

This allows you to see how good your code is within seconds
You can set your own scoring system for your own rules
Code quality results are automatically created for your checking
run (it can optionally be turned off).

Mentor Graphics Corp.

101

www.mentor.com

Assess Reuse Effort

Filter, Group,
Sort Results

View Errors
in Context

View
Summary
Mentor Graphics Corp.

102

www.mentor.com

Customize Your Own Rules

The Process:

Mentor Graphics Corp.

103

www.mentor.com

Create ruleset(s)

Drag & drop

rulesets & rules
into your own
rulesets

Change rule
parameters

Create policies
that link to
rulesets

Good Coding Practices

No extra or unused signals

Isolate or avoid gated clocks

Avoid internally-generated resets

Avoid mixed clock edges

No multiply-driven signals

Assign a value to a signal before reading it

Matching comparison/assignment ranges

Ensure expected & good results

Mentor Graphics Corp.

104

www.mentor.com

Downstream Checks
No combinational feedback loops
Avoid latches & inferred registers
Register outputs
Avoid or isolate gate-level logic
Ensure naming compatibility with downstream tools
Avoid delay times
Avoid default initialization
Establish a subset of allowed constructs
Use complete sensitivity lists

Catch issues before running other tools

Mentor Graphics Corp.

105

www.mentor.com

Document Quality

Export Summary report as CSV, TSV, or HTML

Export Result Table as CSV, TSV, or HTML

Export rules used in ASCII format

Mentor Graphics Corp.

106

www.mentor.com

A RTL Reuse Method

Previous Design
Unknown Quality
Unfamiliar Design

Reused
RTL

Step 1
Design
Integrity

Step 2
Quality
Assessment

Automate Analysis
of RTL Integrity

Assess RTL to
Design Standards

Step 3
Design
Visualization

Visualize Behavior
and Structure

New
Functionality

Quality Quantified
Fully Understood

Reused
RTL
New
Functionality

Share Knowledge

Mentor Graphics Corp.

107

Ready to Reuse

www.mentor.com

Design Visualization Challenges

Learning & Integrating Legacy Designs

Designs are easier to write in Text, but harder to

read

Design structure is hard to see in text

Design functionality is hard to trace across text files
Interfaces are often poorly documented

Re-Drawing by Hand is Inefficient

Very Time Consuming
Error prone
Hard to maintain

Existing
RTL Code

Mentor Graphics Corp.

108

www.mentor.com

Delve Deeper with DesignPad

Code Browser takes you

inside design units

Navigate ports
Navigate declarations

View as graphics
Collapse, open up/down,
& split windows to aid
navigation
Diff similar files

Mentor Graphics Corp.

109

www.mentor.com

Structural Levels are Block Diagrams

Mentor Graphics Corp.

110

www.mentor.com

State Machines are Bubbles & Arcs

Mentor Graphics Corp.

111

www.mentor.com

HTML Export
Automatically create an interactive
website with HTML Export:

Complete control of content

Navigation matches HDL Designer

View results in an HTML browser

Snapshot a project any time

No access to design database

Great for Design Reviews Too

Mentor Graphics Corp.

112

www.mentor.com

The Results

Mentor Graphics Corp.

113

www.mentor.com

Agenda

Verification Overview

Assertion and Functional Coverage

Constrained Random
Requirements Tracing
Algorithmic TB (InFact)
Questa CDC
Questa Formal

Rule Checking

Precision

Mentor Graphics Corp.

114

www.mentor.com

Precision 2009 - Synthesis Leadership

Leading multi-vendor physical synthesis

Average 10 % FMAX Improvement
26 devices supported from all major vendors

Fully automatic incremental synthesis

Physical Synthesis

Up to 60% run-time savings

Unique resource analysis/management improves QoR

Industry leading mixed language support

FPGA P&R

Superior SystemVerilog coverage

Mentor Graphics Corp.

115

www.mentor.com

How Physical Synthesis Works

Synthesized
Netlist

Estimates location
Estimates routing resources
More accurately estimates net
delays
Identifies Critical Paths

Physical Synthesis
Generic Placer &
Delay Estimator
Physical
Library

Physical
Optimizations

Replication

Netlist Optimization
Retiming, Replication, Resynthesis

Retiming

Advanced Delay Estimation

No placement sent to P&R

No DRC/packing violations
Maximum flexibility for P&R

Re-Synthesis

Optimized Netlist

Mentor Graphics Corp.

116

www.mentor.com

Automatic Incremental Synthesis

Up to 60% runtime savings

Design, change dependent

Industrys only automatic

incremental synthesis
No partitioning or prior planning
Maintains QoR with cross-hierarchy
optimizations
Based on real changes in parse tree

All FPGA families supported

design change

Incremental
Compile

Compile
Synthesize

FPGA Vendor
P&R

FPGA
Vendor
P&R

Mentor Graphics Corp.

117

Incremental
Synthesis

www.mentor.com

PRECISE-EXPLORE

Mentor Graphics Corp.

118

www.mentor.com

The Need for Design Exploration

Each design has unique characteristics

Very time consuming to try all synthesis options for each

individual design

Mentor Graphics Corp.

119

What's New in Precision 2012b

www.mentor.com

Precise-Explore

Exploration Goals

Meet Constraints
Max Frequency
Min Area

Exploration Options

Turning off will use

selected mode from
other options page

Place & Route

Runtime Limits

Mentor Graphics Corp.

120

What's New in Precision 2012b

www.mentor.com

Precise-Explore
Benefits

Precise-Explore automates
the process

Explore implementation
options for different design
structures in both synthesis
and P&R

Control for exploration

options

Multiple computing options

Detailed reporting

Mentor Graphics Corp.

121

What's New in Precision 2012b

www.mentor.com

Precise-Explore

Product Configuration

2012b

Controlled feature
Requires special license
License available upon customer request

2012c

Production feature
Included in Precision RTL Plus license

Planned, subject to change

Mentor Graphics Corp.

122

What's New in Precision 2012b

www.mentor.com

Precision Synthesis 2012b Summary

Support for the latest FPGA devices & software

Expanded SystemVerilog coverage

Explore your FPGA design with Precise-Explore

Mentor Graphics Corp.

123

What's New in Precision 2012b

www.mentor.com

Mentor Graphics
THANK YOU

Mentor Graphics Corp.

124

www.mentor.com