VLSI System Design

Verification & Writing Testbench

李昆忠
國立成功大學電機系

06-2757575 X 62371
[email protected]

VLSI TEST LAB. NCKU-EE KJLEE

 Reference Book
• J. Bergeron, Writing Testbenches function of HDL
models , Kluwer Academic Publishers, 2003

VLSI System Design Verification.2 NCKUEE-KJLEE

 What is a Testbench?
• Create a pre-determined input sequence to a design,
then optionally observe the response.

Testbench

Design
Under
Verification

VLSI System Design Verification.3 NCKUEE-KJLEE

 Verification
• A process used to demonstrate the correctness of a
design.
• Verification consumes about 70% of the design effort.
• The methodologies to reduce the verification time
− Parallelism: It is necessary to be able to write - and debug -
testbenches in parallel with each other as well as in parallel with
the implementation of the design.
− Abstraction: The verification process can use higher abstraction
levels by working at the transaction- or bus-cycle levels (or even
higher ones), instead of always dealing with low-level zeros and
ones.
− Automation: Require standard processes with well-defined
inputs and outputs.
VLSI System Design Verification.4 NCKUEE-KJLEE
 Reconvergence Model
• A conceptual representation of the verification process
− Used to illustrate what exactly is being verified.
− Verify the results of some transformations including:
Š RTL from a specification
Š Insertion of scan chain
Š Synthesizing RTL into gate-level netlist
Š Layout from gate-level netlist

Transformation

Verification

VLSI System Design Verification.5 NCKUEE-KJLEE

 The Human Factor
• The same individual
RTL coding

Interpretation
Specification
Verification

• Different individuals (with redundancy)

RTL coding
Interpretation
Specification
Verification
Interpretation
VLSI System Design Verification.6 NCKUEE-KJLEE
 Formal Verification Applications
• Equivalence Checking – Check the RTL codes and gate-
level netlists.
Ex. (Tools) Conformal LEC, Assertain ABV, Formality.
• Model Checking (Property Checking) – Does the RTL
Codes model has desired properties of design specifics?
• Application to Fault Diagnosis.
Synthesis
RTL Coding
RTL
Schematic
Specifics (gate level)
Model
Interpretation Checking Equivalence Checking
Assertions No need any stimuli !
VLSI System Design Verification.7 NCKUEE-KJLEE
 Formal Verification – Equivalence Checking

• Compare two models: Prove mathematically that the origin

and output are logically equivalent and the transformation
preserved its functionality.
• Compare two netlists to ensure some netlist post-
processing did not change the functionality of the circuits.
− Examples: scan chain insertions, clock-tree synthesis, minor
manual changes.

Synthesis
Schematic
RTL
(gate level)

Equivalence Checking
VLSI System Design Verification.8 NCKUEE-KJLEE
 Formal Verification - Model Checking
• Assertions or characteristics of a
design are formally proven or
disapproved. RTL Coding RTL
Specific
− Look for generic problems or violation
of user-defined rules about the Model
behavior or the design. Interpretation Checking
• Examples: Assertions
− Unreachable or isolated states of a FSM
− Occurrence of deadlock
− Given that a signal will be asserted =>
Other signals will be asserted

Problem: What assertions to check???

VLSI System Design Verification.9 NCKUEE-KJLEE
 Functional Verification
• Purpose: Ensure that a design implements intended functionality.
• Show that a design meets the intent of its specifications, but it
cannot prove it.
RTL Coding

Specifics RTL
Functional
Verification

• Specs are usually written in un-precise natural languages

• Easy to show a design does not implement the intended design, but
not vice versa.
• Verification can show the presence of errors, not their absence.
VLSI System Design Verification.10 NCKUEE-KJLEE
 The Cost of Verification (I)

• Verification takes long and costs much.

• Verification does not generate a profit or make money.
• Verification is never truly completed.
• No one can prove that a design is error-free.

VLSI System Design Verification.11 NCKUEE-KJLEE

 The Cost of Verification (II)

• False negative: find an error where none exist

− The implementation of the verification is modified to change the
answer from “no” to “yes”. - the easy ones to identify.
• False positive: a bad design is shipped unknowingly.
− The verification fails to identify an error.

Errors No Errors
Bad Type Ⅱ
Design (False Positive)
Good Type Ⅰ
Design (False Negative)

VLSI System Design Verification.12 NCKUEE-KJLEE

 Verification Tools
• Linting Tools – (e.g., nLint tool)
− Identify common mistakes programmer made, such as syntax errors.
− Similar to spell checkers.
− Only find problems that can be statically deduced by looking at the code
structure, not problems in the algorithm or data flow.
• Simulators (e.g., Riviera-Pro, Assertain tool)
− An approximation of reality.
− Not static tools (∵Simulation requires stimulus)
Linting tools are static tools
− The simulation outputs are validated externally, against design intents.

Assertain Tool - Two mode both static verification (for Linting) and
dynamic verification (for simulator).
Dynamic verification - Use a set of stimuli to exercise one or more models
of a design or a hardware implementation of the design.
VLSI System Design Verification.13 NCKUEE-KJLEE
 Message generated by nLint
Warning: more than one top module detected.
Warning: output signal should not be referenced inside the
module.
Warning: glue logic found in top module.
Warning: more than one clock signal detected in the
module, clocks.
Error: register should have a set or reset signal.
Warning: the port is not in order with port update.
Warning: Asynchronous clock detected.
Warning: the input port and output port should not be
connected directly.
Warning: the port should be connected by name.
VLSI System Design Verification.14 NCKUEE-KJLEE
 Code Coverage & FSM Coverage
• Code coverage refers to the percentage of the code
that has been (and more importantly not been)
executed under verification.
(1) Code Coverage
− Statement Coverage (Block Coverage)
− Branch coverage
− Condition Coverage (Expression Coverage)
− Trigger Coverage
− Toggle Coverage
− Path Coverage …
(2) Finite State Machine Coverage
− State Coverage
− Arc Coverage
− Path Coverage …
VLSI System Design Verification.15 NCKUEE-KJLEE
 Statement Coverage
• How many of the total lines of code were executed?
• Ex:
□ if (parity == ODD || parity == EVEN) begin
□ tx <= compute_parity(data, parity);
□ #(tx_time);
end
□ tx <= 1’b0; When
□ #(tx_time); parity!=ODD &
□ if (stop_bits == 2) begin Parity!=EVEN &
stop_bits=2
□ tx<=1’b0;
□ #(tx_time);
end
• Statement Coverage = 6/8 = 75%
VLSI System Design Verification.16 NCKUEE-KJLEE
 Path Coverage

• All possible ways you can execute a sequence of statements

• Ex:
□ if (parity == ODD || parity == EVEN) begin
□ tx <= compute_parity(data, parity);
□ #(tx_time);
end This path is not exercised.
□ tx <= 1’b0; Hence path Coverage=75%
□ #(tx_time); Statement Coverage=100%
□ if (stop_bits == 2) begin
□ tx<=1’b0;
□ #(tx_time);
end
□ □ □ □

VLSI System Design Verification.17 NCKUEE-KJLEE

 Some Branch Definitions (Tool Depend)
• Branch Coverage - execution all of conditional statements. Ex. All
case or if-else statements.
• Condition Coverage (Expression Coverage) - check all of the
expression cases are exercised.
Ex. If (conditions) and else conditions.
• Toggle Coverage - measures design activity in terms of changes
of signal logic values.
Ex. Signals are active by applying positive or negative edge
• Trigger Coverage - checks whether each process has been
started uniquely by each of the signals.
• State Coverage - check if each state in FSM is reach by input
stimuli.Ex.(4 States design trace) S0 -> S1 -> S3 -> S2
• Arc Coverage – a transition between each adjacent pairs of all
states’ set. Ex. 7 transitions.
S1 S3
S0
VLSI System Design Verification.18 S2 NCKUEE-KJLEE
 What Does 100 % Coverage Mean?
• Completeness does not imply correctness.
• Code coverage lets you know if you are NOT done.
• Code coverage tools can be used as profilers.

General Coverage Targets

Measurement Coverage
Statement 100%
Branch 100%
Condition 50~100%
Path >50%
Toggle 100%

VLSI System Design Verification.19 NCKUEE-KJLEE

 The Role of the Verification Plan
• Schedule: Need a mechanism (metrics) to determine
when you will be done. (degree of confidence)
• Start from the specification including the architectural
and design specification.
− The specification document is the common source for the
verification and implementation efforts.
• Define the 1st time success
− Ensure that all essential features are appropriately verified.
− Features priority in the face of schedule pressure
− A detailed schedule as well as resource allocation from the
verification plan

VLSI System Design Verification.20 NCKUEE-KJLEE

 Metrics of Verification Plan
• Code-Related Metrics
− Code coverage: measure how thoroughly the verification suite
exercises the source code being verified.
− Number of lines of code: (for implementation efficiency) This
metric can be used to compare the productivity offered by new
verification languages or methods.
− Ratio of lines of code: (for complexity measurement) the ratio
between the design being verified and the verification suite.
− Source code changes (over time): used to determine if the
code is becoming stable or identify the most unstable section of a
design.
• Quality-Related Metrics
− They are probably more directly related with the functional
verification than other productivity metrics.
− Number of bugs found during its service life (reliability)
VLSI System Design Verification.21 NCKUEE-KJLEE
 Levels of Verification

• Verification can be performed at various levels of granularity.

− Involve trade-offs and require stable interface.
1. Unit-Level Verification:
− Design units are logical partition. (Recommend ad-hoc verification
process - to ensure no syntax errors in the RTL and that basic
functionality is operational)
2. Reusable Components Verification
− Regression suite to verify the compliance of the component after any
modification
3. ASIC and FPGA Verification
− ASICs and FPGAs are physical partitions.
4. System-Level Verification
− The verification focuses on interaction among the individual
components.
5. Board-Level Verification
− To confirm that the system captured by the board design tool is
correct.
VLSI System Design Verification.22 NCKUEE-KJLEE
 Verification Strategies

• Decision to be made
− Type of testcases, the level of granularity, ...
− Level of abstraction: With higher levels of abstraction, you
have less detailed control over the timing and coordination of the
stimulus and response, but it is much easier to generate a lot of
stimulus and verify long response.

• Verifying the response

− How to check the response
− Some responses are hard to verify
− Detect errors as earlier as possible

VLSI System Design Verification.23 NCKUEE-KJLEE

 The Verification Plan: From
Specification to Features (I)
• Identify features at the first step: interface-based feature,
function-based feature, architecture-based feature
• Label each features: Features should be labeled and have a
short description.
• Features
− Component-level features within the component being verified
− System-level features including connectivity, flow control, and
interoperability

VLSI System Design Verification.24 NCKUEE-KJLEE

 The Verification Plan: From
Specification to Features (II)
• Prioritize: not all features are created equal
− must-have: need to be thoroughly verified for all possible
configuration and usage options
− should-have: may be cancelled if schedule pressure forces the
reallocation of resources to the verification of more important feature
− nice-to-have: purely optional
• Design for verification
− Hard-to-verify features will be identified.
− Modify the design to aid verification
− Provide state preload functions to alleviate the verification cycle. (e.g.,
counter)
− Provide bypass paths for long data paths to have detailed control over
all the operands.
− Provide sample point and error injection mechanism
VLSI System Design Verification.25 NCKUEE-KJLEE
 Directed Testbenches Approach
• Individual features are verified using individual testbenches.
− The stimulus is manually crafted to exercise that feature.
− Useful for small number of testcases.
• Group into testcases
− Features should be grouped together and assigned to the same
verification engineer. (Each group of feature verification forms a testcase.
− Each testcase should be labeled and given a short description of its
objective. → Cross reference into the feature list
− Specify the acceptance criteria and what errors to look for.
− Inject errors to make sure they are detected.
• From testcases to testbenches
− Testcases naturally fall into groups and each group of testcases is then
divided into testbenches.
− Each testbench should be labeled and uniquely identified, e.g. filename.
• Verifying testbenches and measuring progress

VLSI System Design Verification.26 NCKUEE-KJLEE

 Cover-Driven Random-Based
Approach
• Functional coverage and constrained random generation
− With random verification, the inputs are subjected to valid individual
operations. It is the sequence and timing of these operations and the
content of the data transferred that is random.
• Measure functional coverage, not features
− Scenario: There are too many testcases and testcases exercise more than
the target features.
• From features to functional coverage
− Functional coverage measurement is used to identify which testcases were
executed instead of explicitly coding those testcases.
− Functional coverage is used to record which testcases and conditions were
automatically created by the random generator. - It is not code coverage.
− Each feature presents a characteristic or symptom in the input data stream,
the design configuration or the internal state of the design that must be
exercised. - Functional coverage must identify, then record, those c&s.
− Functional coverage modeling or definition is an evolving art.
• From features to testbenches and to generators
VLSI System Design Verification.27 NCKUEE-KJLEE
 Behavioral Models

• High-level modeling
− To efficiently accomplish the verification task, a skilled verification
engineer must be well versed in behavioral (i.e., non-
synthesizable and highly algorithmic) description.
− Necessary to understand the side effects of the simulation
algorithm and limitations of the language.
• Testbenches need a model to be debugged.
− RTL is slow to develop while testbenches continue to be written.
− Codes for testbench surpasses RTL.
− Do not use RTL-like code when writing testbenches.
• Behavioral models are used to debug testbenches.
− Behavioral models are available earlier than RTL models.
− Behavioral models run faster then RTL models.

VLSI System Design Verification.28 NCKUEE-KJLEE

 Benefits of Behavioral Models

• Develop and debug the testbenches in parallel with the

RTL coding.
• System verification can start earlier.
• It can be used as an evaluation and integration tool by
your customers.
RTL Coding Debug Synthesis
Specification
Behavioral Coding
Debug System Debug
Verification Plan Testbench Coding

VLSI System Design Verification.29 NCKUEE-KJLEE

 Synthesizable vs. Behavioral Models
• (Test Bench + Reference model) are not synthesizable
models
− To write a proper behavioral model, you have to focus on the
functionality, not the implementation.
• Synthesizable (RTL) model
− A model that can be automatically translated into a gate-level
netlist by a synthesis tool.

VLSI System Design Verification.30 NCKUEE-KJLEE

 RTL Coding Guidelines
• RTL Coding Guidelines to Avoid Undesirable Hardware
Structures
− To avoid latches, set all out puts of combinational blocks to
default values at the beginning of the block.
− To avoid internal buses, do not assign regs from two separate
always blocks.
− To avoid tri-state buffers, do not assign value 1’bz.

• RTL Coding Guidelines to Maintain Simulation Behavior

− All inputs must be listed in the sensitivity list of a combinational
block.
− The clock and asynchronous reset must be in the sensitivity list
of a sequential block.
− Use a non-blocking assignment when assigning to a reg
intended to be inferred as a flip-flop.

VLSI System Design Verification.31 NCKUEE-KJLEE

 Behavioral Descriptions
• Behavioral descriptions for (Test bench + Reference
model) because they will not be implemented in
hardware.
• Writing a behavioral descriptions requires a different
mindset
− Focus on the functionality, not the implementation.
− Non-synthesizable and highly algorithmic description.

VLSI System Design Verification.32 NCKUEE-KJLEE

 Example: Handshaking Protocol

• It detects that an acknowledge signal (ACK) is asserted

after a requesting signal (REQ) is asserted.
• Once ACK is detected, REQ is de-asserted, and it then
waits for ACK to be de-asserted.

ACK = 1 ACK = 0
REQ = 1 REQ = 0

ACK = 0 ACK = 1

VLSI System Design Verification.33 NCKUEE-KJLEE

 RTL Thinking Behavioral Thinking
(hardware designer) (verification engineer)

always @(state, ACK) begin initial begin

case (state) REQ = 1;
MAKE_REQ: wait (ACK==1);
REQ=1; # delay REQ = 0;
if (ACK ==1) wait (ACK ==0);
next_state = RELEASE; end
RELEASE:
REQ=0;
if (ACK==0)
next_state = ...
endcase ACK = 1 ACK = 0
end REQ = 1 REQ = 0

always @ (posedge clk or ) begin

if (reset =1) ACK = 0 ACK = 1
state = next_state;
else

VLSI System Design Verification.34 NCKUEE-KJLEE

 Characteristics of Behavioral
Models (I)
• They are partitioned for maintenance.
− Behavioral models tend to have very few instances creating a
narrow and shallow structure of large blocks.
• They don’t use a clock
− A behavioral model does not change state synchronously with a
clock. Instead, a behavioral model uses many different
synchronization mechanisms - one of which could be a clock edge.
− A clock would be used when data is sampled or produced
synchronously.
• They don’t use FSMs.
− The control algorithm and the data transformations would be part
of the control flow of the program.

VLSI System Design Verification.35 NCKUEE-KJLEE

 Characteristics of Behavioral
Models (II)
• Data can remain at a high-level of abstraction
− RTL: use a binary value with some # of bits and reduced to
integer.
− Behavior: floating #, records, multiple dimensional arrays.
• Data structure are designed for ease-of-use, not
implementation
− Example: a router composed of 256 bit records
− Physical implementation of a routing table: 16 bit RAM with 16k
locations

VLSI System Design Verification.36 NCKUEE-KJLEE

 Behavioral Descriptions
• Need to understand
− Abstraction & Limitation of Verilog language
Š Techniques to structure and encapsulate behavioral code for
maintainability.
Š Limitation on records, real number, multi-dimension, link-list and file.
− The side effects of simulation algorithm

VLSI System Design Verification.37 NCKUEE-KJLEE

 Encapsulating Useful Subprograms
• Structure and encapsulate behavioral code for maximum
maintainability
− Available constructs for structuring code: Module (entity and architecture),
Function, Task, and Module (package and package body).
− Encapsulation, an application of the structuring principle, can be used to
hide implementation details and package reusable code element.
− RTL models require a well-defined structure strategy. The structure of a
synthesizable model is dictated by the limitations of the synthesis tools,
often with little regard to the functionality.
− Testbenches are structured according to functional needs. You are free to
structure your code any way you like.
• Example: error reporting routines
− To have a consistent error reporting format, a set of standard routines are
used to issue messages during simulation.
− They are implemented as tasks, with two packaging alternatives.

VLSI System Design Verification.38 NCKUEE-KJLEE

 Error Reporting Routine (I)

module testcase; syslog.h

include "syslog.h" task error;
initial input [80*8:1] msg;
begin begin
if (..) error ("unexpected $write (-ERROR- at %t:
response\n"); %s",$time, msg);
terminate; end
end endtask
endmodule

VLSI System Design Verification.39 NCKUEE-KJLEE

 Error Reporting Routine (II)
module syslog;
integer warnings;
integer errors; module testcase;
initial
begin syslog log();
error = 0;
end initial
task error; begin
input [80*8:1] msg; If (…) begin
begin log.error (“unpected response”);
$write (-ERROR- at %t: %s", log.terminate;
$time, msg); end
errors = errors +1; end
end endmodule
endtask
…
endmodule

VLSI System Design Verification.40 NCKUEE-KJLEE

 Verilog Portability Issues
• Two compliant simulators can produce different results
− Most of the time, the differences are due to race conditions.
− Sometimes, the differences are due to different interpretations of
the Verilog standard:
) Because of unspecified situations in the IEEE standard, simulator
vendors are thus free to implement these unspecified portions of
the standard any way they want, yielding different simulation
results.
• The primary cause is the author’s lack of experience.

VLSI System Design Verification.41 NCKUEE-KJLEE

 Read/Write Race Conditions
• Two concurrent blocks attempt to read and write the
same register at the same timestep.
• The execution order among blocks is non-deterministic.

always @(posedge clk)

count = count + 1;

always @(posedge clk) Assume count currently is 3.

$write ("count = %d", count); What value will be displayed once
clk incurs a positive edge event?

VLSI System Design Verification.42 NCKUEE-KJLEE

 To Solve Read/Write (RW) Races
• Use non-blocking assignment.

always @(posedge clk)

count <= count + 1;

always @(posedge clk)

$write ("count = %d", count);

Assume count currently is 3. What

value will be displayed once clk
incurs a positive edge event?

VLSI System Design Verification.43 NCKUEE-KJLEE

 Write/Write (WW) Race Conditions
• Two concurrent blocks attempt to write to the same
register at the same timestep.
• Non-blocking assignment does NOT solve this problem.

always @(posedge clk)

count <= count + 1;

always @(posedge clk)

count <= count - 1;

VLSI System Design Verification.44 NCKUEE-KJLEE

 Initialization Races
• There is no initialization phase in Verilog
• Initial blocks are NOT necessarily executed first

always @(posedge clk)

$write("block #1 at %t", $time);

initial clk = 1'b1;

always @(posedge clk)

$write("block #3 at %t", $time);

VLSI System Design Verification.45 NCKUEE-KJLEE

 Guidelines for Avoiding Race
Conditions
• If a register is declared outside of the always or initial
block, assign to it using a non-blocking assignment.
Reserve the blocking assignment for registers local to
the block.
• Assign to a register from a single always or initial block.
• Do not assign any value at time 0.

VLSI System Design Verification.46 NCKUEE-KJLEE

 Conclusions
• Using behavioral code for test benches & behavioral
model.
• Behavioral model requires a greater degree of discipline.
• Encapsulation techniques allow behavioral model to be
structured to minimize performance.

VLSI System Design Verification.47 NCKUEE-KJLEE