Connecting Pre-silicon and Post-silicon Verification

Sandip Ray and Warren A. Hunt, Jr.

Department of Computer Sciences
University of Texas at Austin
{sandip, hunt}@cs.utexas.edu
http://www.cs.utexas.edu/users/{sandip, hunt}

Ray and Hunt (UT Austin) Post-silicon Verification November 18, 2009 1 / 14
 Motivation

Motivation

Formal analysis has shown promise in increasing reliability of computing

systems.
Can catch “high quality” bugs that are difficult to hit during
simulation.
Has been successfully applied to some industrial design components.
• FP execution units
• Control logic for out-of-order pipelines
But formal analysis has primarily been restricted to pre-silicon
Typical targets are RTL models and netlists.
Almost no connection with post-silicon verification.
How do we make use of formal analysis to facilitate post-silicon
design verification?

Ray and Hunt (UT Austin) Post-silicon Verification November 18, 2009 2 / 14
 Post-Silicon Verification

Post-silicon Verification

Post-silicon verification is the use of pre-production, physical

circuits to determine logical bugs.
Simulation speed may be 1,000,000,000 times faster than pre-silicon.
Facilitates exploration of very deep states.

Ray and Hunt (UT Austin) Post-silicon Verification November 18, 2009 3 / 14
 Post-Silicon Verification

Post-silicon Verification

Post-silicon verification is the use of pre-production, physical

circuits to determine logical bugs.
Simulation speed may be 1,000,000,000 times faster than pre-silicon.
Facilitates exploration of very deep states.

BUT
Control is limited.
Observability is extremely limited.
Factors limiting observability:
• Limited number of pins
• Cost of additional DFD logic.
• ...

Ray and Hunt (UT Austin) Post-silicon Verification November 18, 2009 3 / 14
 Post-Silicon Verification

Post-silicon Verification

Post-silicon verification is the use of pre-production, physical

circuits to determine logical bugs.
Simulation speed may be 1,000,000,000 times faster than pre-silicon.
Facilitates exploration of very deep states.

BUT
Control is limited.
Observability is extremely limited.
Factors limiting observability:
• Limited number of pins
• Cost of additional DFD logic.
• ...

Post-silicon verification is extremely expensive and tedious.

Ray and Hunt (UT Austin) Post-silicon Verification November 18, 2009 3 / 14
 Post-Silicon Verification

Post-silicon Debug Process

Start

Start in a known state

Intermediate Quickly get to a deep state
State
Continue until a bug occurs
Bug is unobserved
Bug! Bug may lay dormant
Observe Finally, observe a problem
Problem

It can take substantial effort to find and fix a bug.

Ray and Hunt (UT Austin) Post-silicon Verification November 18, 2009 4 / 14
 Post-Silicon Verification

Post-silicon Debug Process

Start

Start in a known state

Intermediate Quickly get to a deep state
State
Continue until a bug occurs
Bug is unobserved
Bug! Bug may lay dormant
Observe Finally, observe a problem
Problem

It can take substantial effort to find and fix a bug.

Typical Approach: Add extra hardware “hook” to improve observability.
But the hooks are added on-demand without analysis of design
invariants.
Once added, they are carried over from one design to next.

Ray and Hunt (UT Austin) Post-silicon Verification November 18, 2009 4 / 14
 Post-Silicon Verification

Post-silicon Debug Process

Start

Start in a known state

Intermediate Quickly get to a deep state
State
Continue until a bug occurs
Bug is unobserved
Bug! Bug may lay dormant
Observe Finally, observe a problem
Problem

It can take substantial effort to find and fix a bug.

Typical Approach: Add extra hardware “hook” to improve observability.
But the hooks are added on-demand without analysis of design
invariants.
Once added, they are carried over from one design to next.
A more disciplined process of on-chip instrumentation is necessary.

Ray and Hunt (UT Austin) Post-silicon Verification November 18, 2009 4 / 14
 Goals

Our Goal

Facilitate post-silicon verification by pre-silicon analysis.

Ray and Hunt (UT Austin) Post-silicon Verification November 18, 2009 5 / 14
 Goals

Our Goal

Facilitate post-silicon verification by pre-silicon analysis.

Pre-silicon Models
• Allow complete visibility of internal state.
• Can be mathematically formalized analyzed and reasoned about.

Ray and Hunt (UT Austin) Post-silicon Verification November 18, 2009 5 / 14
 Goals

Our Goal

Facilitate post-silicon verification by pre-silicon analysis.

Pre-silicon Models
• Allow complete visibility of internal state.
• Can be mathematically formalized analyzed and reasoned about.

We use pre-silicon analysis to determine post-silicon observation

points.
• Exploit the connection between pre- and post- silicon models.
• The number of observation points depends on the desired logical guarantee

Ray and Hunt (UT Austin) Post-silicon Verification November 18, 2009 5 / 14
 Goals

Our Goal

Facilitate post-silicon verification by pre-silicon analysis.

Pre-silicon Models
• Allow complete visibility of internal state.
• Can be mathematically formalized analyzed and reasoned about.

We use pre-silicon analysis to determine post-silicon observation

points.
• Exploit the connection between pre- and post- silicon models.
• The number of observation points depends on the desired logical guarantee

Eventual goal is a post-silicon verification methodology that

• provides high correctness assurance.
• helps comprehend post-silicon execution results.
• provides clear trade-offs between logical guarantees and DFD support.

Ray and Hunt (UT Austin) Post-silicon Verification November 18, 2009 5 / 14
 Goals

Overall Vision

External
Tools

ACL2 Modeling and Analysis

System
Guided
Proof
Design
AIG/BDD
AMS

RTL

Microcode Symbolic
Pre−silicon Post− Simulation
Verification silicon
Design Proof
Verification Representation Orchestration

Property/Annotation Information
Flow
Formal Design/Annotation
Database

Formal
Specification
Specification

We envision a single, unified, formal framework for specification,

evaluation, and verification of computing systems.
Ray and Hunt (UT Austin) Post-silicon Verification November 18, 2009 6 / 14
 Approach

An Approach: Partition Trace Analysis

Partition post-silicon trace analysis into two components.

small on-chip integrity unit that has full observability
an off-chip partial trace analyzer
The off-chip component can assume that in-silicon analysis has succeeded.
Formal analysis guarantees that the components together are
equivalent to a monitor that has full observability.
We applied the partitioning approach for post-silicon analysis of a
multiprocessor memory system.

Ray and Hunt (UT Austin) Post-silicon Verification November 18, 2009 7 / 14
 Approach

A Multiprocessor Memory System

CPU(0) CPU(1) ..... CPU(n−1) CPU(n)

Memory

Pre−silicon
Execution
Trace Monitor

The pre-silicon monitor checks for bounded coherence.

Has full observability of all bus transactions.
Obviously impractical for post-silicon.

Ray and Hunt (UT Austin) Post-silicon Verification November 18, 2009 8 / 14
 Approach

Post-silicon Analysis
A post-silicon trace is a subsequence of a pre-silicon trace with lossy
compression.

CPU(0) CPU(1) ..... CPU(n−1) CPU(n)

Memory
Integrity Unit Lossy Compression

Partial Execution Trace

Pre−silicon
Execution
Post silicon analyzer Trace Monitor

SAT solver

The integrity unit keeps track of internal bus transactions.

It is sufficient to externally observe only a small number of critical
events.
Ray and Hunt (UT Austin) Post-silicon Verification November 18, 2009 9 / 14
 Approach

Post-silicon Certification

Theorem. If the integrity unit does not interrupt, then any post-silicon
trace that passes the post-silicon analysis is a subsequence of a trace that
would pass pre-silicon analysis under full observability.
The theorem is proven is ACL2.
Makes use of underlying protocol invariants.

Ray and Hunt (UT Austin) Post-silicon Verification November 18, 2009 10 / 14
 Approach

Post-silicon Certification

Theorem. If the integrity unit does not interrupt, then any post-silicon
trace that passes the post-silicon analysis is a subsequence of a trace that
would pass pre-silicon analysis under full observability.
The theorem is proven is ACL2.
Makes use of underlying protocol invariants.
Proven by exploiting a decidable subclass of the logic.

Ray and Hunt (UT Austin) Post-silicon Verification November 18, 2009 10 / 14
 Approach

Post-silicon Certification

Theorem. If the integrity unit does not interrupt, then any post-silicon
trace that passes the post-silicon analysis is a subsequence of a trace that
would pass pre-silicon analysis under full observability.
The theorem is proven is ACL2.
Makes use of underlying protocol invariants.
Proven by exploiting a decidable subclass of the logic.

The theorem formally connects post-silicon verification with

pre-silicon analysis.

Ray and Hunt (UT Austin) Post-silicon Verification November 18, 2009 10 / 14
 Results

Using the System

The system can identify subtle design errors.

Ray and Hunt (UT Austin) Post-silicon Verification November 18, 2009 11 / 14
 Results

Using the System

The system can identify subtle design errors.

Processor 0 shared request

Processor 1 exclusive request

Processor 0 shared grant

Processor 2 exclusive request

Processor 1 exclusive grant

Such errors are very difficult to exercise in simulation because of the

non-determinism in the protocol.

Ray and Hunt (UT Austin) Post-silicon Verification November 18, 2009 11 / 14
 Results

Using the System

The system can identify subtle design errors.

Processor 0 shared request

Processor 1 exclusive request

Processor 0 shared grant

Such errors are very difficult to exercise in simulation because of the

non-determinism in the protocol.

The system identifies the error even under very poor observability.

Ray and Hunt (UT Austin) Post-silicon Verification November 18, 2009 12 / 14
 Related Work

Related Work

Gopalakrishnan and Chou: Limited observability checkers based on

constraint solving and abstract interpretation.
Aschlager and Wilkins: Model checking to generate a short trace
containing an observed bug.
Safarpour et al: SAT solving to automatically find and repair stuck-at
faults.
De Paula et al: SAT solving to develop a “backspace” from a crashed
state.
Our approach is to introduce some of the analysis or checking into
the silicon.

Ray and Hunt (UT Austin) Post-silicon Verification November 18, 2009 13 / 14
 Conclusion

Conclusion and Future Work

To our knowledge our work is the first effort on connecting

pre-silicon and post-silicon verification through formal proofs.
Provides a flexible mechanism for making use of pre-silicon analysis in
post-silicon verification.
Makes use of existing design artifacts to facilitate post-silicon
analysis.

Ray and Hunt (UT Austin) Post-silicon Verification November 18, 2009 14 / 14
 Conclusion

Conclusion and Future Work

To our knowledge our work is the first effort on connecting

pre-silicon and post-silicon verification through formal proofs.
Provides a flexible mechanism for making use of pre-silicon analysis in
post-silicon verification.
Makes use of existing design artifacts to facilitate post-silicon
analysis.
Of course, the results are preliminary.
Future work:
Exploit information flow for automatic signal winnowing.
Automate partitioning, given an observability and hardware bound.
Tighten connection between pre-silicon and post-silicon.
Exploit faster post-silicon simulation to facilitate pre-silicon analysis.

Ray and Hunt (UT Austin) Post-silicon Verification November 18, 2009 14 / 14