Reusable Verification Environment For Verification of Ethernet Packet in Ethernet IP Core, A Verification Strategy-An Analysis
Reusable Verification Environment For Verification of Ethernet Packet in Ethernet IP Core, A Verification Strategy-An Analysis
Summary
Design reuse and verification reuse are important to satisfy time- 1. Introduction
to-market requirements. Designer must be able to reuse
Intellectual Property in the design as golden model. Reuse of Verification is a methodology used to demonstrate the
verification environment across different designs of the domain
functional correctness of a design. With automation
saves time to market further and improves total design
verification quality. The Physical Layer is a fundamental layer
human errors in a process are minimized. Automation
upon which all higher level functions in a network are based. takes human intervention completely out of the process [1,
However, due to the plethora of available hardware technologies 2, 3]. However, automation is not always possible,
with widely varying characteristics, this is perhaps the most especially in processes that are not well defined and
complex layer in the OSI architecture. The implementation of this continue to require human ingenuity and creativity, such as
layer is often termed Physical layer device (PHY). The Physical hardware design.
Layer defines the means of transmitting raw bits rather than
logical data packets over a physical link connecting network Another possibility is error due to human intervention by
nodes. A PHY chip is commonly found on Ethernet devices. Its
reducing it to simple and foolproof steps. Human
purpose is digital access of the modulated link and interface to
Ethernet Media Access Control (MAC) using media independent
intervention is needed only to decide on the particular
interface (MII) interface. This paper discusses Verification sequence or steps required to obtain the desired results. It
process, issues involved in verification process and Test is usually the last step toward complete automation of a
Methodologies. A broad outline of the comparison of traditional process. However, just like automation, it requires a well-
verilog and specman verification methodologies has been defined process with standard transformation steps. The
presented here. It also explains verification strategy and reuse of verification process remains an art that, to this day, does
design environment with reference to verifying the Ethernet not yield itself to well-defined steps.
packet in Ethernet Intellectual Property (IP) Core. Design Reuse
is achieved through verilog tasks which were used in specman
Choosing the common origin and reconvergence points
environment. Ethernet Phy e Verification component (eVC) is an
in house development. Ethernet eVC is built with phy as a
determines what is being verified. These origin and
separate eVC and host being a task driven verilog Bus functional reconvergence points are often determined by the tools
model (BFM). This allowed us to create a virtual host used to perform the verification. It is important to
environment using a combination of verilog BFM and eVC. understand where these points lie to know which
Verification environment reuse for different application with transformation is being verified.
different interface is done by developing a wrapper around the
Design Under Test (DUT) interface and then interfacing it to the The main purpose of functional verification [9] is to
environment. A detailed test plan is made for the complete and ensure that a design implements intended functionality.
exhaustive test for Ethernet MAC Receiver. Coverage goals, without functional verification, one must not trust that the
coverage obtained and coverage analysis indicate efficiency of
the verification methodology.
transformation of a specification document into design and
Register Transfer Logic (RTL) code was performed
correctly, without misinterpretation of the specifications.
Key Words:
Figure 1 shows an overview of the steps involved in Those components can then be used to verify multiple
verification of a design under test. entities which have a particular interface. eVCs are
verification components written in the e verification
Verification activity consists of driving vector stream into language [11]. The e language is designed specifically for
device and checking the vector stream coming out of the verification. Reuse and extensibility are fundamental e
device. Higher level languages [12] are needed, as High- language design principles.
level language eases the creation of an expected value
generator. Data check verifies the data correctness while The document is organized as follows: It discuss
temporal check verifies timing and protocol. Here the Verification process in Section 2, issues in verification in
shared objects are used for input stimulus. It supports Section 3, issues in verification methodologies in section 4,
cycle-based behavior, events, and synchronizes with HDL Test methodologies in section 5, Specman based
simulator. Verification in section 6 and Traditional verification based
methodology in section 7. Section 8 presents a detailed
case study with reference to a reusable Verification
Environment for verifying Ethernet packet in Ethernet IP
core. It discusses aspects such as, Management Data Input
Output (MDIO), Verification strategy, Macro language,
Test cases, Test Plan for Ethernet MAC receiver, Coverage
Goals and the Bug file. Finally, Section 9 presents the
conclusions drawn from the entire work.
2 Verification process
Collecting
Checking Output
Cover
Plan
Coverage
Figure 1. Generalized Verification Flow
Perhaps the most important problem faced by design and Pre-run generation is a newer methodology for generating
verification engineers is the lack of effective metrics to tests that addresses some of the productivity problems
measure the progress of verification. Indirect metrics, such associated with deterministic testing by automating the test
as toggle testing or code coverage, indicate if all the flip- generation process. C or C++ programs (and sometimes
flops are toggled or all lines of code were executed, but even VHDL and Verilog, despite the lack of good software
they do not give any indication of what functionality was constructs) are usually used to create the tests prior to
verified. For example, they do not indicate if a processor simulation. The programs read in a parameter/directives
executed all possible combinations of consecutive file that controls the generation of the test. Often these files
instructions. There is simply no correspondence between contain simple weighting systems to direct the random
any of these metrics and coverage of the functional test selection of inputs.
plan. As a result, the verification engineer is never really
sure whether a sufficient amount of verification has been The generator normally outputs the test into a file, which
performed. is then read by the simulator and stored in memory. The
simulator reads the next entry whenever it is prepared to
IJCSNS International Journal of Computer Science and Network Security, VOL.8 No.11, November 2008 229
inject the next set of inputs. Although pre-run generation model in C or C++. Stimuli are injected into the reference
provides much higher throughput than deterministic testing, model as well as the device, and their outputs are
it is difficult to control. The parameters are static and do compared. In gray and white-box methodologies, the
not provide much flexibility. Also, most generators of this comparisons also include the state of internal registers and
type do not allow interdependencies between data streams nodes.
if each data stream is generated independently. This can
cause the generator to generate unlikely or even illegal Rule-based approaches are more common in
tests. communication devices for networking applications, where
there can be several legal outputs for the same input, it is
Reaching corner cases using pre-run generation is nearly not easy to predict the correct result. In this case, the
impossible. The engineer has very little control over the engineer often uses specialized techniques to check data
sequences generated. This makes it difficult to force the integrity and protocols, such as scoreboarding, which
occurrence of specific combinations. As such, pre-run tracks information about cells or packets without worrying
generation makes a suitable complement to deterministic about the order in which they appear on the output ports.
testing, but cannot replace it.
Engineers perform these checks either on-the-fly or post-
Another problem with pre-run generation is that it is hard run. Simple checks and protocol checks can be performed
to maintain. As the verification process progresses, new on-the-fly by the stubs and monitors using an HDL. Post-
parameters are often needed. This normally requires run checks are often performed using a C/C++ or PERL
modifying the program, sometimes affecting delicate program. The outputs of the test are either saved in a
interdependencies between different parts of the generator. simulator memory and then dumped into a file, or written
into the file directly. The program reads the inputs ,
Maintenance problems can also occur when updating the outputs and checks the correctness of the results. Often,
program after a bug is found in the RTL design. To these methodologies still require some amount of manual
temporarily avoid generating the same bug test sequence checking, usually achieved by viewing actual waveforms
again, the engineer must modify the code until the bug is or data dumps.
fixed in the RTL design. When the bug is fixed, the code
must be again remodified. Several such modifications The problem with these checking strategies stems from
often coexist in the code. The modifying and remodiying the way they are most commonly implemented today.
process sometimes introduces bugs into the generator, Post-run checking wastes cycles. If a test runs for 500,000
which may not be noticed until several hours or days of cycles, but a bug occurred after cycle 2,000, then 498,000
simulation have transpired. The cost of developing and cycles were wasted. In addition, since the post-run
maintaining such a generator requires a minimum of checking cannot detect a problem in real-time, the designer
several man-months per project, and increases significantly does not have access to the values of the registers and
as the generation becomes more complex. memories of the device at the time the problem occured. In
general, debugging these problems requires rerunning the
A side-effect of this methodology is that the full test is simulation to the appropriate point.
usually very large, since it is generated in advance. It is
commonly loaded into a simulation memory at the On-the-fly checking is more powerful. However, on-the-
beginning of the test and run from there. This significantly fly checks are most often implemented in Verilog or
increases the memory requirements for simulation, often VHDL. These languages do not have a powerful temporal
causing the simulator to swap memory. This can slow language to simplify protocol checks. They are low level
down the simulation by orders of magnitude. and lack features like dynamic memory, which simplifies
the process of writing the stubs/monitors and increases
5.3 Checking Strategies performance.
The two most popular ways to determine test results are In addition, reference model checking is often hard to
to compare them to a reference model or to create rule- implement on-the-fly, since intermediate results are not
based checks. Both of these checking methods must always available. On-the-fly reference models also require
include both the temporal behavior and protocols of the a direct interface to the simulator (through PLI or FLI)
device as well as the verification of data. which is not easy to write and maintain.
Reference models are most common for processor-like 5.4 Coverage Metrics
designs where the correct result can be predicted with
relative ease. Designers usually develop the reference Measuring progress is one of the most important tasks in
230 IJCSNS International Journal of Computer Science and Network Security, VOL.8 No.11, November 2008
process has reached a point of diminishing returns. Error injector Error injector
Collecto Collecto
r r
Unfortunately, none of the metrics described above has
any direct relation to the functionality of the device, nor is
there any correlation to common user applications. Neither
Monitor
Monitor
toggle testing nor code coverage can indicate if all the
types of cells in a communication chip with and without Score Board
Cyclic Redundancy Check (CRC) errors have entered on
all ports. These metrics cannot determine if all possible
sequences of the instructions in a row were tested in a Figure 3. Generalized Verification environment
processor.
• Provides functional coverage analysis capabilities to
As a result, coverage is still measured mainly by the gut help measure the progress and completeness of the
feeling of the verification manager, and eventually the verification effort.
decision to tape out is made by management without the
support of concrete qualitative data. Not knowing the real Raises the level of abstraction used to describe the
state of the verification progress causes verification environment and tests from the RTL level to the
engineers to perform many more simulations than specification level, capturing the rules defined in the specs
necessary, trading off CPU cycles for "confidence". This in a declarative form and automatically ensuring
usually results in redundant tests that provide no additional conformance to these rules.
coverage or assurance that verification is complete. The
real risk is that the design will be sent to production with Figure 3 gives a generalized verification environment
bugs in it, resulting in another round of silicon. The cost of which can be used as guideline environment for
re-spinning silicon includes non-recoverable engineering developing verification environment for most of the
(NRE) costs to do the additional production process, the designs. The environment consists of the components,
cost of extending the teams work on the project, and the Input BFM driver, Collector, Coverage, Test case
major cost of reaching the market a few weeks late. generator, Error injector, constraints, Scoreboard and
Monitor.
6. Specman based verification
7. Traditional Verification Methodology
Specman based verification is a methodology for
Some important points are:
functional verification that solves many of the problems of
design and verification engineers encountered with today's
7.1 Productivity & Quality Issues
methodologies. This is done by capturing the rules
embodied in the specifications (design/ interface/
Verification is more than 50% of an overall project cycle.
functional test plan) in an executable form. An effective
It May require tens of thousands of lines of verification
application of this methodology provides four essential
code. Design spec changes cause major verification delays.
capabilities to help to break through the verification
Implementing all identified tests in test plan within the
bottleneck.
project schedule is the Productivity issue.
IJCSNS International Journal of Computer Science and Network Security, VOL.8 No.11, November 2008 231
7.2 Requirement for Productivity Improvement MDIO, Direct Memory Access (DMA) support,
Configuration registers, Control logic, Transmitter FSM
The verification environment must be created and/or and Receiver FSM.
maintained efficiently. Human should spend more time at
higher level details providing simulation goals, analyzing
errors reported by checkers and providing more direction
when goals are not being met. 8.1 MDIO
The MDIO is a simple serial interface between the host
and an external PHY device. It is used for configuration
7.3 Quality Issues and status read of the physical device. A host processor
responsible for system configuration and monitoring
Verification complexity makes it a challenge to think of all typically uses the MDIO to perform individual accesses to
possible failure scenarios. It does not provide a way to try the various PHY devices.
scenarios beyond the expected failure scenarios.
It implements the IEEE 802.3 Clause 22 standard MDIO
7.4 Requirement for Quality Improvement interface used in Ethernet systems up to 1Gbit/s. MDIO
Master core allows access to registers within multiple
Confidence about the ratio of identified bugs must connected Slaves. The features such as simple register
increase. An automatic way to know what has been tested based user application interface for the MDIO, MDIO
must be available. frame generation with serial port tristate control, busy
indication to user application during ongoing transaction
7.5 Task-based Strategy are provided. PHY interrupt goes active when status
change is indicated to application.
To improve test-writing productivity higher level of
abstraction is used for specifying the vector stream, and Host initiates an operation by writing into the
higher-level tasks are created in HDL or C. Task-based configuration registers of the MAC. MDIO reads these
strategy limitations are, high test writing effort, many registers and performs the tasks. It then reports it to the
parameters values must be selected manually and high - host by writing into the configuration registers which is
level intent is not readily apparent. polled by the Host continuously.
There is no need to buy new tools or licenses and it
provides homogeneous environment.
MAC
8. Case study:
HOST
Configur
A Verification Environment for verifying ation
MII/GMII
The control block consists of counters which generates quickly as possible. The verification environment designed
enable signal for the functioning of mux, demux, SIPO and is shown figure 5.
PISO. It also generates the status signals to be written into
the configuration registers. It also generates the PHY Same components of the existing environment can be
enable signal to drive PHY data as the data line is a shared used for different application with different interface by
tri-stateable bus, which is driven by the MAC for write developing a wrapper around the DUT interface and then
transactions or by the PHY devices during read. interfacing it to the environment. Verification environment
consists of BFM, test case generator, monitor and checker.
The clock generator module generates Management Data
Clock by dividing host clock. The division factor is set in As with any commercial IP or SOC development with an
the configuration register field. aggressive timescale, the minimization of risk is key to
delivery. For many commercial developers the decision to
8.2 Verification strategy adopt a new verification paradigm can seem too risky.
Verification strategy for the Ethernet core was fairly The Specman Elite environment was built with the PHY
sophisticated. The design was very complex and the eVC and the host eVC with few verilog tasks. This
verification team was tasked with ensuring as high a allowed us to create a virtual host environment using e
quality device as possible. Also, the verification team had masters, slaves and bus arbitrators.
the requirement that the environment lend itself easily to
reuse for future generations, and that engineers who didn't
create the environment be able to be productive within it as
T A S K L IB R A R Y (G E N E R A L P U R P O S E )
T r a n s c a tio n C o v e ra g e D U T
G e n e ra to r
T e s tC a s e M a s te r
PCI BUS
IP C o re
G /T B F M
D a ta
G e n e ra to r P IB
M o n it o r
D a ta C h e c k e r
(S c o re B o a rd )
Specman Elite automated key aspects of the verification Just like an IP design core, this allowed time to be spent on
environment. Specifically, it works by generating stimuli verifying the unique aspects of the design rather than the
into the device, inputs which can be fully random or fully standard components.
directed. In this way, we can reach specific corner cases in
the design without having to force a specific state. We can 8.3. Macro language
generate the inputs pre-run, on the fly, or a combination of
the two.
One of the major objectives for the verification
environment was the ability for the non-Specman "savvy"
. An eVC is a reusable piece of verification code written in engineers to be able to easily use the environment to
e verification language with ERM (e reusable develop tests. We achieved this through “verilog tasks"
methodology) methodology. Specifically, the PHY eVC is that were built in a layered way using Specman Elite
a reusable verification environment developed in house.
IJCSNS International Journal of Computer Science and Network Security, VOL.8 No.11, November 2008 233
macros. In fact, a huge number of the Ethernet verification displays a detailed plan for verification of MAC receiver
tests are sequences of code similar to the above. stages.
Monitor
8.4 Test Cases
8.5. Test Plan for the Ethernet MAC Receiver IN RCB Bit50
if '1' if '0'
Various Environment components used in the verification
of Receiver (Figure 6) are, Receiver DUT, BFM to drive Half Duplex Full Duplex
different types of frames and various Inputs to the DUT,
Monitor to monitor the various inputs and outputs of the Datawidth=4 bits from Phy Datawidth=8Bit from Phy
Receiver DUT, Collector to collect the data from output of
the DUT and Scoreboard to compare the data driven by the
BFM and the output of the DUT. Figures 7, 8, 10 to 12 Figure 8.Receiver for modes MII/GMII at different
display the details of configuration for test plan. Figure 9 speeds
234 IJCSNS International Journal of Computer Science and Network Security, VOL.8 No.11, November 2008
dont receive
receiver enable(bit52=1)
frames
yes
Individual
Multicast Broadcast
Address
addressed field addressed field
Field
if too short frame Data Frame Control Frame Too long frame VLAN
Check the
In Full Duplex Mode
Allignment of Length Error
Only
Octets
yes No
yes if full
to transmiiter no
Length
Data
to mac client
Coverage goals
Half Duplex Full duplex The line coverage to be achieved is 100%.
The condition coverage to be achieved is 85%.
The FSM coverage to be achieved is 100%.
If RXRAM Full
The toggle coverage to be achieved is 90%.
The average coverage to be achieved is 90%.
Half Duplex Full duplex
Acknowledgements
Ms.L.Swarna jyothi is working currently as a Professor Mr. Harish has 13 years of experience in reputed
of Information Science and Engg Department at JSS industries, out of which 8 years in
Academy of Technical Education, Bangalore Visvesvaraya ASIC/VLSI/Verification/Testing/Design Involving
Technological University, INDIA. She has a total of ASICs, Models, and SoCs. He has experience in test plan
25years of academic experience. Her research interests are development, environment development, test cases
assertion-based verification, design for verification, implementation, models development, random testing and
verification reuse and VLSI Education. She has six reputed coverage. His Verification Experience include Hyper
conference Publications in the related field. Her research Transport Tunnel/Cave, Controller Area Network (CAN),
has been funded by All India Council for Technical PCIe, GB Ethernet, Flexray. He has expertism in Verilog,
Education under Research Promotion scheme. She has Specman e, ESEPro. VCS, Ncsim, Verilog-XL and
already coordinated three projects sanctioned by AICTE Specview..His education includes Bachelor of engineering
and project grants are about 30 lakhs. She has received her in Telecommunications and Master of Business
Bachelors Degree in Engineering in Electronics and Management from Bangalore University, Bangalore.
Communication Engg from Bangalore University and Hereceived his MS (Software Systems) Degree from BITS,
Masters Degree in Engineering in Comp Sc and Engg Pilani, India
from Anna University, Chennai, INDIA. She is pursuing
her PhD in Dr.MGR Research and Educational Institute, Dr.A.S. Manjunath is the Managing Director and CEO of
Chennai Manvish eTech Pvt, Ltd, Bangalore, INDIA. He has been a
successful entrepreneur for over 12 years. He has done his
PhD and UG degrees are from Bangalore University. His
Masters Degree is form Mysore University India. He has
worked as Professor and as an academician for over 13
years, in Bagalore University, India. He is guiding three
PhD scholars and has 22 conference/journal papers to his
credit.