Verification Basics

Jean-Michel Chabloz

ASIC design flow

Goes in order through the following steps:
Specifications /System Design Register Transfer Level Netlist Layout Physical chip

ASIC design flow

From specs to RTL:
human translation

From RTL to Netlist to Layout to Physical:

mostly automated translation (solved problem) If RTL is bug-free, the physical chip will work

The main source of bugs is the specs-toRTL translation

Software design flow

Goes in order through the following steps:
Specifications High-level language (C, ...) Low-level language (assembly) Machine language

Only the specs-to-high language translation is made mostly by humans If high-level language is correct, the machine language will work

Software implementation flow

The high-level language to machine-language flow is:

Fast Inexpensive

Software flow:
1) 2) 3) 4) 5) 6) 7) Write HL language Translate HL language to machine language Run machine language and find bugs Fix HL language Translate HL language to machine language Run machine language and find bugs Repeat from 4 until finished

FPGA implementation flow

Can use something similar to software:
1) 2) 3) 4) 5) 6) 7) Write RTL language Translate RTL language to gates and implement in FPGA Run FPGA language and find bugs Fix RTL language Translate RTL language to gates and implement in FPGA Run FPGA and find bugs Repeat from 4 until finished

ASIC implementation flow

1) 2) 3) 4) 5) 6) Write RTL language Simulate RTL and find bugs Fix RTL language Simulate RTL and find bugs Repeat from 3 until finished Synthesize, make layout, implement chip

Validation vs Verification
Validation:
are we making the right thing, that will answer to the needs of the user? are the specs right?

Verification:
are we making what we wanted to make? is the model equivalent to the specs?

We consider only verification

Verification plan
Plan of all what should be verified in a DUT Should include all possible features and potential sources of bugs When all tests in the verification plan have been tested and no bugs were found, then the verification work is over

Checking RTL to spec equivalence

The person (team) who wrote the RTL must not be the one who checks it correctness Bugs might not be found due to doublemistake in interpreting the specs In case of error, the mistake in interpreting the specs might be due to both the designer and the verifier

Formal vs simulation-based verification

Formal verification is a new paradigm:
Tools prove that the RTL is equivalent to a high-level model or that it satisfies certain properties No need for simulation Might one day totally substitute simulationbased verification

We consider only simulation-based verification

Verification models
Basic model: give inputs, check outputs black box verification We might use white-box verification (give inputs, check internal signals) Or grey-box verification (give inputs, check some internal signals specifically inserted for debug purpose)

Verification
~70% of effort when developing RTL trend: growing Testbenches are more complex than RTL models Growth of testbench complexity is more than linear with RTL complexity.
example:
10 state machines with 4 states: 4^10 total states 20 state machines with 4 states: 4^20 total states 10 state machines with 8 states: 8^10 total states

Have to test the RTL model under situations similar to those that the manufactured chip will encounter during use (have to develop a model of the universe)

Needs of verification language

very different needs from RTL
RTL: language must be simple enough for the stupid synthesis tools to understand them and know how to synthesize them
ex fifo: put a RAM, use pointer A, pointer B, increment pointers when reading or writing

Verification: language must be super-powerful to be able to implement quickly and efficiently the model of the universe, no need to be understood by synthesis tools
ex fifo: virtual storage with push and pop

Hardware description languages

VHDL/Verilog developed in the 1980s good languages for RTL design synthesizable code very few non-synthesizable constructs for writing testbenches

Hardware verification languages

e/Vera Developed in the 1990s Only high-level constructs, impossible to write RTL code People had to mix VHDL/Verilog RTL models with e/Vera testbenches Three main features lacking in HDLs:
Random constrained stimuli generation Assertions Functional coverage

Random Constrained Stimuli Generation

Not to be confused with the random construct in Verilog/VHDL Main idea:
define random variables and constraints ask the random solver to find a random set of variables that satisfies the constraints constraints can be added, disabled to create different tests

Random Constrained Stimuli Generation

example:
random bit a random integers b, c between 0 and 255 constraint A: if a=1 then (b+c)!=256 constraint B: b>c

It would be hard to make a routine to randomly generate one of the legal combinations using only direct randomization of variables

Assertions
Tools for automatic checking of properties automated waveform checker Example:
when req goes at one grant must be 1 between 2 and 3 cycles later req must never be at one for more than two consecutive cycles

Can be used in testbenches or bundled with the RTL to check input correctness

Functional Coverage
Testing if the testbench is good enough Did we do all the tests that we wanted to do based on our verification plan? Example:
A crossbar can put in correspondence all inputs with all outputs Did we try all combination of input/outputs? With functional coverage we can record how many times all combinations of input/outputs were tested, and see the results in a report

Code Coverage
Except functional coverage, there are other coverage metrics that are tool features and can be used independently on the language Do not require to write code to enable coverage Statement coverage: has every statement in the DUT be executed? Path coverage: have all paths been followed? Expression coverage: have all causes for control-flow change been tried? FSM coverage: has every state in an FSM be visited?

Code coverage Statement coverage

y y y y y y n y y y y y if (a>1 || b>1) begin c <= d; d <= d+1; end else begin if (a==2) begin d <= d-1; end else d <= d-2; end end

Code coverage Path coverage

if (a>1 && b>1) begin c <= d; d <= d+1; end if (a>2) begin if (a==3) begin d <= d-1; end else d <= d-2; end end

Run 1

Code coverage Path coverage

if (a>1 && b>1) begin c <= d; d <= d+1; end if (a>2) begin if (a==3) begin d <= d-1; end else d <= d-2; end end

Run 2

Code coverage Path coverage

if (a>1 && b>1) begin c <= d; d <= d+1; end if (a>2) begin if (a==3) begin d <= d-1; end else d <= d-2; end end

At the end of all the runs, we find out this legal path was not exercised

Code coverage expression coverage

if ((a>1 && b>1) || (a<0) || (b<0)) begin c <= d; d <= d+1; end

All statements were executed (100% statement coverage), but not all values for the expressions became true

Code coverage - FSM coverage

Checks if all states in a state machine were exercised

Directed tests
Testbenches without randomness, targeting a specific item in the verification plan. Example:
write in the fifo for 16 cycles after each other, check that the fifo is full, then read all 16 elements, check that it is empty

If the design is complex enough, it is impossible to cover all features with directed testbenches

Random verification
1) Generate random tests using random constrained stimuli generation 2) Check for bugs and correct them if there are 3) Check for the coverage values. If not satisfying, add constraints and repeat from 1 Note: some directed testbenches might be necessary to cover the corner cases

SystemVerilog
Hardware description and verification language Superset of Verilog all Verilog systems work in SystemVerilog First standardized by IEEE in 2005 IEEE standard 1800-2009 21st february 2013: IEEE standard 1800-2012
Download the standard Holy book of SystemVerilog Answer to all of your questions are inside Good for reference, not for learning

SystemVerilog is the standard IEEE 1800-2012

Simulators/Synthesizers implementations might be incomplete

SystemVerilog
RTL subset of the language:
small superset of the Verilog RTL subset some constructs have been inserted to simplify different things

Verification subset of the language:

very very very big superset of Verilog
Object-oriented constructs Random constrained stimuli generation Assertions Functional Coverage

SystemVerilog
SystemVerilog testbenches can also be used to test VHDL/Verilog RTL models mixed-language simulation

Testbenches
Basic model: give inputs, read outputs
Inputs generator RTL model Outputs checker

The element to test is called DUT (Design Under Test) or DUV (Design Under Verification)

Testbenches
Better structure:
Generator of High-level inputs Outputs checker Checks outputs with Expected HL outputs

Inputs driver Gives the inputs to The DUT

RTL model

Outputs monitor Reads the outputs and Translates into HL

Testbenches
Often:
Generator of High-level inputs Golden model (matlab, TLM, Timed, untimed, ) Outputs checker Checks outputs with Expected HL outputs

Inputs driver Gives the inputs to The DUT

RTL model

Outputs monitor Reads the outputs and Translates into HL

Golden model and RTL must be developed by different teams, errors might

SoC Verification
Collection of IPs Each IP must first be verified at block-level Then top-level verification follows Verification systems for IPs are packaged into VIPs (verification IPs), with drivers, monitors, assertions to check input correctness, high-level models, etc. A scoreboard keeps track of which tests have been run and coverage Possible to build a chip in which only some components are RTL, the others are golden models Using VIPs it is easy to build fast complex models of what is around a block or a chip

UVM
Universal Verification Methodology Methodology on top of SystemVerilog that automates all this Key focus: reuse Components are enclosed into agents, containing checkers, monitors, drivers, etc. A chip can be built connecting together the different VIPs We do not consider UVM, it is only adapted to complex systems.