VLSI Design Verification - Concordia...
Transcript of VLSI Design Verification - Concordia...
-
VLSI Design Verification: VLSI Design Verification: VLSI Design Verification: VLSI Design Verification: Challenges and StateChallenges and State--ofof--thethe--artart
P fP f S fiS fi T hT hProf. Prof. SofieneSofiene TaharTahar
D t t fD t t fDepartment of Department of Electrical and Computer EngineeringElectrical and Computer Engineering
Concordia UniversityConcordia UniversityMontreal Quebec CANADAMontreal Quebec CANADAMontreal, Quebec, CANADAMontreal, Quebec, CANADA
-
1
1
Design Verification
Mike Butts
Synopsys
Prof. Kurt Keutzer
Dr. Serdar Tasiran
EECS
UC BerkeleyMike Butts
Design Process
Design : specify and enter the design intent
Implement:refine the design through all phases
Verify:verify the correctness of design and implementation
-
2
Design Verification
RTLSynthesis
HDL
netlist
logicoptimization
netlist
Library/modulegenerators
physicaldesign
layout
manualdesign
specification
Is the design
consistentwith the originalspecification?
Is what I think I wantwhat I really want?
Kurt Keutzer 4
Implementation Verification
RTLSynthesis
HDL
netlist
logicoptimization
netlist
Library/modulegenerators
physicaldesign
layout
manualdesign
Is the implementation
consistentwith the originaldesign intent?
Is what I implemented
what Iwanted?
a
b
s
q0
1
d
clk
a
b
s
q0
1
d
clk
-
2
Kurt Keutzer 3
Design Verification
RTLSynthesis
HDL
netlist
logicoptimization
netlist
Library/modulegenerators
physicaldesign
layout
manualdesign
specification
Is the design
consistentwith the originalspecification?
Is what I think I wantwhat I really want?
Implementation Verification
RTLSynthesis
HDL
netlist
logicoptimization
netlist
Library/modulegenerators
physicaldesign
layout
manualdesign
Is the implementation
consistentwith the originaldesign intent?
Is what I implemented
what Iwanted?
a
b
s
q0
1
d
clk
a
b
s
q0
1
d
clk
-
3
Manufacture Verification (Test)
RTLSynthesis
HDL
netlist
logicoptimization
netlist
Library/modulegenerators
physicaldesign
layout
manualdesign
Is the manufactured
circuitconsistent
with the implemented
design?
Did theybuildwhat I
wanted?
a
b
s
q0
1
d
clk
a
b
s
q0
1
d
clk
Kurt Keutzer 6
Design Verification
RTLSynthesis
HDL
netlist
logicoptimization
netlist
Library/modulegenerators
physicaldesign
layout
manualdesign
specification
Is the design
consistentwith the originalspecification?
Is what I think I wantwhat I really want?
-
4
Verification is an Industry-Wide Issue
Intel: Processor project verification: “Billions of generated vectors”“Our VHDL regression tests take 27 days to run. ”
Sun: Sparc project verification: Test suite ~1500 tests > 1 billion random simulation cycles“A server ranch ~1200 SPARC CPUs”
Bull: Simulation including PwrPC 604“Our simulations run at between 1-20 CPS.” “We need 100-1000 cps.”
Cyrix : An x86 related project“We need 50x Chronologic performance today.”“170 CPUs running simulations continuously”
Kodak: “hundreds of 3-4 hour RTL functional simulations”Xerox: “Simulation runtime occupies ~3 weeks of a design cycle”Ross: 125 Million Vector Regression tests
Design Teams are Desperate for Faster SimulationDesign Teams are Desperate for Faster Simulation
Kurt Keutzer 8
Verification Gap
1
Log
ic T
rans
istor
s per
Chi
p(K
)
Prod
uctiv
ityTr
ans./
Staf
f -M
o.
10
100
1,000
10,000
100,000
1,000,000
10,000,000
10
100
1,000
10,000
100,000
1,000,000
10,000,000
100,000,000Logic Tr./ChipTr./S.M.
58%/Yr. compoundComplexity growth rate
21%/Yr. compoundProductivity growth rate
Source: SEMATECH1
981
1983
1985
1987
1989
1991
1993
1995
1997
1999
2003
2001
2005
2007
2009
xxx
x xx
x
2.5µ
.10µ
.35µVerification Gap
-
4
Kurt Keutzer 7
Verification is an Industry-Wide Issue
Intel: Processor project verification: “Billions of generated vectors”“Our VHDL regression tests take 27 days to run. ”
Sun: Sparc project verification: Test suite ~1500 tests > 1 billion random simulation cycles“A server ranch ~1200 SPARC CPUs”
Bull: Simulation including PwrPC 604“Our simulations run at between 1-20 CPS.” “We need 100-1000 cps.”
Cyrix : An x86 related project“We need 50x Chronologic performance today.”“170 CPUs running simulations continuously”
Kodak: “hundreds of 3-4 hour RTL functional simulations”Xerox: “Simulation runtime occupies ~3 weeks of a design cycle”Ross: 125 Million Vector Regression tests
Design Teams are Desperate for Faster SimulationDesign Teams are Desperate for Faster Simulation
Verification Gap
1
Log
ic T
rans
istor
s per
Chi
p(K
)
Prod
uctiv
ityTr
ans./
Staf
f -M
o.
10
100
1,000
10,000
100,000
1,000,000
10,000,000
10
100
1,000
10,000
100,000
1,000,000
10,000,000
100,000,000Logic Tr./ChipTr./S.M.
58%/Yr. compoundComplexity growth rate
21%/Yr. compoundProductivity growth rate
Source: SEMATECH1
981
1983
1985
1987
1989
1991
1993
1995
1997
1999
2003
2001
2005
2007
2009
xxx
x xx
x
2.5µ
.10µ
.35µVerification Gap
-
5
logic_transistors
chipX
lines_in_design
logic_transistors
bugs
line_of_designX
=bugs
chip
Why the Gap?
Kurt Keutzer 10
logic_transistors
chipX
lines_of_design
logic_transistors
bugs
lines_of_designX
10,000,000 trs
chipX
1
10
1
10,000X
=100 bugs
chip
Filling in Reasonable Numbers
-
5
Kurt Keutzer 9
logic_transistors
chipX
lines_in_design
logic_transistors
bugs
line_of_designX
=bugs
chip
Why the Gap?
logic_transistors
chipX
lines_of_design
logic_transistors
bugs
lines_of_designX
10,000,000 trs
chipX
1
10
1
10,000X
=100 bugs
chip
Filling in Reasonable Numbers
-
6
logic_transistors
chipX
lines_of_design
logic_transistors
bugs
lines_of_designX
10,000,000 trs
chipX
1
100
1
10,000X
=10 bugs
chipthis year!!
Raising the Level of Abstraction
Kurt Keutzer 12
logic_transistors
chipX
lines_of_design
logic_transistors
bugs
lines_of_designX
100,000,000 trs
chipX
1
100
1
10,000X
=100 bugs
chipwithin 5 years!!
Moore’s Law Implies More Bugs
-
6
Kurt Keutzer 11
logic_transistors
chipX
lines_of_design
logic_transistors
bugs
lines_of_designX
10,000,000 trs
chipX
1
100
1
10,000X
=10 bugs
chipthis year!!
Raising the Level of Abstraction
logic_transistors
chipX
lines_of_design
logic_transistors
bugs
lines_of_designX
100,000,000 trs
chipX
1
100
1
10,000X
=100 bugs
chipwithin 5 years!!
Moore’s Law Implies More Bugs
-
7
The Verification Bottleneck
Verification problem grows even faster due to the
combination of increased gate count and increased vector count
1990
1996
2002
1M
100M
10B
100k 10M1M
10,0
00x
mor
e V
ecto
rsR
equi
red
to V
alid
ate
100 x 10,000 = 1 million times more Simulation Load
100x Gate Count
Kurt Keutzer 14
1 million instructions, assume 2 million cycles
Today’s verification choices:
50M cps: 40 msec Actual system HW
5M cps: 400 msec Logic emulator 1 (QT Mercury)
500K cps: 4 sec Cycle-based gate accelerator 1 (QT CoBALT)
50K cps: 40 sec Hybrid emulator/simulator 2 (Axis)
5K cps: 7 min Event-driven gate accelerator 2 (Ikos NSIM)
500 cps: 1.1 hr
50 cps: 11 hr CPU and logic in HDL simulator 3 (VCS)
5 cps: 4.6 days
1: assumes CPU chip 2: assumes RTL CPU 3: assumes HDL CPU
Time to boot VxWorks M. Butts - Synopsys
-
Simulation
• Traditionally used to check for correct operation of systems
• Use oftest benches(set of input vectors, expected outputs, environment constraints, etc.).
Behavioral Specification
Register Transfer Description
Logic Gates Description
Layout
Simulation
Simulation
Simulation
Extraction
Compare Results
Verify (validate)some properties
of the “concept”
Compare Results
Compare Results
-
Verification by Simulation (cont’d)
The “standard” verification technique is testing (simulation), but
Program testing can be a very effective way to show the presence of bugs, but it ishopelessly inadequate for showing their absence.
Edsgar W. Dijkstra
Bottom line: Not feasible to simulate all input sequences tocompletely verify a design.
Example 1:Suppose you want to test a 64-bit floating-point division routine. There are 2128
combinations. At 1 test/µs, it will take 1025 years
Example 2:How long does it take to exhaustively simulate a 256 bit RAM?
2256=10 80 combinations of initial states and inputs
Assume
- Use all the matter in our galaxy (1017kg) to build computers
- Each computer is of the size of single electron (10-30 kg)
- Each computer simulates 1012 per second
⇒ 10 10 years will reach0.05% of test cases!
-
Verification by Simulation (cont’d)
The “standard” verification technique is testing (simulation), but
Program testing can be a very effective way to show the presence of bugs, but it ishopelessly inadequate for showing their absence.
Edsgar W. Dijkstra
Bottom line: Not feasible to simulate all input sequences tocompletely verify a design.
Example 1:Suppose you want to test a 64-bit floating-point division routine. There are 2128
combinations. At 1 test/µs, it will take 1025 years
Example 2:How long does it take to exhaustively simulate a 256 bit RAM?
2256=10 80 combinations of initial states and inputs
Assume
- Use all the matter in our galaxy (1017kg) to build computers
- Each computer is of the size of single electron (10-30 kg)
- Each computer simulates 1012 per second
⇒ 10 10 years will reach0.05% of test cases!
-
8
Aspects of Design Verification
Event Driven
– Interactive Phase– High flexibility– Quick turnaround time– Good debug capabilities
Cycle -based simulation
– Regression Phase– Highest performance– Highest capacity
Emulation and Acceleration
– In-System Verification– Highest performance – Highest Capacity– Real system environment
Emulation
Cycle-basesimulation
Specification
Validation
Specification
Validation
Functional
Verification
(interactive)
Functional
Verification
(interactive)
Implementation
Verification
Implementation
Verification
Functional
Verification
(regressions)
Functional
Verification
(regressions)
In-System
Verification
In-System
Verification
Equivalence Checking
Event-driven Simulation
Kurt Keutzer 16
Software Simulation
– Application of simulation stimulus to model of circuit
Hardware Accelerated Simulation
– Use of special purpose hardware to accelerate simulation of circuit
Emulation
– Emulate actual circuit behavior - e.g. using FPGA’sRapid prototyping
– Create a prototype of actual hardwareFormal verification
– Model checking - verify properties relative to model
– Theorem proving - prove theorems regarding properties of a model
Approaches to Design Verification
-
8
Kurt Keutzer 15
Aspects of Design Verification
Event Driven
– Interactive Phase– High flexibility– Quick turnaround time– Good debug capabilities
Cycle -based simulation
– Regression Phase– Highest performance– Highest capacity
Emulation and Acceleration
– In-System Verification– Highest performance – Highest Capacity– Real system environment
Emulation
Cycle-basesimulation
Specification
Validation
Specification
Validation
Functional
Verification
(interactive)
Functional
Verification
(interactive)
Implementation
Verification
Implementation
Verification
Functional
Verification
(regressions)
Functional
Verification
(regressions)
In-System
Verification
In-System
Verification
Equivalence Checking
Event-driven Simulation
Software Simulation
– Application of simulation stimulus to model of circuit
Hardware Accelerated Simulation
– Use of special purpose hardware to accelerate simulation of circuit
Emulation
– Emulate actual circuit behavior - e.g. using FPGA’sRapid prototyping
– Create a prototype of actual hardwareFormal verification
– Model checking - verify properties relative to model
– Theorem proving - prove theorems regarding properties of a model
Approaches to Design Verification
-
9
Simulation: The Current Picture
Simulationdriver
Simulationengine
Monitors
SHORTCOMINGS:
• Hard to generate high quality input stimuli
– A lot of user effort
– No formal way to identify unexercised aspects• No good measure of comprehensiveness of validation
– Low bug detection rate is the main criterion
• Only means that current method of stimulus generation is not achieving more.
Kurt Keutzer 18
Simulation Drivers
Input stimuli consistent with circuitinterface must be generated
Environment of circuit must be represented faithfully
Tests can be generated
– pre-run (faster, hard to use/maintain)
– on-the-fly (better quality: can react to circuit state)Environment and input generation programs written in
– HDL or C, C++, or– Object-oriented simulation environment
• VERA, VerisitySometimes verification environment and test suite come with
product, e.g. PCI implementations, bridges, etc.
Simulationdriver
Simulationengine Monitors
Symbolicsimulation
Coverageanalysis
Diagnosis ofunverifiedportions
Vectorgeneration
-
9
Kurt Keutzer 17
Simulation: The Current Picture
Simulationdriver
Simulationengine
Monitors
SHORTCOMINGS:
• Hard to generate high quality input stimuli
– A lot of user effort
– No formal way to identify unexercised aspects• No good measure of comprehensiveness of validation
– Low bug detection rate is the main criterion
• Only means that current method of stimulus generation is not achieving more.
Simulation Drivers
Input stimuli consistent with circuitinterface must be generated
Environment of circuit must be represented faithfully
Tests can be generated
– pre-run (faster, hard to use/maintain)
– on-the-fly (better quality: can react to circuit state)Environment and input generation programs written in
– HDL or C, C++, or– Object-oriented simulation environment
• VERA, VerisitySometimes verification environment and test suite come with
product, e.g. PCI implementations, bridges, etc.
Simulationdriver
Simulationengine Monitors
Symbolicsimulation
Coverageanalysis
Diagnosis ofunverifiedportions
Vectorgeneration
-
10
Kurt Keutzer 19
Simulators
EVENT DRIVEN
• VCS
• Affirma
• Verilog-XL, ...
CYCLE-BASED
• Cyclone VHDL
• Cobra, ...
HYBRID
• VSS
Simulationdriver
Simulationengine Monitors
Symbolicsimulation
Coverageanalysis
Diagnosis ofunverifiedportions
Vectorgeneration
Monitors
Reference models (e.g. ISA model)
Temporal and snapshot “checkers”
Can be written in C, C++, HDLs, andVERA and Verisity: A lot of flexibility
Assertions and monitors can be automatically generated: 0-in’s checkers
Protocol specification can be given as
a set of monitors
a set of temporal logic formulas
(recent GSRC work)
Simulationdriver
Simulationengine Monitors
Symbolicsimulation
Coverageanalysis
Diagnosis ofunverifiedportions
Vectorgeneration
-
11
Types of software simulators
Circuit simulation
– Spice, Advice, Hspice – Timemill + Ace, ADM
Event-driven gate/RTL/Behavioral simulation
– Verilog - VCS, NC-Verilog, Turbo-Verilog, Verilog-XL– VHDL - VSS, MTI, Leapfrog
Cycle-based gate/RTL/Behavioral simulation
– Verilog - Frontline, Speedsim– VHDL - Cyclone
Domain-specific simulation
– SPW, COSSAPArchitecture-specific simulation
Kurt Keutzer 22
Event-driven simulation
Key elements:
– Circuit models and libraries• cells
• interconnect– Event-wheel
• Maintains schedules of events• Enables sub-cycle timing
Advantages
– Timing accuracy – Handles asynchronous
Disadvantage - performance and data management
-
15
Kurt Keutzer 29
Gate-level Cycle-based Acceleration
Much faster than SW or event-driven accelerator
: Runs actual code and data, in actual target systemsHarder to use than SW or event-driven accelerator, but easier than emulator
Severe restrictions on design style
- Purely synchronous design OK, else No.Expensive, complex, proprietary HW, SW
- Custom chips, interconnect, PCBs, connectors, chassis, instrumentation
- Compiler is substantial effort to develop & maintainIsolated from simulation, separate environment, proprietary simulator
Conclusion:
– Good solution for large fully synchronous projects that can afford it
– Not a mainstream technology
M. Butts - Synopsys
Software Simulation
– Application of simulation stimulus to model of circuit
Hardware Accelerated Simulation
– Use of special purpose hardware to accelerate simulation of circuit
Emulation
– Emulate actual circuit behavior - e.g. using FPGA’sRapid prototyping
– Create a prototype of actual hardwareFormal verification
– Model checking - verify properties relative to model
– Theorem proving - prove theorems regarding properties of a model
Approaches to Design Verification
-
16
FPGAs as logic evaluators
Today: 2 trillion gate evaluations per second per FPGA (200K gates, 10M cps)
– Growing with Moore’s Law as designs do– $1.5B industry behind it (XLNX+ALTR+ACTL)
Potent tool for logic verification and validation
How best to put the FPGA to use?
M. Butts - Synopsys
Kurt Keutzer 32
Logic Emulation
Ultra-large “FPGA”
Live hardware, gate-for-gate.
Entire design or major module is flattened, and compiled at once into multi-FPGA form.
Logically static circuit-switched interconnect.
In-circuit or vector-driven
Regular clock rate, > 1M cps.
M. Butts - Synopsys
-
17
Verification using Emulation
System Hardware
– Customized parallel processor system for emulating logic
– In-circuit target interface
Software Compiler
– Mapping RTL & Gate designs to emulator
Runtime Software
– C-API – Open SW architecture for
tight integration– Flexible modes of stimulus
In-circuit Target Board
Compiler
RTL or Gate design
Mapper
SBUS i/f
uP
Emulation Box
Kurt Keutzer 34
Logic Emulation HW
Tens to hundreds of large FPGAs
Interconnect, either:
– Programmable crossbars (QT), or
– Nearest-neighbor with time-multiplexing (Ikos).
SRAMs for modeling memory
CPUs for behavioral simulation &testbenches (QT Mercury)
Dedicated logic analyzer / pattern generator for visibility & vectors
In-circuit cable plugs into target
FPGA
Logic Board
XBar
FPGA
XBar
FPGA
Logic Board
XBar
FPGA
XBar
Control Computer
Network
XBarXBar
Logic Analyzer Pattern Generator In-Circuit Cable
M. Butts - Synopsys
-
16
Kurt Keutzer 31
FPGAs as logic evaluators
Today: 2 trillion gate evaluations per second per FPGA (200K gates, 10M cps)
– Growing with Moore’s Law as designs do– $1.5B industry behind it (XLNX+ALTR+ACTL)
Potent tool for logic verification and validation
How best to put the FPGA to use?
M. Butts - Synopsys
Logic Emulation
Ultra-large “FPGA”
Live hardware, gate-for-gate.
Entire design or major module is flattened, and compiled at once into multi-FPGA form.
Logically static circuit-switched interconnect.
In-circuit or vector-driven
Regular clock rate, > 1M cps.
M. Butts - Synopsys
-
20
Kurt Keutzer 39
The Emulation Interconnect Problem
Rent's Rule (p = Kg r) applies to partitioned designs.
FPGA logic capacity: 2X / 1.5 yr (Moore's Law)
FPGA pins needed by emulator: 2X / 2.5 yr (Moore + Rent)
Package pins: 2X / 4 yr - Can't keep up.
Vendors are time-multiplexing pins more and more to compensate.
– But that’s only a linear effect; it does not change the doubling time.
1000
10000
100000
1000000
10000000
1990 1995 2000 2005 2010100
1000
10000
100000
gates pinsPackage Pins
Pins neededFPGA capacity FPGA capacity is emulation usage:
8 gates / 4-LUT+FF, 75% packing.
Pins needed is for emulation usage: p = 2.75g 0.58
Package pins are Xilinx FPGA IOBs(1991-2000, extrapolated afterwards).
M. Butts - Synopsys
Emulation Conclusions
Market is flat at $100M/year
Expensive HW, SW, cost of sales
– High-end supercomputer-like businessCurrent competition
– Simulation farms have similar $/cycle/sec for regression vector sets
– FPGA-based rapid prototyping for validation, SW execution
Good solution for large projects that can afford it
Ultimately the basic concept is limited by IC packaging
M. Butts - Synopsys
-
24
How to make it smarter: Intelligent Simulation
Simulationdriver
Simulationengine
Monitors
Symbolicsimulation
Coverageanalysis
Diagnosis ofunverifiedportions
Vectorgeneration
Conven
tional
Novel
Kurt Keutzer 48
Symbolic Simulation SimulationdriverSimulation
engine Monitors
Symbolicsimulation
Coverageanalysis
Diagnosis ofunverifiedportions
Vectorgeneration
IDEA: One symbolic run covers many runs with concrete values.
Some inputs driven with symbols instead of concrete values•2(# symbols) equivalent binary coverage
-
24
Kurt Keutzer 47
How to make it smarter: Intelligent Simulation
Simulationdriver
Simulationengine
Monitors
Symbolicsimulation
Coverageanalysis
Diagnosis ofunverifiedportions
Vectorgeneration
Conven
tional
Novel
CLOSED FEEDBACK LOOP
Symbolic Simulation SimulationdriverSimulation
engine Monitors
Symbolicsimulation
Coverageanalysis
Diagnosis ofunverifiedportions
Vectorgeneration
IDEA: One symbolic run covers many runs with concrete values.
Some inputs driven with symbols instead of concrete values•2(# symbols) equivalent binary coverage
-
25
Kurt Keutzer 49
Symbolic Simulation
INNOLOGIC:BDD-based symbolic Verilog simulators
l ESP-XV: For processor and networking applications
l ESP-CV: For memory verification and sequential equivalence checking
l Monitors can have symbolic expressions
l Can symbolize time, e.g., event occurring after time T, 10 < T < 20.
l If bug is found, computes actual values exercising it
l Current “sweet-spots” of technology
– Memory verification: CAMs, caches, register files
– Unit level RTL functional verification: DMA, PCI,100-1000K gate blocks
– Data movement, datapath
Simulationdriver
Simulationengine Monitors
Symbolicsimulation
Coverageanalysis
Diagnosis ofunverifiedportions
Vectorgeneration
Symbolic Simulation
INNOLOGIC: Limitations
l Capacity limits:– ~ 1 million gate equivalents– # of symbols - design dependent.
• < 50 in worst cases (multipliers)
• several thousand in the best cases (memory, data movement).
• When out of memory, turn symbols into binary values - coverage lost but simulation completes.
l Roughly 10 times slower than Verilog-XLl Can’t use in conjunction with Vera or Verisity currently.
è Definitely worth a shot: Extra cost of symbols offset quickly, doesn’t require major change in framework.
è Full benefits of technology have not been realized yet.
Simulationdriver
Simulationengine Monitors
Symbolicsimulation
Coverageanalysis
Diagnosis ofunverifiedportions
Vectorgeneration
-
26
Coverage Analysis
Why?
• To quantify comprehensiveness of validation effort
– Tells us when not to stop– Even with completely formal methods, verification
is only as complete as the set of properties checked• To identify aspects of design not adequately exercised
– Guides test/simulation vector generation
• Coordinate and compare verification efforts
– Different sets of simulation runs– Different methods: Model checking, symbolic
simulation, ...
Simulationdriver
Simulationengine Monitors
Symbolicsimulation
Coverageanalysis
Diagnosis ofunverifiedportions
Vectorgeneration
Kurt Keutzer 52
Software Simulation
– Too slow– Moving to higher levels is helping – but not enough
Hardware Accelerated Simulation
– Too expensiveEmulation
– Even more expensiveRapid prototyping
– Too ad hocFormal verification
– Not robust enoughIntelligent Software Simulation
– Symbolic simulation – not robust enough– Coverage metrics – useful, but not useful enough– Automatic vector generation – not robust enough
Status of Design Verification
-
26
Kurt Keutzer 51
Coverage Analysis
Why?
• To quantify comprehensiveness of validation effort
– Tells us when not to stop– Even with completely formal methods, verification
is only as complete as the set of properties checked• To identify aspects of design not adequately exercised
– Guides test/simulation vector generation
• Coordinate and compare verification efforts
– Different sets of simulation runs– Different methods: Model checking, symbolic
simulation, ...
Simulationdriver
Simulationengine Monitors
Symbolicsimulation
Coverageanalysis
Diagnosis ofunverifiedportions
Vectorgeneration
Software Simulation
– Too slow– Moving to higher levels is helping – but not enough
Hardware Accelerated Simulation
– Too expensiveEmulation
– Even more expensiveRapid prototyping
– Too ad hocFormal verification
– Not robust enoughIntelligent Software Simulation
– Symbolic simulation – not robust enough– Coverage metrics – useful, but not useful enough– Automatic vector generation – not robust enough
Status of Design Verification
-
Formal Verification
-
Formal Verification
Formal Verification is the process of constructing a proof that a target system will behave inaccordance with its specification.
• Use ofmathematical reasoning to prove that an implementation satisfies a specification
• Like a mathematical proof: correctness of a formally verified hardware design holdsregardless of input values.
• Consideration ofall cases is implicit in formal verification.
• Must establish:
- A formalspecification(properties or high-level behavior).
- A formal description of theimplementation (design at higher level of abstraction —model (observationally) equivalent to implementation or implied by implementation).
[IEEE Spectrum, January 1996]
“As designs grow ever more complex, formal verifiers have left the research lab for theproduction arena.”
“Formal methods have already proven themselves, and have a bright future in electronicdesign automation.”
-
Formal Verification
• Complete with respect to a given property (!)
• Correctness guaranteed mathematically, regardless the input values
• No need to generate expected output sequences
• Can generate an error trace if a property fails: better understand, confirm by simulation
• Formal verification useful to detect and locate errors in designs
• Consideration ofall cases is implicit in formal verification
System Model System Specification
Verifier
not correct!correct! (error trace)
-
Simulation vs. Formal Verification
Example:
Simulation Values:
0 1 1
1 4 4
2 9 9
3 16 16
9 100 100
67 4624 4624
x 1+( )2 x2 2x 1+ +=
x x 1+( )2 x2 2x 1+ +
… … …
-
Simulation vs. Formal Verification (cont’d)
Formal Proof
1. definition of square
2. distributivity
3. substitution of 2. in 1.
4. neutral element 1
5. distributivity
6. substitution of 4. and 5. in 3.
7. neutral element 1
8. substitution of 7. in 6.
9. definition of square
10. definition of 2x
11. substitution of 9. and 10. in 8.
x 1+( )2 x2 2x 1+ +=
x 1+( ) x 1+( ) x 1+( )x x 1+( )1+=
x 1+( )2 x 1+( )x x 1+( )1+=
x 1+( )1 x 1+=
x 1+( )x xx 1x+=
x 1+( )2 xx 1x x 1+ + +=
1x x=
x 1+( )2 xx x x 1+ + +=
xx x2=
x x 2x=+
x 1+( )2 x2 2x 1+ +=
-
System Design and Verification
• Typical levels of abstraction in design:
• Behavioral synthesis: behavioral description into RTL description
• RTL synthesis: RTL description into logic description
• Logic synthesis: logic description into netlist of primitive gates for a target technology
• Layout synthesis: gate netlist to mask geometry
Idea
Behavioral Model
Register Transfer Model
Logic Gates Model
Layout
= ?
= ?
= ?
= ?Transistor Model
= ?
??
Manual
and
Automatic
Transformations
and
Local
Modifications
Correctness ofSynthesisTools??
-
© 1997 E. Cerny, X. Song, © 1999 E. Cerny, © 2000 S. Tahar 5/24/02 2.3 (of 73)
Combinational Equivalence Checking
Formal Proof
• Propositional resolution
• Stålmarck’s procedure
• ROBDDs
= flag: T/FTautology Check
(f1 = f2) = T
f1
f2
-
© 1997 E. Cerny, X. Song, © 1999 E. Cerny, © 2000 S. Tahar 5/24/02 2.16 (of 73)
Simple Example
Two circuits C1 and C2
Propositional Resolution
C1: out1 = a ∨ b
C2: out2 = (¬ a ∧ b) ∨ (a ∧ a)
(Mux: out2 = (¬ c ∧ b) ∨ (c ∧ a) )
G = (out 1 ⇔ out 2)
a
bout
ab out
c1
c2= ?
10
-
Formal Proof
y = (¬ s ∧ x2) ∨ (s ∧ x1) y' = x1∨ x2
= (¬ x1 ∧ x2) ∨ ( x1 ∧ x1)
= (¬ x1 ∧ x2) ∨ x1
= (¬ x1 ∧ x1) ∧ ( x2 ∨ x1)
= 1 ∧ ( x2 ∨ x1)
= x2 ∨ x1
⇒ y' = y
yx1x2
s
10
y’x1x2
-
© 1997 E. Cerny, X. Song, © 1999 E. Cerny, © 2000 S. Tahar 5/10/02 1.24 (of 43)
Formal Logic
What Does “Formal” mean?
• Webster’s dictionary gives the following as one of the definitions of “formal”:
“related to, concerned with, or constituting the outwardform of something asdistinguished from itscontent”
• A method isformal if its rules for manipulation are based on form (syntax) and not oncontent (semantics)
• Majority of existing formal techniques are based on some flavor of formal (symbolic)logic: Propositional logic, Predicate logic, other logics.
Formal logic
• Every logic comprises a formal language for making statements about objects andreasoning about properties of these objects.
• Statements in a logic language are constructed according to a predefined set of formationrules (depending on the language) calledsyntax rules.
• A logic language can be used in different ways.
-
© 1997 E. Cerny, X. Song, © 1999 E. Cerny, © 2000 S. Tahar 5/10/02 1.25 (of 43)
Types of Logic
• Propositional logic: traditionalBoolean algebra, variables∈{0,1}
• First-order logic (Predicate logic): quantifiesfor all (∀) andthere exists (∃) over variables
• Higher-order logic: adds reasoning about (quantifying over) sets and functions (predicates)
• Modal/temporal logics: reason about whatmust or may happen
• Propositional logic: decidable and complete
• First-order logic: decidable but not complete
• Higher-order logic: not decidable nor complete
Less expressive(-) Very expressive(+)
Decidable(+) Undecidable(-)
Propositional logic First-order logic Higher-order logic
Complete(+) Incomplete(-)
-
Limitations of Formal Verification
Just because we have proved something correct does not mean it will work!
There are gaps where formal verification connects with the real world.
• Does thespecification actually captures the designer’s intentions?
- Specification must be simple and abstract
- Example of a good specification for a half-adder:out = (in1 + in2) mod 2
• Does theimplementation in the real world behave like the model?
- Canin1 drive three inputs
- What happens if the wires are fabricated too close together?
- Do we need to model quantum effects on the silicon surface?
-
State of the Art
• Recently more promising picture
- Software specification: industry trying out notations like SDL or Z to documentsystem’s properties
- Protocol verification successful
- Hardware verification: industry adopting model checking and some theorem provingto complement simulation
- Industrial case studies increasing confidence in using formal methods
- Verification groups:IBM, Intel, Motorola, HP, Nortel, NEC, Fujitsu, SUN, Cadence,Siemens, Synopsys, Lucent Technologies, .......
- Commercial tools from:Chrysalis, Cadence, Synopsys, Verysys, IBM, .......
• Current tools focus on formal verification methods of digital hardware
• ... but new methods are making inroads into verification of real-time reactivesystems with continous and statistical behavior (e.g. analog circuits)
-
Theorem Proving
Prove that an implementation satisfies a specification by mathematical reasoning.
Implementation and specification expressed asformulas in a formal logic.
Relationship (logical equivalence/logical implication) described asa theorem to beproven.
A proof system:
A set of axioms and inference rules (simplification, rewriting, induction, etc.)
implication
equivalence
Implementation Specification
-
Theorem Proving (cont’d)
Some known theorem proving systems:
Boyer-Moore/ACL2 (first-order logic)
HOL (higher-order logic)
PVS (higher-order logic)
Lambda (higher-order logic)
Advantages:
• High abstraction and powerful logic expressiveness
• Unrestricted applications
• Useful for verifying parameterized datapath-dominated circuits
Limitations:
• Interactive (under user guidance)
• Requires expertise for efficient use
• Automated for narrow classes of designs
-
FSM-based Methods
Finite State Machines (FSM)
• Well-developed theory for analyzing FSMs (e.g., reachable states, equivalence)
• An FSM (I, O, S,δ, λ, S0)I : input alphabet,
O: output alphabet,
S: set of states,
δ: next-state relation,δ ⊆ S×I×S,λ: output relation,λ ⊆ S×I×O (Mealy),λ ⊆ S×O (Moore)S0: set of initial states.
• Deterministic machines:δ: S×I -> S andλ: S×I −> Ο are functions, S0 = {s0}.
FSM Equivalence Verification
• Basic method:
- If same state variables —Combinational Equivalence of δ andλ
- If state space different -State Enumeration by Reachability AnalysisTwo FSMs are equivalent if they produce the same output for every possible inputsequence —Sequential Equivalence Checking
-
Equivalence Checking
Equivalence by reachability analysis of the Product Machine
Reachability Analysis:
Start from initial state
repeat
Apply transition relation to determine next state
In each reached state, check equivalence of corresponding outputs of M1, M2
until all reachable states visited
• Involves building astate transition graph (Finite Kripke structure)
• Problem: “State explosion” - e.g., 32-bit→ register 232 states• Partial solution: Implicit State Enumeration with
- Reduced Ordered Binary Decision Diagrams (ROBDD)Represent transition/output relations and sets of states symbolically using ROBDD
M1
M2= ?input
output-1
output-2
yes/no
Product Machine M = (M1, M2)
deterministic
-
Equivalence Checking (cont’d)
Application example:
Combinational equivalence:
• possible if one-to-one state mapping do exit
• relatively straightforward (equivalence of sets of functions (BDDs))
• tools already part of verification flow
Sequential equivalence:
• no state mapping required (building of product machine)
• hard to handle large circuits (also must consider all initial states)
• no tools for industrial use
RTL Design
Gate Netlist
= ?inputoutput-1
output-2
yes
noCounter-example
-
Model CheckingBasic idea:
• Property described by temporal logic formula.
• System modeled by Labeled Transition Graph (LTG, LTS,Finite Kripke structure).
• Exhaustive search through the state space of the system (Reachability Analysis) todetermine if the property holds (provides counterexamples for identifying design errors).
• Problem: “State explosion”
• Partial Solution: Symbolic Model CheckingRepresent transition/output relations and sets of states symbolically using ROBDD
BehavioralModelor
Gate or RTDesign
True / Counterexamples
Property
Model Checker
FSM
-
Symbolic Model Checking
Basic idea:
• Problem: again “State explosion” (max ~ 400 Boolean variables), low abstraction level.
SpecificationDesign
Finite State Machine CTL Formula
Model Checker
OK / Counter-example
ROBDD
-
Model Checking vs. Simulation
Environment Constraints
DescriptionBeh./RTL
Model Checker Simulator
Description Test Bench
Simulation output(e.g. waveform)
True/Counterexample
Properties
Beh./RTL
-
Theorem Proving vs. Model Checking
Theorem Proving: useful for architectural design and verification
• Process: Implementation description: Formal logic Specification description: Formal logic Correctness: Imp ⇒ Spec (implication) or Imp ⇔ Spec (equivalence)
• High abstraction level possible, expressive notation, powerful logic and reasoning
• Interactive and deep understanding of design and higher-order logic required
• Need to develop rules (lemmas) and tactics for class of designs
• Need a refinement method to synthesizable VHDL / Verilog
Model Checking: at RT-level (or below) with at most ~400 Boolean state variables
• Process: Implementation description: Model as FSM Specification description: Properties in temporal logic Correctness:Impl Spec(property holds in the FSM model)
• Easy to learn and apply (completely automatic), properties must be carefully prepared
• Integrated with design process, refinement from skeletal model
• State space explosion problem (not scalable to large circuits)
• Increase confidence, better verification coverage
-
Formal Verification Tools
Supplier Tool Name Class of Tool HDL Design Level
COMMERCIAL TOOLS
Chrystalis Design Verifier Equiv. Checking VHDL/Verilog RTL/GateSynopsys Formality Equiv. Checking VHDL/Verilog RTL/GateCadence Affirma Equiv. Checking VHDL/Verilog RTL/GateCompass VFormal Equiv. Checking VHDL/Verilog RTL/GateVerysys Tornado Equiv. Checking VHDL/Verilog RTL/GateAbstract Hardware Ltd. Checkoff-E Equiv. Checking VHDL/Verilog RTL/GateIBM BoolsEye Equiv. Checking VHDL/Verilog RTL/GateCadence FormalCheck Model Checking VHDL/Verilog RTLAbstract Hardware Ltd. Checkoff-M Model Checking VHDL/Verilog RTL/GateIBM RuleBase Model Checking VHDL RTLAbstract Hardware Ltd. Lambda Theorem Proving VHDL/Verilog RTL/GatePUBLIC DOMAIN TOOLS
CMU SMV Model Checking own language RTLCadence Cadence SMV Model Checking Verilog RTLUC Berkeley VIS Model/Equ. Check. Verilog RTL/GateStanford U. Murphy Model Checking own language RTLCambridge U. HOL Theorem Proving (SML) universalSRI PVS Theorem Proving (LISP) universalUT Austin/CLI ACL2 Theorem Proving (LISP) universal
-
Simulation in Verilog-XL
AWK program
Seq. equ. checking in VIS
Analyze counterexample in XL
Analyze counterexample in XL
Analyze counterexample in XL
Verilog RTL description
Synopsys-Verilog netlist description
VIS/XL-Verilog netlist description
Model checking in VIS
Model checking in VIS
Synthesize in Synopsys
EDIF generated by Synopsys
Design Flow and Formal Verification (VIS)
-
Design Flow and Formal Verification
• RT level
⇒ Simulation of RTL(+) efficient for less interacting concurrent components
(- ) incomplete for complicated control parts and difficult error trace
⇒ Model checking of RTL(+) efficient for complicated interacting concurrent components
(+) counter-examples can trace design errors
• Netlist (Gate level)
⇒ Equivalence checking of netlist vs. RTL(+) check the equivalence of submodules to ensure the correctness of synthesis
(+) trace synthesis errors using counter-examples
⇒ Model checking of netlist(+) correctness of the entire gate-level implementation
(- ) unpractical: state space explosion
-
References
1. T. Kropf.Introduction to Formal Hardware Verification, Springer Verlag, 1999.
2. C. Kern and M. Greenstreet. “Formal Verification in Hardware Design: A Survey”,ACMTransactions on Design Automation of E. Systems, Vol. 4, April 1999, pp. 123-193.
3. A. Gupta. “Formal hardware verification methods: a Survey”.Formal Methods in SystemDesigns, Vol. 1, pp. 151-238, 1992.
4. Carl-Johan Seger. “An Introduction to Formal Verification”, Technical Report 92-13, UBC,Department of Computer Science, Vancouver, B.C., Canada, June 1992.
5. M. Yoeli. “Formal Verification of Hardware Design”, IEEE Computer Society Press, 1991.
6. M.C. McFerland. “Formal Verification of Sequential Hardware: a Tutorial”.IEEEtransaction on CAD, 12(5), May 1993.
7. E. M. Clarke and J. M. Wing. “Formal Methods: State of the Art and Future Directions”.ACM Computing Surveys, December 1996.
8. R. P. Kurshan, “Formal Verification in a Commercial Setting”,Proc. Design AutomationConference, Anaheim, California, June 9-13, 1997, pp 258-262.
9. Various Contributors, “Survey of Formal Verification”,IEEE Spectrum, June 1996, pp. 61-67.
10.K. Keutzer. “The Need for Formal Verification in Hardware Design and What FormalVerification has not Done for Me Lately”.Proc. HOL Theorem Proving System and itsApplication, Miami, Florida, USA, 1991.