Post on 31-Dec-2015
2007 Tehran University 1
Functional Functional Verification of HDL Verification of HDL
ModelsModels
Introduction to VerificationIntroduction to Verification
2007 Tehran University 2
Reference BookReference Book
• Janick Bergeron Janick Bergeron [WTB][WTB]
Writing Testbenches:Writing Testbenches:
Functional Verification Of HDL Functional Verification Of HDL Models.Models.
• First Edition, Kluwer, 2000, ISBN: 0-First Edition, Kluwer, 2000, ISBN: 0-7923-7766-47923-7766-4
• Second Edition, Kluwer, 2003, ISBN: Second Edition, Kluwer, 2003, ISBN: 1-4020-7401-81-4020-7401-8
2007 Tehran University 3
Other referencesOther references
• Comprehensive Functional Verification: Comprehensive Functional Verification: The complete industry cycleThe complete industry cycle– B. Wile, J. Goss, W. Roesner [IBM]B. Wile, J. Goss, W. Roesner [IBM]
• Principles of Functional VerificationPrinciples of Functional Verification– Andreas MeyerAndreas Meyer
• Functional Verification Coverage & Functional Verification Coverage & Measurement and AnalysisMeasurement and Analysis– Andrew PizialiAndrew Piziali
• Advanced Verification Techniques: A Advanced Verification Techniques: A SystemC Based Approach for Successful SystemC Based Approach for Successful TapeoutTapeout– Leena Singh, L. Drucker, Neyaz KhanLeena Singh, L. Drucker, Neyaz Khan
2007 Tehran University 4
Course GoalsCourse GoalsAimAim: : To familiarise you with the routine tasks in To familiarise you with the routine tasks in
verificationverificationmanagementmanagement, and to give you the , and to give you the technical background technical background
plus some of the plus some of the practical skills practical skills expected from a Design expected from a Design Verification Engineer.Verification Engineer.
• PrerequisitePrerequisite: Hardware Language programming: Hardware Language programming
On successful completion of this unit, you will be able On successful completion of this unit, you will be able to:to:
• understand the complexities and limits of verification;understand the complexities and limits of verification;• carry out functional verification and determine its carry out functional verification and determine its
effectiveness;effectiveness;• set appropriate verification goals, select suitable set appropriate verification goals, select suitable
verification methods and assess the associated risks;verification methods and assess the associated risks;• compile a verification plan that fits into the flow of a compile a verification plan that fits into the flow of a
design projectdesign project
2007 Tehran University 5
19741974
Brian W. KernighanBrian W. Kernighan• Debugging is twice as hard as Debugging is twice as hard as
writing the code in the first place. writing the code in the first place. Therefore, if you write the code as Therefore, if you write the code as cleverly as possible, you are, by cleverly as possible, you are, by definition, not smart enough to definition, not smart enough to debug it.debug it.
2007 Tehran University 6
Bugs: What is the Cost?Bugs: What is the Cost?
• Time to MarketTime to Market is affected is affected• Bugs found early in design have little cost!Bugs found early in design have little cost!• Finding a bug at chip/system level has Finding a bug at chip/system level has
moderate cost:moderate cost:– Requires more debug time and isolation time.Requires more debug time and isolation time.– Could require new algorithm, which could effect Could require new algorithm, which could effect
schedule and cause board rework.schedule and cause board rework.
• Finding a bug in System Test (test floor) Finding a bug in System Test (test floor) requires re-spin of a chip.requires re-spin of a chip.
• Finding a bug after customer delivery costFinding a bug after customer delivery cost millons
2007 Tehran University 7
Cost of Bugs Over TimeCost of Bugs Over Time
system
Number of bugs found
chipmodule
Cost of finding bugs
Timecustomer
2007 Tehran University 8
Importance of Importance of VerificationVerification
• 70% of ASIC / IP / SoC design effort goes to 70% of ASIC / IP / SoC design effort goes to verificationverification– Is on the critical pathIs on the critical path– Has effects on Schedule, Cost, QualityHas effects on Schedule, Cost, Quality– Fewer revisions through fabrication process Fewer revisions through fabrication process
means lower cost and faster time-to-marketmeans lower cost and faster time-to-market
• Twice more Verification engineers than RTL Twice more Verification engineers than RTL designerdesigner
• Re-spinning cost time and moneyRe-spinning cost time and money• Getting the design right first time is an artGetting the design right first time is an art
– Due to chip complexity it is more and more Due to chip complexity it is more and more difficultdifficult
2007 Tehran University 9
Silicon ValidationSilicon Validation
• Silicon Validation Silicon Validation times increase as times increase as features size decreasesfeatures size decreases
• The difficulty is The difficulty is expected to increase at expected to increase at 65nm and below 65nm and below because existing ad-because existing ad-hoc methodologies do hoc methodologies do not scale with the not scale with the unprecedented levels unprecedented levels of SoC device of SoC device complexity. complexity.
2007 Tehran University 10
SoC VerificationSoC Verification
• Complete system-level verification of a Complete system-level verification of a complex SoC at 90 nm or below is not complex SoC at 90 nm or below is not feasible pre-silicon feasible pre-silicon – Even the most sophisticated SoC design Even the most sophisticated SoC design
methodology cannot fully account for all the methodology cannot fully account for all the parameters that impact silicon behavior, or parameters that impact silicon behavior, or for all logic "corner cases" that occur in the for all logic "corner cases" that occur in the real life of a chip working at speed and in real life of a chip working at speed and in system [Miron Abramovici, Sep. 2007 ]system [Miron Abramovici, Sep. 2007 ]
• post-silicon validation has become an post-silicon validation has become an essential step in the design essential step in the design implementation methodology implementation methodology
2007 Tehran University 11
Importance of Verification Importance of Verification (Cont.)(Cont.)
• International Technology Roadmap International Technology Roadmap for Semiconductors identified for Semiconductors identified Verification as a crisis area Verification as a crisis area (http://public.itrs.net/)
• Shortfalls in qualified engineers and Shortfalls in qualified engineers and software-trained designerssoftware-trained designers
• Verification can be reduced through Verification can be reduced through parallelism, abstraction and parallelism, abstraction and automationautomation
2007 Tehran University 12
Course OutlineCourse Outline
Lecture Topics:Lecture Topics:• What is Verification? What is a Testbench?What is Verification? What is a Testbench?• Verification Tools (including Verilog basics)Verification Tools (including Verilog basics)• Verilog HDL coding and Verification PlanVerilog HDL coding and Verification Plan• Assertion-based VerificationAssertion-based Verification• Advanced Testbench Design Methodology Advanced Testbench Design Methodology • Functional Formal Verification and Functional Formal Verification and
Property CheckingProperty Checking• Labs (Verilog, ModelSim, C/C++, SystemC)Labs (Verilog, ModelSim, C/C++, SystemC)
2007 Tehran University 13
Introduction to IC Design Introduction to IC Design ProcessProcess
1.1. SpecificationSpecification2.2. Behavioral DescriptionBehavioral Description
Architectural SynthesisArchitectural Synthesis3.3. RTL DescriptionRTL Description
Logical SynthesisLogical Synthesis4.4. Gate Level DescriptionGate Level Description
Physical SynthesisPhysical Synthesis5.5. Layout and tape-outLayout and tape-outIn practice all steps almost start at the In practice all steps almost start at the
same time and run in parallelsame time and run in parallel
2007 Tehran University 14
Steps in IC Design Steps in IC Design ProcessProcess
Architectural SynthesisArchitectural Synthesis Input: behavioral Input: behavioral descriptiondescription
• generates architecture for design at RTLgenerates architecture for design at RTL– data path (interconnection of adders, multipliers, etc)data path (interconnection of adders, multipliers, etc)– control unit (signals to control data path)control unit (signals to control data path)
• scheduling and bindingscheduling and bindingLogic SynthesisLogic Synthesis Input: RTL descriptionInput: RTL description• converts into network of logic primitives (depending on converts into network of logic primitives (depending on
designdesign• style)style)• Two groups:Two groups:
– sequential/combinatorialsequential/combinatorial
Physical DesignPhysical Design Input: gate level Input: gate level descriptiondescription
• generates geometric patterns that define physical layout of generates geometric patterns that define physical layout of chipchip
2007 Tehran University 15
Verification Role in IC Verification Role in IC DesignDesign
IC design is a complex process:IC design is a complex process:• Conflict: design time (decreased) versus complexity Conflict: design time (decreased) versus complexity
(increased)(increased)• Aim: correct by construction, right-first-time designAim: correct by construction, right-first-time design
More and more time-consuming to obtain an More and more time-consuming to obtain an acceptable level of confidence in correctness of acceptable level of confidence in correctness of designdesign
• Design time << Verification timeDesign time << Verification time• Needs its own trained verification engineersNeeds its own trained verification engineers• 80% of written code is for the verification 80% of written code is for the verification
environmentenvironment
2007 Tehran University 16
How does a designer know How does a designer know a circuit is correct?a circuit is correct?
concept
Specification
HDL design (RTL)
Tape out
Silicon
Verify: What you specified is what you envisioned
Verify: What you designed is what you specified
Verify: What you taped out is what the RTL described
Verify: What you manufactured is what you taped out
There is no single methodology that covers allThere is no single methodology that covers all!
2007 Tehran University 17
Some DefinitionsSome Definitions
• What is Verification?What is Verification?– A process used to demonstrate the functional A process used to demonstrate the functional
correctness of a design in its implementationcorrectness of a design in its implementation
Hence the term Hence the term Functional VerificationFunctional Verification
• What is a Testbench?What is a Testbench?– Code used to generate predetermined input Code used to generate predetermined input
sequence to a designsequence to a design– Implemented using Verilog, VHDL, Implemented using Verilog, VHDL, ee or Open or Open
Vera, also include C routinesVera, also include C routines– Always refers to test case/scenarioAlways refers to test case/scenario
2007 Tehran University 18
Testbench and DUVTestbench and DUV
• Completely closed systemsCompletely closed systems• No inputs or outputs go in or outNo inputs or outputs go in or out• Generic Structure of a testbench and Generic Structure of a testbench and
DUV:DUV:
DesignUnder Verification
Testbench
Stimulus Generator Checker
Supply stimulus to checker for comparison with the specification
2007 Tehran University 19
Verification is on critical Verification is on critical pathpath
Need to minimise verification time e.g. by Need to minimise verification time e.g. by using:using:
• ParallelismParallelism: Add more resources: Add more resources• AbstractionAbstraction: Higher level of abstraction (i.e. : Higher level of abstraction (i.e.
C vs Assembly, TLM)C vs Assembly, TLM)– This often means a reduction of control!This often means a reduction of control!
• AutomationAutomation: Tools to automate standard : Tools to automate standard processesprocesses– Requires standard processes/methodology.Requires standard processes/methodology.– Usually a variety of functions, interfaces, Usually a variety of functions, interfaces,
protocols, and transformations must be verified.protocols, and transformations must be verified.– Not all (verification) processes can be automated.Not all (verification) processes can be automated.– Randomization can be used as an automation toolRandomization can be used as an automation tool
2007 Tehran University 20
Reconvergence ModelReconvergence Model
• Conceptual representation of the Conceptual representation of the verification process: verification process: Illustrate what Illustrate what is exactly being verifiedis exactly being verifiedTransformation
Verification
Purpose of verification is to ensure that the result of some transformation is as intended or as expected.
2007 Tehran University 21
Source of Source of ambiguity/errorambiguity/error
• Human Factor: Human Factor: – How a designer read and interpret the How a designer read and interpret the
specification specification – Specification can be Specification can be ambiguousambiguous– Designer can introduce errors by Designer can introduce errors by
(mis-)interpretation (mis-)interpretation (re-convergence model)(re-convergence model)
DANGERDANGER: When a designer verifies her/his own : When a designer verifies her/his own design - then she/he is verifying her/his own design - then she/he is verifying her/his own interpretation of the design - not the interpretation of the design - not the specification!specification!
2007 Tehran University 22
How to Reduce Human How to Reduce Human ErrorsErrors
Reduce human errors by:Reduce human errors by:• AutomationAutomation
– Take the human out but not really feasibleTake the human out but not really feasible– Process not well defined enough, not formal Process not well defined enough, not formal
enoughenough• Make mistake-proof the human intervention Make mistake-proof the human intervention
by reducing it to simple steps (Poka-Yoka)by reducing it to simple steps (Poka-Yoka)• RedundancyRedundancy
– Two individuals check each other’s workTwo individuals check each other’s work– Costly mechanism but still cheaper than re-Costly mechanism but still cheaper than re-
design and chip recalldesign and chip recallBut never let a designer verify his own designBut never let a designer verify his own design
2007 Tehran University 23
What is being verified?What is being verified?
• Choosing a common origin and the Choosing a common origin and the reconvergence points determines what is reconvergence points determines what is being verified.being verified.– Determined by the tool used to perform the Determined by the tool used to perform the
verificationverification
• The following approaches have different The following approaches have different origin and reconvergence points:origin and reconvergence points:– Formal VerificationFormal Verification– Functional VerificationFunctional Verification– Testbench GeneratorsTestbench Generators
2007 Tehran University 24
Formal VerificationFormal Verification
This is This is notnot a tool that mathematically a tool that mathematically determines the correctness of a design determines the correctness of a design without having to write testbenches!without having to write testbenches!
Build a formal mathematical model of some Build a formal mathematical model of some aspect of the system and calculate aspect of the system and calculate whether or not it has some desired whether or not it has some desired propertyproperty
• 2 types of Formal Verification2 types of Formal Verification– Equivalence CheckingEquivalence Checking– Model CheckingModel Checking
2007 Tehran University 25
Equivalence CheckingEquivalence Checking• Compares two models to check for equivalenceCompares two models to check for equivalence
– Mathematically proves that both are logically Mathematically proves that both are logically equivalent equivalent
• Commonly used on lower level of designCommonly used on lower level of design
• Make sure that scan chain, clock tree insertion Make sure that scan chain, clock tree insertion did not change the functionalitydid not change the functionality
• It is only interested in comparing boolean and It is only interested in comparing boolean and sequential logic functions – no tech mappingsequential logic functions – no tech mapping
• Why can’t we entirely trust the synthesis tool?Why can’t we entirely trust the synthesis tool?
Equivalence Checking
Synthesis
RTL Gate
2007 Tehran University 26
Model CheckingModel Checking
• Assertions or characteristics of a design are Assertions or characteristics of a design are formally proven or disprovedformally proven or disproved
• Use to check for generic problems or violations of Use to check for generic problems or violations of user-defined properties of the behavior of the user-defined properties of the behavior of the designdesign
• Usually employed at higher levels of design process
• Properties are derived from specification• Properties are expressed in some (temporal) logic• Checking often involves a (finite) state machine
model
2007 Tehran University 27
Model CheckingModel Checking
Problems:Problems:• Knowing Knowing which properties which properties to check and to check and how to how to
express them express them needs expertise.needs expertise.– Very few designers are also good mathematicians.Very few designers are also good mathematicians.– (Verification engineers should be closing this gap.)(Verification engineers should be closing this gap.)
• Very few user-friendly tools for model checking.Very few user-friendly tools for model checking.• Tools that do exist can only check small designs.Tools that do exist can only check small designs.• There is a lot of There is a lot of researchresearch to make to make Model Model
CheckingChecking more practically usable. more practically usable. For an For an introduction see ”Model Checking” by Clarke, introduction see ”Model Checking” by Clarke, Grumberg and PeledGrumberg and Peled
2007 Tehran University 28
Functional VerificationFunctional Verification
• Ensure that a design implements Ensure that a design implements intended functionalityintended functionality
• Without it, one must trust that the Without it, one must trust that the transformation of a specification into transformation of a specification into RTL code was performed correctlyRTL code was performed correctly
• We can only prove the presence of We can only prove the presence of bugs, but we cannot prove their bugs, but we cannot prove their absence!absence!
2007 Tehran University 29
Testbench GeneratorsTestbench Generators• Tools to generate stimuli to exercise code or expose bugs• Designer/Verification engineer input is still required.• RTL code is the origin and there is no reconvergence point
• Verification engineer has responsibility to determine if the testbench applies valid stimuli
• If used with parameters, one can control the generator in order to focus the testbench on more specific scenarios
• 2 input ”and” gate - ... combinations of inputs (exhaustive)• Instruction stream generators to verify processor cores.Is exhaustive verification (always) feasible?
RTL Testbench
Testbench Generation
Code coverage/proof Metrics
2007 Tehran University 30
Functional Verification Functional Verification ApproachesApproaches
There are 3 complementary There are 3 complementary approaches:approaches:
• Black-Box VerificationBlack-Box Verification• White-Box VerificationWhite-Box Verification• Grey-Box VerificationGrey-Box Verification
2007 Tehran University 31
Black Box VerificationBlack Box Verification• The black box has inputs, outputs, and performs some The black box has inputs, outputs, and performs some
function.function.• The function may be well documented (or not ...).The function may be well documented (or not ...).• To verify a black box, you need to To verify a black box, you need to understand the understand the
function function and be able to predict the outputs based on the and be able to predict the outputs based on the inputs.inputs.
• The black box can be a full system, a chip, a unit of a The black box can be a full system, a chip, a unit of a chip, a module, etc.chip, a module, etc.
• No knowledge of the actual implementation is requiredNo knowledge of the actual implementation is required• Independent of implementation.Independent of implementation.• Lacks controllability and observability.Lacks controllability and observability.• Difficult to locate source of problem, only exposes effectsDifficult to locate source of problem, only exposes effects• Example: round-robin arbiterExample: round-robin arbiter
2007 Tehran University 32
White Box VerificationWhite Box Verification
• Opposite of Black BoxOpposite of Black Box• Full visibility and controllability of the Full visibility and controllability of the
internal structure and implementation of internal structure and implementation of DUVDUV
• Easy to set up interesting conditionsEasy to set up interesting conditions• But testbench is tied to a specific But testbench is tied to a specific
implementation – needs to be changed if implementation – needs to be changed if design modifieddesign modified
• Best for module or system level verificationBest for module or system level verification• Example: counter roll-overExample: counter roll-over
2007 Tehran University 33
Grey Box VerificationGrey Box Verification
• Compromise between BB and WBCompromise between BB and WB• Like BB, it observes and controls top level Like BB, it observes and controls top level
interface signalinginterface signaling• But increases coverage metrics by adding But increases coverage metrics by adding
some non-functional modification to increase some non-functional modification to increase visibilityvisibility– Include additional software-accessible registers to Include additional software-accessible registers to
observe internal statesobserve internal states
• In practice for SoC most environments are In practice for SoC most environments are GB because prediction of results are GB because prediction of results are sometimes impossible without looking at sometimes impossible without looking at internal signalsinternal signals
2007 Tehran University 34
Perfect VerificationPerfect Verification
• To fully verify a black box, you must show that the logic works correctly for all combinations of inputs.
This means:• Driving all permutations on the input
lines.• Checking for proper results in all cases.• Full verification is not practical on
large designs, but the principles are valid across all verification approaches
2007 Tehran University 35
Verification versus Verification versus TestingTesting
• Often Confused:Often Confused:– Test: show the design was Test: show the design was
manufactured correctly (manufactured correctly (post-siliconpost-silicon) – ) – verifies internal nodes can be toggled verifies internal nodes can be toggled by applying test vectorsby applying test vectors
– Verification: Ensure the design meets Verification: Ensure the design meets its functional intent (its functional intent (pre-siliconpre-silicon))
Spec Gates Silicon
HW Design Manufacturing
Testbench Testing
HDL
Equivalence CheckingEquivalence Checking
SynthesisSynthesis
2007 Tehran University 36
Verification versus Testing Verification versus Testing (Cont.)(Cont.)
• One test method is scanning: One test method is scanning: Design For Design For TestTest– Links all registers into a long shift register (chain)Links all registers into a long shift register (chain)– Chain accessible from the I/O pins, observe and Chain accessible from the I/O pins, observe and
control internal statescontrol internal states• Scan-based testing restricts the design Scan-based testing restricts the design
(sync, no gated clock, single clock) but keeps (sync, no gated clock, single clock) but keeps cost low (reference book: Digital System cost low (reference book: Digital System Testing and Testable Design, Abramovici)Testing and Testable Design, Abramovici)
Research topicResearch topic: “Design for Verification”!: “Design for Verification”!Consider what the design is supposed to do? Consider what the design is supposed to do?
How will it be verified?How will it be verified?
2007 Tehran University 37
Design and Verification Design and Verification ReuseReuse
• You won’t use what you don’t trust.You won’t use what you don’t trust.• How to trust it? (Verify it)How to trust it? (Verify it)• For reuse, designs must be verified with more For reuse, designs must be verified with more
strict requirementsstrict requirements– All claims, possible combinations and uses must be All claims, possible combinations and uses must be
verified.verified.– Not just how the design is (intended to be) used in Not just how the design is (intended to be) used in
one environmentone environment
• Verification/testbenches re-usable to some Verification/testbenches re-usable to some degreesdegrees– BFMs have different requirements at block or system BFMs have different requirements at block or system
levellevel– From project to project From project to project
2007 Tehran University 38
Cost of VerificationCost of Verification• It is always too long and costs too much but It is always too long and costs too much but
is absolutely necessaryis absolutely necessary– It does not generate revenue!It does not generate revenue!– As number of bugs decreases, the cost and time As number of bugs decreases, the cost and time
for finding the remaining ones increases: for finding the remaining ones increases: When When is the verification done?is the verification done?
• Yet indispensable:Yet indispensable:– To create revenue, design must be functionally
correct and provide benefits to customer.– Proper functional verification demonstrates
trustworthiness of the design.– Right-first-time designs demonstrate
professionalism and ”increase” reputation of design team
2007 Tehran University 39
When Is Verification When Is Verification Done?Done?
Never truly done on complex designs.Never truly done on complex designs.• RememberRemember: Verification can only show presence of : Verification can only show presence of
errors, not their absence.errors, not their absence.• Skill and experience are needed to:Skill and experience are needed to:
– Determine WHAT needs to be verified.Determine WHAT needs to be verified.• Be selective/exhaustive.Be selective/exhaustive.• Identify corner cases.Identify corner cases.
• Given enough time, errors Given enough time, errors will will be uncovered.be uncovered.• Critical question:Critical question:
– Is the error likely to be severe enough to warrant the Is the error likely to be severe enough to warrant the effort spent to find/correct it?effort spent to find/correct it?
Verification is similar to statistical hypothesis testing.Verification is similar to statistical hypothesis testing.• Hypothesis ”under test” is: Hypothesis ”under test” is: Is the design Is the design
functionally correct?functionally correct?
2007 Tehran University 40
Hypothesis MatrixHypothesis Matrix
• Type I mistakes: Easy to identify - found error where none exists.
• Type II mistakes: Most serious - verification failed to identify an error!– Can result in a bad design being shipped unknowingly!
• Knowing where you are in the verification process is much easier to estimate than how long it will take to complete the job.
Type II(False Positive)
Type I(False Negative)
Bad Design
Good design
Fail Pass
2007 Tehran University 41
SummarySummary• Seen what a testbench is• Understood how important verification is and why
it is so important.• Seen that parallelism, automation and abstraction are
strategies to reduce the time necessary to implement testbenches.
• Used reconvergence models to help us identify what is verified.
• Been introduced to the various verification approaches that exist.
• Investigated the importance of verification for design reuse.
• Discussed the cost of verification• Next: Verification tools, especially simulators,
waveform viewers and simulation languages (Verilog)