Better Living Through Formal - Test and Verification ... · •Add a formal element to your sanity...
Transcript of Better Living Through Formal - Test and Verification ... · •Add a formal element to your sanity...
1 © 2016 Broadcom. All Rights Reserved. "Broadcom" refers to Broadcom Limited and/or its subsidiaries. | © 2016 Broadcom. All Rights Reserved. "Broadcom" refers to Broadcom Limited and/or its subsidiaries.
Better Living Through Formal
16th June 2016
Alex Orr
2 © 2016 Broadcom. All Rights Reserved. "Broadcom" refers to Broadcom Limited and/or its subsidiaries. |
• D.A.V.E in various UK startups for ~25 years – Purely verification for last 5 years
– 3 years at Broadcom
• Exclusively dynamic simulation pre-Broadcom – Primarily constrained random methodology
– Migrating to UVM
– “Just show me the waves”
– Red Xs
• An EDA user – Future employment depends on pragmatic and timely application of methodologies and tools
Quick Biography
© 2016 Broadcom. All Rights Reserved. "Broadcom" refers to Broadcom Limited and/or its subsidiaries.
My First 100 Days in Formal Land
4 © 2016 Broadcom. All Rights Reserved. "Broadcom" refers to Broadcom Limited and/or its subsidiaries. |
• First presented at FVC2015 (Reading)
• Main conclusions – Dynamic verification can be seen as attempting to prove a negative
– “I’ve never seen ...” caveated with “yet!”
– Formal verification is positive
– You get a mathematically correct proven status …
– … but can still be wrong (GIGO)
– Formal verification has some design sweet spots
– e.g Arbiters
– Can take billions of cycles to hit all corner cases but just a few assertions.
– Deliver these as separate, exhaustively proven modules along with proofs (Imagination)
– And so does dynamic verification
– Max and match approaches
– Don’t try to use just one methodology for all
My First 100 Days In Formal Land
FF1 RAM1 RAM2 IP
OP 1
2
3
4
Priority
5 © 2016 Broadcom. All Rights Reserved. "Broadcom" refers to Broadcom Limited and/or its subsidiaries. |
– Don’t be afraid to use behavioral helper code
– There’s no need to capture everything as a single complex assertion
– Lowers the bar for adoption
– Formal verification has applications beyond pure RTL
– STA constraints capture design intent
– A primary input to the back end
– Often little to no verification beyond loading them into tool
– Formal verification run times can be much quicker than dynamic simulation
– A boon for design discovery and understanding
– “Time to bug” can be significantly faster then dynamic simulation
– Good for continuous integration/sanity tests
My First 100 Days In Formal Land
6 © 2016 Broadcom. All Rights Reserved. "Broadcom" refers to Broadcom Limited and/or its subsidiaries. |
• Ultimately:
“Hard to imagine a modern flow describing itself as robust that does not have a Formal verification
element.”
My First 100 Days In Formal Land
© 2016 Broadcom. All Rights Reserved. "Broadcom" refers to Broadcom Limited and/or its subsidiaries.
One Year Later
8 © 2016 Broadcom. All Rights Reserved. "Broadcom" refers to Broadcom Limited and/or its subsidiaries. |
• Continue to use Formal verification – An enthusiast rather than an expert
– Even on iterative/delta projects
– Making light of the more prosaic verification tasks
– Reverse connectivity for the win!
• Seeing it become a standard weapon in the arsenal of the verification engineer
• Speaker at JUG 2015 in San Jose – Saw some very advanced applications of Formal techniques
– Using tools to bug hunt not prove using specialised engines*
– Use cover statements to guide engine
– typically 10x wrt to properties
– Easy to believe you must strive to be at this level
– This advanced usage often requires a dedicated Formal verification engineer
– Significant gains to be made with more modest levels of expertise …
*I still assert that I shouldn’t care about engines outside of these specialised use cases
One Year Later
9 © 2016 Broadcom. All Rights Reserved. "Broadcom" refers to Broadcom Limited and/or its subsidiaries. |
故善战者,求之于势,不责于人
“The expert in battle seeks his victory from strategic advantage and does not
demand it from his men”
Sun Tsu – The Art of War
10 © 2016 Broadcom. All Rights Reserved. "Broadcom" refers to Broadcom Limited and/or its subsidiaries. |
“The expert in verification seeks zero bugs from best design practice and DfV and does not compel their team
verify harder”
Alex Orr – JUG 2015
11 © 2016 Broadcom. All Rights Reserved. "Broadcom" refers to Broadcom Limited and/or its subsidiaries. |
“Put less bugs in the implementation for the verification team to find”
(paraphrasing even more)
12 © 2016 Broadcom. All Rights Reserved. "Broadcom" refers to Broadcom Limited and/or its subsidiaries. |
• Shift left/start earlier/share the love
• Hand over better quality RTL – Improve the state of the RTL before it gets to the verification team
– Ideally the only bugs found will be diversions from the specification not implementation errors
• Find implementation bugs before 0ns – Formal doesn’t require a testbench
• Liberal use of assertions and covers – If you can make an assertion about a line of code
– capture it and prove it formally, before you simulate anything
– Put covers on your decodes and transitions
– Look for coverage holes before you fall into them!
• Use the mathematical rigor of the tool to your advantage – Let it do the de-morgans
– Let it highlight unexpected behaviours and side-effects
– Like having a omniscient minor deity at your command
Empower your RTL Engineers
13 © 2016 Broadcom. All Rights Reserved. "Broadcom" refers to Broadcom Limited and/or its subsidiaries. |
• Use covers and assertions to explore your inherited design – It is the expert in the implementation
– Use assumptions/what-if analysis to understand implementation motivations
• It is easy to break the testbench *improving* it – You can add as many assertions as you like!
• Missing coverage on legacy blocks – Prove you can get there with covers
– Separate coverage item makes it easy to close ticket
• Use assertions to stimulate *features* found in silicon – Prove you’ve addressed them without re-writing the testbench/test suite if you are time limited
Building on legacy
14 © 2016 Broadcom. All Rights Reserved. "Broadcom" refers to Broadcom Limited and/or its subsidiaries. |
• Use Formal to prove STA constraints – False and multicycle paths
– Often only reviewed by designer
– Are an assertion about the temporal design intent
– One of the reasons we still run SDF simulations
– Be part of the solution not the problem!
– Assertions will fire in RTL if modified
– detect and fix early
• Use Formal connectivity checks to prove DfT structures – Often don’t exist in RTL
– Quite often captured in csv format anyway!
– DfT engineers may not have skills to author UVM testcases
• Use assertions to prove test modes won’t wreak havoc on your IP
Silicon Backend
15 © 2016 Broadcom. All Rights Reserved. "Broadcom" refers to Broadcom Limited and/or its subsidiaries. |
• Easy to report junit format for Jenkins – Speed of execution enables effective continuous integration
• Add a formal element to your sanity test – Likely to be first thing to fire
• Advanced flows such as UNR can massively help closure efforts
Automation
© 2016 Broadcom. All Rights Reserved. "Broadcom" refers to Broadcom Limited and/or its subsidiaries.
Conclusions
17 © 2016 Broadcom. All Rights Reserved. "Broadcom" refers to Broadcom Limited and/or its subsidiaries. |
• I find myself using Formal verification more and more – Often for less glamorous tasks than full automated proofs …
– … just like my verification tasks!
– Formal makes it easier to address deltas and small changes
– Less destructive than extending the testbench
• Encourage the use of Formal earlier in the flow (shift left) – Improves implementation quality and therefore makes verification easier
– Can concentrate on debugging divergences from specification
– Less time spent debugging implementation mistakes
– Reduces the reliance on SDF sims
• Plenty of scope for use further down the flow – Prove you STA constraints
– Verify your test structures
Conclusions
18 © 2016 Broadcom. All Rights Reserved. "Broadcom" refers to Broadcom Limited and/or its subsidiaries. |
• Ultimately the best thing I can do to reduce bugs and make my life better …
…is to help the RTL engineer put less of them in the design in the first place!
• To that extent Broadcom trains both the RTL and verification teams in Formal
Conclusions
19 © 2016 Broadcom. All Rights Reserved. "Broadcom" refers to Broadcom Limited and/or its subsidiaries. | © 2016 Broadcom. All Rights Reserved. "Broadcom" refers to Broadcom Limited and/or its subsidiaries.
Thank You