Case Study of End-to-End Formal Verification MethodologyJake MaasNirabh RegmiKrishnan PalaniswamiAshish Kulkarni

Methodology for verifying a block completely using formal methods• Alternative for traditional UVM based IP verification

environment• Avoid high-cost development of custom agents, sequence

libs, checkers for UVM test benches with custom interfaces• UVM agents or VIPs are replaced by “proof-kits”

IP functionality is fully described using formal properties• All properties derived from spec and reviewed with

arch/design teams• Includes covers, black-box and white-box assertions, for all

control/data/debug interfacesWork can start after specifications are ready. • First bugs are available almost immediately after the first RTL drop

Registers are checked with assertions generated from an IPXACT XML source

IP is refined as assertions are implemented & proofs are debugged

IP is fully verified when all the assertions and proof kits are proven

What is End-to-End Formal Verification?

• Experiment with end-to-end formal methods to achieve same quality as traditional UVM test benches, especially for certain types of control-path designs

• Reduce verification effort by deploying a formal verification only environment due to resource/schedule pressures on IP verification

• Develop an end-to-end formal verification methodology for custom IP blocks

• Identify areas of improvement with the EDA vendors on metrics used for formal end-to-end IP verification signoff

Motivation Behind FE2E

Benefits• Early feedback on RTL due to formal methodology• Exhaustive verification compared to simulation based bug-

hunting methodologies. Multiple SEED regressions not required to close coverage• Shorter DV Cycle compared to UVM • Portability of assertions to other verification environments

(subsystem/chip level test benches)• Uses proof-kits, which are lower cost (effort) than traditional

UVM VIP

Challenges• Relating formal verification “metrics” to traditional metrics

used for simulation based test benches (e.g. code/functional coverage)• Reviews were more time consuming to ensure the

completeness of the assertions and assumptions to correctly constrain the design• Partitioning of proofs to achieve a reasonable run time• Trade-offs between automatic vs manual generation of

assertions

Impact of FE2E

Interrupt Controller (INTC)

IP Architecture Overview

Maskable Level Interrupts

Interrupt Routing

Interrupt Bit-Vectors

Bus Slave

Maskable Output Interrupts

MMIO

Crossbar (XB)

IP Architecture OverviewU

nit 0

Unit 2

Bank Modules

Uni

t 1

Hierarchical Strategy• When an IP has repetitive components, total runtime can be

reduced by separately verifying the IP’s sub-blocks and connections. Proofs can focus on unique interactions and ignore repeated subblocks, or repeated subblocks can be blackboxed with additional constraints.

Divide and Conquer• Dividing large bodies of assertions into separate jobs,

keeping existing limits, abstractions, et al. the same between jobs, yielded a reduction in total runtime.

Abstractifying • Large families of assertions to be reduced to equivalent, single

assertions with limited addition of Verilog in the glue logic. This reduction in property count significantly improved runtime, despite checking identical functionality.

Traditional Strategies to Improve Formal Verification• Tool specific optimizations• Constraining inputs

Managing Runtime Explosion

INTC XBAR Average-UVM-Simple0

20

40

60

80

100

120

140

70 57 71.5

1725

23.50 0

23

Effort Comparison - PersonDays

Effort - RTL 0.5 Effort - RTL 0.8 Effort - RTL 1.0

INTC XBAR Average-UVM-Simple0.001.002.003.004.005.006.007.008.009.00

8.46

3.40 2.79

Total Bugs found by DV per 1K LOC-DV

INTC XBAR Average-UVM-Simple0

5

10

15

20

25

16

3

19.5

Time to First Design Bug (First Bug - First Verif Checkin) Calendar Days

Item INTC XBAR Average-UVM-SimpleEffort - RTL 0.5 70 57 72Effort - RTL 0.8 17 25 24Effort - RTL 1.0 0 0 23LOC-RTL 21000 4500 3100LOC-DV 1300 5000 5350

Total Bugs found by IP Verif 11 17 17Gate Count (Est) 835734 60186 58502Escaped Bugs from L1 1* 1 1

*Bug found after new logic post RTL 1.0

Shallow Cone of Logic• Do any possible states of the IP require more cycles to reach,

than can be tolerated in FE2E?

High Symmetry• The IP reuses sub-blocks or has repeating patterns

Low Code Complexity• If the IP can reasonably be fully specified in SVA, it is likely a

good candidate for FE2E

Avoid Blocks with Known Large State Spaces• Signal Processing Blocks

Recommendations for Selecting IP for FE2E

Make Formal coverage reporting more like simulation based metrics• Add features to improve compatibility with existing coverage

tracking. e.g. merging reports from hierarchical and partitioned FE2E environments.

Provide tools to help appraise how fit an IP is for FE2E

Automate specific methods to reduce runtime that are normally done manually• Automate Divide and Conquer• Provide suggestions to user to blackbox repeated modules

Warnings for properties which are anticipated to not converge in a reasonable amount of time based on certain heuristics

Provide more proofkits for standard interfaces

Suggestions for Future Improvement

Summary• End-To-End Formal IP Verification is well suited for certain types of symmetric and control path intensive designs.

• FE2E is worth the savings in verification effort & resources, compared to a traditional simulation based approach

• We achieve the same quality of verification with FE2E as with traditional UVM testbenches.

• The main challenge is to determine if an IP can be verified using this methodology and manage the proof run times.