Incisive Coverage Introduction and RAK...

This also includes Incisive 12.2 workshop/labs overview

Incisive Coverage Introduction and RAK Overview

© 2013 Cadence Design Systems, Inc.

2 © 2013 Cadence Design Systems, Inc. All rights reserved.

Coverage Workshop Agenda

• Introduction to Metric-Driven Verification

• Coverage Metrics

• Code coverage– Generating Code Coverage

– Analyzing Code Coverage Using IMC

– Lab 1: generating & analyzing code coverage

• Functional coverage– Building Functional Coverage Model

– Analyzing Functional Coverage Using IMC

– Lab 2: functional coverage collection & analysis

• IMC Merge/Rank/Refine/Report– Lab 3: merge tests & generate report

• Summary

• Download Workshop – Click here -> Coverage Workshop

http://support.cadence.com/wps/mypoc/cos?uri=deeplinkmin:DocumentViewer;src=wp;q=ProductInformation/Functional_Verification/Incisive_RAKs/Coverage/coverage_rak.zip


APB

Labs and DUT Introduction

• This workshop includes hands-on labs – This is a verification environment for a simple UART DUT

component that is part of a larger SoC system. We follow the steps of instrumenting, collecting, and analyzing the coverage.

– The labs allow further understanding of the lectures.

– To download complete workshop with labs – click here - Download

UART

CPU DMA

UART USB

Memory

Core

AHB

APB

Bridge

Ethernet

Serial UART

protocol

Parallel

APB bus

UART block: Converts parallel

data from the APB bus into serial

data and vice versa

UART

Serial



Test 1

UVM Testbench

UART DUT and Verification Environment

Interface & white-box

coverage

Scoreboard

Multi-channel controller

Imitates external

device by

generating frames

Programs the

UART and

transfers traffic

APB

Verification

Component

Module Verification

Component

UART DUT (Verilog RTL)

APB

InterfaceMode

Switch

Control/Interrupt

Logic

Rx FIFO Receiver

TransmitterTx FIFO

Control/Status

Registers

txd

rxd

UART

Verification

Component

Controls APB and

UART components

Test 2 Test 3

Tests indicate which

stimulus sequences to

execute

APBUART

Serial


Functional Verification Methodologies

Test targets

DUT

Test 1:

Test 2:

Test 3: xTest 4:

Least effective

in finding the

hidden bugs

Directed-Test DrivenResults

DUTG

CoverageCG1

CG2

CG3 x

CG4

Coverage DrivenResults Adds quality &

productivity,

but difficult to

estimate

completionCoverage targets

80%

20%

10% 70%

DUTG

Chip Features

Feature ASubset 1 ….

Subset 2 ….

Feature B

Feature C

Feature D

Verif. Plan

• Feature A

• Feature B

• Feature C

• Feature D

Metric Driven

50%

20%

Feature-based

plan with

extended

metrics

enables

efficient

and accurate

project closure

Results

ABDC

Feature-based

Verification targets

(Metrics for cov+checks)


Plan

Construct

Execute

Measure /

Analyze

Metric-Driven Verification (MDV) OverviewPlanning with Unified Verification Metrics

Done

Yes No

Signoff?

Metric-Based

Executable

Verification PlanPDF

VE Start ProductionPrototypeChip

Integration

Module

Set TwoModule

Set One

Actual Metrics AchievedTarget Metrics Milestones

Missed Milestone

Successful Milestone

VE Start ProductionPrototypeChip

Integration

Module

Set TwoModule

Set One

Actual Metrics AchievedTarget Metrics Milestones

Missed Milestone

Successful Milestone

Failure and

Metric AnalysisIEM

Checks

Assertions

Coverage

IncisiveVIP Portfolio

Testbench Simulation, Formal,HW/SW Co-Sim, LPV, MSV, Sim-Acceleration, Emulation

ISXIFV

IES SNIEV

http://www.uvmworld.org/index.php

http://www.uvmworld.org/index.php


Metric-Driven Verification ProductivityKnow Where You are Going and Get There Faster

Functional

and

Design

Specs

Create Verification

Plan from Spec;

Capturing Metrics

vPlan

Scale Verification Throughput:

Automated Parallel Verification Runs

vs. Manual Engineering Work

Analyze Results

Debug, Fix,

Resubmit

Focus and manage complex

projects using Verification

Planning

Find the most bugs early, and

use analysis to identify and

work on critical bugs first


Coverage Is the Key for MDV

• Coverage implemented to measure goals– Plan specifies Metrics required for DUT features.

– After plan is developed, we need to instrument coverage into the verification environment.

• Coverage implemented to track progress– Plan provides feature-based tracking of progress

– Regression results annotated back to Plan Features

• Let us dive deeper into coverage












• Summary


Coverage Metrics

Code Coverage: method of assessing how well the test cases have

exercised the design.

Functional Coverage: focuses on functional aspects of a design and

provides a very good insight on how the verification goals set by a test

plan are being met.

Code Coverage : Block & Branch, Expression,Toggle, FSM

Functional Coverage : Assertion and CoverGroup


Code Coverage vs Functional Coverage

• Code coverage

• Reflect how thoroughly the DUT code has

been exercised

• Enable detection of

• Untested areas of DUT

• Dead code or unused code

• Most of the work are done by coverage

tools automatically

• Functional coverage

• Measure what features of DUT were

exercised

• Lot of manual work:

• Planning

• What are the features?

• Where/how to measure?

• Coding of assertions and covergroups

DUTUART

UVC

APB

UVC

Scoreboard












• Summary


Code Coverage : Block Coverage

Identifies the lines of code that are executed during a

simulation run. It helps you determine what areas of the

DUT design are not exercised by the testbenches.

block coverage is an essential first step in the overall verification process.

A block is a statement or sequence of statements in

Verilog/VHDL that executes with no branches or delays.

Either none or all of the statements in a block are executed.


Code Coverage : Block Coverage

always @ (in )

begin

$display("TRUE");

if ( in[1] == 1’b1 )

$display("IF1 TRUE");

else

begin

$display("IF1 FALSE");

$display("IF1 FALSE");

end;

$display("TRUE");

if ( in[1] == 1’b1 )

$display("IF2 TRUE");

$display("TRUE");

end

BLK5BLK6

BLK4

BLK3

BLK2

BLK1

Any construct that breaks execution flow creates a new

block, for example:

begin, if, else, case, wait, #, @, for, forever, repeat, while,

disable


Code Coverage : Branch Coverage

The above statement has the following branches:

- If cond1 evaluates to true, out2 is assigned 1’b1

- If cond1 evaluates to false, out2 is assigned 1’b0

assign out2 = (cond1) ? 1’b1 : 1’b0;

Considered as a single block for block coverage. With branch coverage this

statement is considered 100% covered when each branch of the statement

has executed.

• Branch Coverage: Yields more precise coverage details than block coverage by obtaining coverage results for various branches

individually.

• With branch coverage, a piece of code is considered 100% covered when each branch of a conditional statement has been executed at least once.


Block Coverage Analysis Report

This block of statements

was never executed

This block of statements

was never executed

either

Notice that conditional assign

statement was not scored with

block coverage analysis

Score is 0 for the true

part of if statement line 33Score is 0 for the true

part of if statement line 38


Branch Coverage Analysis Report

With “Branch Coverage” enable,

conditional assign statement is

now scored.


Code Coverage : Expression Coverage

Before scoring expression coverage, make sure you have high

block coverage in your regression.

• Expression coverage measures how thoroughly the testbenchexercises expressions in assignments and procedural control constructs (if/case conditions). It identifies each input condition that makes the expression true or false and whether that condition happened in simulation.


3 types of Expression Coverage

• SOP – Sum of Products Scoring (Default Mode)

• Does each term take a 0 and non-0 value?

• Vector inputs are scored as logical (single bit)

• Control Scoring

• Does each term take a 0 and non-0 value *AND*

• Does each term control the output result of the

expression during simulation?

• Vector Scoring

• Does each term take a 0 and non-0 value *AND*

• Does each bit of each term control the result of the

expression?

You can set different scoring modes for selected modules in your design.


SOP Scoring

Reduces expressions to a minimum set of expression inputs that

make the expression both true and false, inherently first-level

b & (c | d)

<1> <2> <3>

hit rval <1> <2> <3>

1 1 1 - 1

1 1 1 1 -

0 0 - 0 0

1 0 0 - -

Expression Inputs

<1>: b

<2>: c

<3>: d

Resulting value of the expression

(either zero or non-zero)

Simulation

Scoring

Count

Coverage

Hole


Control Scoring

Checks if each input has controlled the output value of the expression

at some time during the simulation

− Known as “sensitized condition coverage” or “focused condition coverage”

− Breaks an expression into a hierarchy of sub-expressions, more accurate

b & (c | d)

<1> <-- 2 -->

<3> <4>

hit <1> <2>

& --------------

0 0 1

0 1 0

1 1 1

hit <3> <4>

| --------------

0 0 1

0 1 0

1 0 0

Level 1:

<1>: b

<2>:(c | d)

Level 2:

<3>: c

<4>: d


Vector Scoring

Vector scoring mode is an extension of control scoring mode.

Each bit of a multi-bit signal is scored and reported separately and

lots of data to analyze.

reg [3:0] a, b;

Wire [3:0] val = (a && b);

(a && b)

<1> <2>

hit <1> <2>

&& --------------

0 0 1

0 1 0

1 1 1

Hit T1[3] [2] [1] [0]

0 0 0 0 1

0 0 0 1 0

0 0 1 0 0

1 1 0 0 0

0 0 0 0 0

Level 1:

<1>: a

<2>: b

Bit table for each

vector operand

hit T2[3] [2] [1] [0]

0 0 0 0 1

0 0 0 1 0

0 0 1 0 0

1 1 0 0 0

0 0 0 0 0


Expression Coverage Analysis Report

Source code window & selected

expression

Detailed expression reports

(v1 == 1) was never false

(v2 == 1) was never false

Need additional test cases?


Coverage Metrics – Toggle Coverage

• What does toggle coverage do?– Collect and report design signal toggle activity

• What is a design signal toggle?– Normally a binary transition (and return after a finite delay) of a DUT

signal.

– Signals may transition through (but not terminate at) an unknown state.

• Why bother with toggle coverage?– It’s the only “code” coverage available for a gate-level netlist

– Verify that design interconnect is connected and “wiggles”

1

Z

0

partial full set_toggle_includez


Code Coverage : FSM Coverage

• FSMs are critical control logic of DUT

• It is important that FSMs logic are exercised thoroughly

by the testbench to ensure that there are no bugs

– FSM coverage measures how well this logic is exercised

- State Coverage : reports what states were visited

- Transition Coverage : reports what transitions occurred

- Arc Coverage : reports why each transition occured

FSM Coverage Types


Generating Code CoverageWhat coverage metrics should I enable?

• Add in stages code coverage to identify holes in test environment– First-order: block or branch

– Second-order: expression

• Add in stages the simplest form of functional coverage in the form of FSM coverage– First-order: states and transitions

– Second-order: arcs

Note: There is little value in scoring second-order coverages if

simpler ones like block or branch still show low coverage












• Summary


Generating Coverage DataTwo-Step Simulation

Design Testbench

irun -c

irun -r

coverage

options

OR

coverage

configuration

file

Coverage

Database(s)

+

Compile,

Elaborate and

instrument for coverage

Simulate, score and store

coverage


Generating Coverage DataIRUN coverage options

•B Enable Block coverage

•E Enable Expression coverage

•F Enable FSM coverage

% irun –f rtl_debug.f –coverage B:E:F –covdut chipA –covdut chipB

-coverage <coverage_types> Enables coverage data generation

•T Enable Toggle coverage

•U Enable Functional coverage

•A Enable ALL coverage types

-cov_file <cov_config_file> Specify coverage Configuration File

-covdut <DUT_module> Specify Module for instrumentation

% irun –f rtl_debug.f –coverage A –covdut chipA

Instrument Block/Expression/FSM coverage for Module chipA & chipB

Instrument All coverage types for Module chipA


Generating Coverage Datairun runtime commands

-covworkdir <workdir> Basename for the work directory

Default work directory is cov_work

-covscope <scope> Specifies an alternate directory for

storing coverage model files.

Default is scope

-covtest <test> Test directory name. Default is test

/home> irun -r <snapshot>

Invoke irun (during run time) with coverage instrumented during

elaboration

-covoverwrite Enable overwriting of coverage files

Coverage model generated in /home/cov_work/scope (.ucm)

Coverage data in /home/cov_work/design/test (.ucd)


Generating Coverage DataDirectory & file structure created for coverage

% irun –c *.v –coverage B:E:T –covdut chipA

./

% irun –r top –covtest testA ...

% irun -r top –covtest testB ...

cov_work/

scope/

testA/

.ucd

testB/

.ucd

.ucm

% irun –r top –covworkdir regr_10 –covtest testC

regr_10/

scope/

testC/

.ucd

.ucm

Module Hierarchy

chipA

mod2

top

mod1


Generating Coverage Data

IRUN Coverage Summary of Commands

Switch name Compile/elab/

runtime

Description

-coverage <string> elaboration Enable coverage

instrumentation

-covfile <file> elaboration Coverage instrumentation

control file

-covdut <dut> elaboration Coverage DUT name

-covworkdir <workdir> runtime Coverage work directory

-covscope <scope> runtime Coverage scope name

-covtest <test> runtime Coverage test name

-covoverwrite runtime Overwrite coverage output

files


Generating Coverage DataDefining a Coverage configuration file

select_coverage –bet –module chipA…

deselect_coverage –t –module mod2…

select_coverage –t –module mod3

# This is a comment

set_branch_scoring

select_coverage -fsm –module mod1

select_functional

chipA

top

mod1

covfile.ccf

mod2

mod3

Run simulation with coverage according to configuration file covfile.ccf

% irun *.v –cov_file covfile.ccf

Enable B:E:T in chipA and all its descendents

Disable Toggle in module mod2…

Enable Toggle in module mod3

Score branches (ternary assignments)

FSM Extraction in mod1

Score assertions / Covergroups

Matches all instances of mod and all of their descendents


Coverage Configuration File

• Coverage Configuration File Command

Command Applies to Description

select_coverage B:E:T:F Select the design units or instances to score for coverage

type

deselect_coverage B:E:T:F Subtract from what is instrumented

-remove_empty_instances remove empty instances from

the coverage hierarchy

set_glitch_strobe B:E Specify the glitch rejection time interval

set_hit_count_limit B:E Sets the upper limit of hit counts

set_subprogram_scoring B:E Disable scoring of unused subprograms

set_com B:E:T Identify items never happen and marks them ignored

set_com_interface B:E:T All ports of the modules will be marked as variables

set_code_fine_grained_m

erging

B:E Provide merge resilience at concurrent block level instead of

the default module/instance level

set_libcell_scoring ALL Enable scoring of verilog library cells

set_merge_with_libname ALL Match library name of module for merging

set_parameterized_modul

e_coverage

ALL Different modules dumped to different parameter values

combinations used to instantiate the module



• CCF Command apply only to Block Coverage

Command Description

set_implicit_block_scoring -off: disable scoring of implicit else and/or

default case blocks

set_explicit_block_scoring -off: disable scoring of explicit else and/or

default case blocks

set_assign_scoring Enable scoring of verilog continous

assignments

set_branch_scoring Enable scoring of individual branches

set_statement_scoring Enable statement coverage scoring



• CCF Command apply only to Expression Coverage

Command Description

set_expr_scoring -all: enables scoring of all operators and assignments

-sop/control/vector: specifies a scoring mode

-event: enables scoring of verilog events

-no_vhdl_shortcircuit: disables short-circuits

evaluation of VHDL AND/NAND/OR/NOR operators

-vlog_remove_redundancy: disables splitting of

subexpressions

-vhdl_not_as_operator: enables scoring of VHDL not

operator

set_expr_coverable_operators By default, coverage is scored only for Verilog logical

operators (|| and &&) and VHDL logical operators

(OR, AND, NOR, and NAND), and is scored only in

condition expressions. This comand enables scoring

coverage for all operators in condition expressions.



• CCF Command apply only to Toggle Coverage

Command Description

set_toggle_strobe Define a strobe interval for filtering glitches within the

time window on nets

set_toggle_limit Set maximum count for rise and fall transitions after

which ncsim stop counting

set_toggle_includex Record transitions x -> 0 and x -> 1.

set_toggle_includez Record transitions z -> 0 and z -> 1.

set_toggle_noports Disable scoring of module ports

set_toggle_portsonly Score only module ports

set_toggle_excludefile Exclude specific signals from the design during toggle

simulation



• CCF Command apply only to FSM Coverage

Command Description

set_fsm_attribute Tag an FSM state vector

set_fsm_reset_scoring Enable scoring of reset states and transitions for

identified FSMs.

set_fsm_arc_scoring Enable scoring of arcs for identified FSMs.

set_fsm_arc_termlimit Set the limit on the number of input terms created for an

arc expression

set_fsm_scoring -hold_transition: enable scoring of FSM hold transitions.



• CCF Command apply only to Functional Coverage

Command Description

set_optimize -ial_ovm_inst_asrt: treat instances of IAL and OVL components as

individual assertions

select_functional Enables scoring of all compiled assertions and covergroups

-ams_control: enables scoring of assertions coverage in AMS

modules

set_covergroup Multiple switches that allow users to “fine tune” the collection and

generation of coverage data; see ICC User Guide for more details.












• Summary


Incisive Metrics Center (IMC)Key Features

• Unified metrics center for all languages andall coverage metrics types

– Code (block / expression / toggle)

– FSM

– Functional (Assertions / covergroup)

• State of the art GUI capabilities to browse andanalyze coverage

– Instance and Type based views

– Robust filtering and sorting capability

– Intuitive page navigation

• Unified batch mode Command Line Interface (CLI)

• Good merge, rank, refinement and report(text, html) operations.

• Exclusion


2 modes to launch IMC

GUI Mode: This invokes Incisive Metric Center, which is a

graphical user interface to observe coverage results,

compare simulations, and refinement file

% imc

% imc -gui

Batch Mode: In this mode, imc command prompt is

launched allowing user to issue IMC commands

interactively. Alternately, all commands can be placed in a

<command_file> which is passed to imc for execution.

% imc –batch [command_file]


IMC Help Utilities

“How To”s display

topics available

for searchFavorites page

displays list of

pages added as

favorites

Analysis History

page displays list

of runs launched

in this or previous

IMC sessionsSearch

“How To”s


IMC Summary View

“Activity Center”

Page Navigation

Unified

Instance/Type

Verification

Metrics Tree

Info Tab

Sub-elements

Details Tab

Tools & Options

Sorting / Filtering


IMC Block Analysis View

Block

Coverage

Analysis

Green Indicates covered ;

Red indicates uncovered


IMC Expression Analysis View

Various

Expressions

Selected

Expression

Selected

Expression

Table

Source Code


IMC Toggle Analysis View Source Code

Individual

Vector bits


IMC FSM Analysis View

State

Transitions / Arcs

FSM bubble

diagram

Source viewer

with syntax

highlighting


Lab 1: Code Coverage












• Summary


Functional Coverage – What are we measuring?

• Stimulus completeness– All address blocks

– Packet sizes: short, jumbo, max

– All instructions

– Important sequences – Read-after-write

– Store* - Branch – Read*

• Typically, implemented with SV covergroups

• Where should we implement these covergroups?– In data items (uvm_sequence_item, uvm_sequence) directly?

– In monitors?

– In scoreboards?

DUTUART

UVC

APB

UVC

Scoreboard


Implementing Coverage for Stimulus – Unique Data Item

• If “data item definition” is unique for each DUT port: – Covergroup(s) can be defined in the data item itself

– Type-coverage measures the completeness of the stimulus

• In the example above, APB and UART ports have unique data items– Covergroups can be defined directly in the apb_transfer, uart_frame

– Type-coverage of apb_transfer measures the apb stimulus coverage

– Type-coverage of uart_frame measures the uart stimulus overage

DUTUART

UVC

APB

UVC

Scoreboard

apb_transfer uart_frame


Covergroup Example – in data item

DUTUART

UVC

APB

UVC

Scoreboard

apb_transfer uart_frame

Covergroup in

apb_transfer

data item


Implementing Coverage for Stimulus – “Shared” Data Item

• If “data item definition” is not unique for each DUT port: – Covergroup should be defined in the monitor or scoreboard

– Instance-coverage measures the completeness of the stimulus

• In the example above, DUT has two APB ports– Do not define covergroup in the apb_transfer

– Covergroup of APB stimulus should be defined in the monitor– Instance-coverage of monitor in Agent1 = stimulus coverage of APB Port1

– Instance-coverage of monitor in Agent2 = stimulus coverage of APB Port2

DUTUART

UVC

Scoreboard

uart_frameAPB

UVC

apb_transfer

apb_transfer

APB Port1

APB Port2

Agent 1

Mon

Agent 2

Mon


Covergroup Example – in monitor

DUTUART

UVC

Scoreboard

uart_frameAPB

UVC

Agent1

Agent2

apb_transfer

apb_transfer

APB Port1

APB Port2

Enable instance

coverage

Use set_inst_name()

to specify meaningful

instance name


Explicit naming of covergroup instances

• Do not rely on the default (tool-generated) name– Covergroup instance could have multiple paths (object handles)

– ==> default name may not always be logical enough for users to follow

– Default name could be different between tests => merge issue

• Two techniques are available for naming the covergroupinstances explicitly– Use of set_inst_name() built-in method

– Use of option.name

• We recommend use of set_inst_name() for naming the covergroup instances explicitly


Functional Coverage – What else are we measuring?

• DUT features

– Operational modes– 1GB, 10GB

– 720p, 1080p

– Corner cases– FIFO empty/full

– Wrong branch prediction -> flushes of prefetch queues

• Typically, implemented with covergroups and assertions

• Where should we implement covergroups & assertions?– Register model (UVM_RGM or UVM_REG)

– Monitors, scoreboards

– vunit

DUT UVC2UVC1

Scoreboard


Covergroup Example – in Register Model

Generated

automatically by

Register package


Covergroup Example – FIFO Corner Cases

Covergroups to

check for corner

cases of FIFOs












• Summary


Building a Functional Coverage Model

• Coverage sampling is expensive‒ Determine minimum number of variables required to verify a design feature

‒ Don’t sample every clock; use sample event or call .sample()

• For each feature of the DUT:– Determine what variables we need to sample to verify this feature

– Determine where to implement covergroups for these variables – In data items definition?

– In monitors?

– In scoreboards?

– Group the values of each sample variable into meaningful “bins” – Name them explicitly => easier for report analysis


Building a Functional Coverage Model

• For each feature of the DUT (continue):– To determine if certain condition occurred, you might need to cross

the value of two or more variables. – These variables need to be in same covergroup

– Cross coverage can generate a lot of data; use “ignore_bins” and “illegal_bins”

– If you don’t do it now, you will have to “exclude” those during analysis phase

– Check the “Covergroup type options” (e.g., weight, goal)

– Name the covergroup using set_inst_name() to help with analysis and merging

• Review your coverage model with the design team to ensure accuracy












• Summary


IMC Assertion Analysis View

Assertion Finished /

Failed Counts

List of Assertions

Source Code


IMC Covergroup Analysis View

List of Covergroups

Coverpoints

Bins


Incisive Metrics Center Search Capability

Search on a tree

Table search bar


Incisive Metrics Center Refinement Capability

Refinement can be applied on all metric kinds including block,

Expression (sub-expressions), toggle signal (individual bits), Covergroup,

Coverpoint, bins, cross-bins, assertions ,etc.

Batch Mode also supports ability to save/load refinement files

Exclude items are gray

out by default

But can be

hidden as well

Exclude (recursively)

Exclude Local (non-recursive)

Exclusion Ability to save/read

current state to/from a file


Lab 2: Functional Coverage












• Summary

Merging Coverage Data


Merging Coverage Data

• Coverage analysis generally is performed on aggregated data from multiple tests run on one or more design configurations.

• Coverage data must be merged to provide a comprehensive view of what is covered.

72 © 2013 Cadence Design Systems, Inc. All rights reserved.72

Merging Coverage Data with IMC“merge” command in IMC

./

cov_work/

scope/

test2/

test1/

test3/

% imc -batch

imc> merge test1 test2 test3 –out all

imc> merge * –out all

all/

OR

./

regr_jan7_12 /

system1/

test2/

test1/

test3/

all/

% imc -batch

imc> merge regr_jan7_12/system1/* –out

regr_jan7_12/system1/all

IMC assumes tests are under cov_work/scope/

ORimc> preferences –set workDir regr_jan7_12

imc> preferences –set scopeDir system1

imc> merge * –out all

merge only supported in IMC batch mode


Merge Command

merge

-out <out_path>

[-overwrite]

{<runs> | -runfile <runfile>}

[-metrics <metrics_type>]

[–run_order {as_specified | by_timestamp}]

[–initial_model {empty | <runpath> | union_all |

primary_run}]

[-message [<message_level>]]

[-covmodeldir <modeldir>]


Merge CommandInitial Model

• Initial Model affects how the Merged coverage model looks like

• -initial_model– <runpath>: <runpath> coverage model used as the starting target

model

– primary_run: Primary test coverage model used as the starting target model.– Primary test is the first test used to merge (based on run_order)

– union_all– Target model is the union of all test coverage models

– empty– User creates a target model

– Based on: -target, -targettype, -source, -sourcetype


Initial Model: <runpath> or primary_run

imc> merge Run1 Run2 –initial_model primary_run –out all


Initial Model: union_all

imc> merge Run1 Run2 –initial_model union_all –out all


Merge Config – System to Block level merge


Merge Config – Block to System level merge

merge_config –sourcetype A –targettype A

merge Run1 Run2 –initial_model Run1 –out merged_run

merge_config –source System.A1 –targettype A

merge_config –source System.A2 –targettype A

merge Run1 Run2 –initial_model Run1 –out merged_run


Merge Config – Create New Hierarchy


Merge Config – Create New Hierarchy (one more)

Ranking Coverage Data


Ranking runs in IMC

• Ranking runs helps identify a subset of runs that have the lowest cost and still provide the same coverage grade as the full test suite.

• By ranking the runs one can identify the runs that:– Make the most efficient test suite for regression testing eliminating the

redundant runs.

– Provide the best possible coverage considering the CPU time or the simulation time.

– Help in achieving the target coverage goal.

• Ranking the runs involves:– Setting the Ranking Configuration using the rank_config command (optional)

– Ranking the Runs using the rank command

• NOTE: Ranking runs does not require loading of runs.


IMC rank_config Command

• metric_type_weight_option– increases or decreases the weight of a given metric type. The default

weight if not specified is 1 for all metrics. If weight is 0, the coverage metric type is not considered while ranking– -w_block <weight>

– -w_expression <weight>

– -w_toggle <weight>

– -w_fsm <weight>

– -w_assertion <weight>

– -w_covergroup <weight>

• -max <integer_value>– specifies the number of runs after which ranking runs must stop

– helps identify the best N runs.

• -target <cumulative_coverage_goal>– Specify target goal to be achieved while ranking

– ranking stops when goal hit


IMC rank_config command (continued)

• -initial_runfile <runfile>– Specifies a runfile that contains a list of runs that must be included

always in the ranking results.

– The runs specified in the <runfile> appear above the runs specified for ranking.

• -cpu | -sim– Specifies if CPU time or Simulation time should be considered for

ranking. By default, neither CPU time nor simulation time are considered when ranking


IMC rank command

• <runs> | -runfile <runfile>– lists runs to rank or place in <runfile>

• -output <output_file>– Specifies the name of the output file where the ranking results are

saved. By default ranking results are written to the standard out, unless redirected to a file

• -run_order {as_specified | by_timestamp}– specifies the order in which the runs are processed during model

union stage – as_specified : process runs in order specified in command line or runfile. This is

the default.

– by_timestamp : process runs in reverse chronical order, the latest database is processed first


IMC rank command (continued)

• -use_prev_merge– helps rank runs that were merged earlier

• rank_element_option– specifies the elements for which coverage grading is calculated during

ranking. By default ranking is done for all the elements in the design.– -type <types> : specify the module/entity for which coverage grading are

calculated during ranking

– -inst <instances> : specify instances for which coverage grading are calculated during ranking

– -covergroup <covergroup_element_full_path> : specify the covergroups for which coverage grading are calculated during ranking

– -coverpoint <coverpoint_element_full_path> : specify the coverpoints for which coverage grading are calculated during ranking

– -assertion <assertion_element_full_path> : specify the assertions for which coverage grading are calculated during ranking


IMC rank command examples

• imc> rank test1 test2 test3 test4 -type ALU

Rank runs considering only the coverage grade of module/type ALU

• imc> rank_config -target 85

• imc> rank test1 test2 test3 test4

Rank runs to achieve a cumulative

coverage goal of 85%

• imc> rank_config -w_block 0 -w_expression 0

• imc> rank test1 test2 test3 test4

Ignore block and expression coverage


IMC rank command examples (continued)

Doing model union of IUS runs ...

With primary run (cov_work/scope/test1): cov_work/scope/icc_59616df2_5bf8faed.ucm

IUS target model generated successfully.

Coverage ranking options:

============================================================================

Rank elements: Model

Weight: Block = 1, Expression = 1, Toggle = 1, Fsm = 1, Assertion = 1, Covergroup =

1

Ranking Cost: N.A

Target Cumulative Grade(%): 100

Number of Best Runs: 4

Ranking of coverage runs:

Run Cum. Self Cum. Wt Self Wt Contrib. Run

Id Grade(%) Grade(%) Grade(%) Grade(%) # items Cost Name

============================================================================

Optimized Runs:

1 58.18 58.18 58.18 58.18 6634 0 cov_work/scope/test2


Redundant Runs:



imc>rank test1 test2 test3 test4 –out rank_result

Ranking

configuration

Optimized Runs

Redundant Runs

HTML reporting


Analyzing Coverage DataGenerating Text Reports in IMC

% imc -batch

imc> load -run all

imc> load –refinement my_allexclude.vRefine

imc> report -summary –inst [-html] –out myreportsum.html

*…

imc> report -detailed –inst [–html] -out myreport.det *…

Load run “all” containing merged data from all tests

Optional: Load refinement file (if any)

Generate Summary Report for all instances

Generate Detailed Report for all instances


IMC: HTML reportsExample of a report – Top Level Summary

Concise useful HELP is one click away

High-level summary

Link to flat CoverGroup coverage – all CoverGroups in one table

Coverage Numbers

Navigate hierarchyTotal (average)

Key data


IMC: HTML reportsExample of a report - Summary

Click column header to sort table

Click link to see details Click instance to descend

Click to open detailed report for instance


IMC: HTML reportsExample of Global CoverGroup Summary Report

Actual coverage

Covergroupoptions

Bin details Covergroupinstance name


IMC: HTML reportsExample of Per Covergroup Coverage

Click on item to view details


Lab 3 – Coverage Analysis












• Summary


Coverage Guidelines Summary – Page 1

• Start with verification planning

• Functional coverage design– For each design feature:

– Determine minimum number of variables required

– Select sampling event carefully

– Check the “Covergroup type options” (e.g., weight, goal) to ensure correct grading

– Name the covergroup using set_inst_name()

• Running tests and collect functional coverage

• Once functional coverage reaches target & design is stable, start code coverage



• Code coverage– Start with Block coverage + FSM coverage

– Add branch coverage

– Expression coverage for critical blocks; work with designers

– Is SOP good enough?

– If not, work with designers to determine if Control or Vector is required

– Toggle coverage for interfaces between sub-blocks

• What about SOC level?– Functional coverage determined by the SOC verification plan

– What about code coverage?

– Since the concerns here are: interconnect between blocks and “glue” logic– Toggle coverage for interconnect between blocks

– Block/branch/expression? coverage for “glue” logic



• Be selective about what you measure & collect– Capture only what you really need otherwise….

– …you might have to spend lot of time analyzing results and writing exclusions!


Additional Documentation / Tutorial

IMC User Guide:

<Incisiv_install_dir>/doc/imcuser/imcuser.pdf

IMC Quick Reference Guide:

<Incisiv_install_dir>/doc/imcug/imcqrg.pdf

ICC User Guide:

<Incisiv_install_dir>/doc/iccug/iccug.pdf

To Download this workshop – Click here -> Coverage Workshop

SOC and IP level Functional Verification - Rapid Adoption Kits (RAK)

Download Landing Page


http://support.cadence.com/wps/mypoc/cos?uri=deeplinkmin:DocumentViewer;src=wp;q=ProductInformation/Functional_Verification/Incisive_RAKs/Incisive_RAK.htm

Support Contact

http://support.cadence.com

http://www.cadence.com

http://www.cadence.com/

http://www.cadence.com/

Incisive Coverage Introduction and RAK...

Documents

Transcript of Incisive Coverage Introduction and RAK...