Minimal Instrumentation for Software Feature Location

Sharon Simmons

Dennis Edwards

Norman Wilde

Eric Daniels

UWF - Simmons 2

Software Reconnaissance

Fall 2009 Showcase

Where in this program is feature X implemented?

100’s KLOC, 10’s MLOCModification requiredInteractions forgotten

2. Run tests with the featureFeature is

active

Feature is NOT active

~Set difference

Solution: track execution

3. Compare what was executed

1. Run tests without the feature

Components executed in the first group of tests, but not in the second groupMarker ComponentsBegin exploration here!

UWF - Simmons 3

(demo of version 3) http://www.cs.uwf.edu/~recon/recon3/r3wDemo.htm

TraceGraph 4

Fall 2009 Showcase

RowInstrumentation point

Columntest case

Col 1without feature

Col 2+with feature

Identifies “markers”

Gives big picture

Subroutine (En… 0 be_aas_bspq3.c /research/ope… 147

UWF - Simmons 4

Software Recon Status

Fall 2009 Showcase

Developed for several years at UWF– Stable, usable, and free– Published works enumerate benefits and limitationsBrief Benefits– Identifies a small number of marker components– Identifies good places to start efficient code exploration– Can decrease software maintenance time

UWF - Simmons 5

Instrumentation AlternativesAtom (Digital Equipment Corp)

– Instrumentation at load time

Pin (Intel Corp)– Instrumentation at dynamic link time

CodeTEST (Freescale Semiconductor)– Instrumentation of preprocessed source code

Recon (UWF)– Instrumentation of source code

Gcov (GNU)– Instrumentation by (GNU) compiler

Fall 2009 Showcase

None completely successful

UWF - Simmons 6

Problems Encountered

Fall 2009 Showcase

Newer systems time-sensitiveDeadlines must be metTracing can not be intrusiveTarget must execute normally

Initialization was hours instead of minutesDeadlines were not metTracing intrusive and prevented normal execution

Hard / soft real-timeRace conditionsTiming faults

UWF - Simmons 7

• Instrument preprocessed source• Efficient code coverage tracing• Build IDB prior to execution• Provide facilities to partition execution into

intervals• Simplify installation and use

Fall 2009 Showcase

Our Approach

UWF - Simmons 8

Information Flow

Fall 2009 Showcase

Driver

Collection Engine

Target

IDB

Bitstring Traces

Trace Records

Data Joiner

Source

Instrumenter

Interval Delimiter

Test

Cases

During Compilation

During Execution

After Execution

UWF - Simmons 9

MinInst Overview

Fall 2009 Showcase

Instrumentation point represented as bit– 1MB RAM ≈ 8.4M instrumentation points

Ordered set of bits (bit string) represents code– Bit index identifies IDB record– Bit string cleared at interval start– Bit set when instrumentation point executed

0 not executed; 1 executed– Execution order not retained– Execution frequency not retained– Bit string saved at interval end

UWF - Simmons 10

MinInst Details

Fall 2009 Showcase

Two bitstrings allocated in shared memory– Active (A): currently accessed by target– Passive (P): prepared for next interval

Collection Engine– Started as first instruction in main()1. Wait for signal (no overhead here)2. Exchange A & P bitstrings (pointer swap)3. Save P bitstring to trace file (I/O overhead)4. Clear P bitstring for next interval5. Go back to step 1

UWF - Simmons 11

Version A• Instruments C/C++• Linux platform• CodeTEST

– Identifies instrumentation points– Builds IDB

• MinInst– Inserts instrumentation code– Builds bit index IDB map

• Driver– Delimits trace intervals– Performs test cases

• Collection Engine– Records bitstring trace at runtime

• Joiner– Combines IDB, MAP and bitstring trace– Creates usable trace file

Fall 2009 Showcase

UWF - Simmons 12

Information Flow

Fall 2009 Showcase

Driver

Collection Engine

Target

IDB

Bitstring Traces

Trace Records

Data Joiner

Source

Instrumenter

Interval Delimiter

Test

Cases

During Compilation

During Execution

After Execution

Map

CodeTEST

MinInst

UWF - Simmons 13

Test CasesGenerated Code

– 1 file, simple functions (5 stmts, 5 inst pts)– 15KLOC, 30KLOC, 45KLOC, 60KLOC– CodeTEST: +200% MinInst: +11%

Apache web server (httpd)– 550 files, 300KLOC, multithreaded, multiprocess– CodeTEST: +717% MinInst: +1%

Openarena & OpenGL– 1K files, 700KLOC, time sensitive, multiplayer, networked– CodeTEST: +9371% MinInst: +6%

Fall 2009 Showcase

UWF - Simmons 14

Shortcomings• Complexity of use• Reliance on CodeTEST• CodeTEST failed some test cases• Execution differences between platforms• Licensed software• Unsupported for a while

Fall 2009 Showcase

UWF - Simmons 15

Version B• Prototype written in C for C/C++ source

– Removed CodeTEST dependency – Tested in Linux– Windows port is pending

• Simplified installation and use– Installation/removal scripts– Integrated into GCC/Make in Linux– Integrated into Visual Studio IDE

• Interval options– Timed or user controlled

Fall 2009 Showcase

UWF - Simmons 16

Information Flow

Fall 2009 Showcase

Driver

Collection Engine

Target

IDB

Bitstring Traces

Trace Records

Data Joiner

Source

Instrumenter

Interval Delimiter

Test

Cases

During Compilation

During Execution

After Execution

MinInst

UWF - Simmons 17

SummaryVersion B status

– Execution time unaltered from Version A– Linux and Windows XP– GCC and Visual Studio– Test cases in progress

Next steps– Conclude test cases– Northrop Grumman large system testing– Complete distribution package

Future Work– Modify TraceGraph for bit string operation

Fall 2009 Showcase

UWF - Simmons 18

Information Flow

Fall 2009 Showcase

Driver

Collection Engine

Target

IDB

Bitstring Traces

Source

MinInst

Interval Delimiter

Test

Cases

During Compilation

During Execution

After Execution

UWF - Simmons 19

Contact Information

Sharon Simmons ssimmons2@uwf.edu

Dennis Edwards dedwards@uwf.edu

Norman Wilde nwilde@uwf.edu

Fall 2009 Showcase