CSE Senior Design IICSE Senior Design IITest PlanningTest Planning

Mike O’DellMike O’Dell

Based on an earlier presentation by Based on an earlier presentation by Mike O’Dell, UTAMike O’Dell, UTA

1

CSE 4317 2

Test Planning … when?Test Planning … when?Occurs throughout the product life cycle

Requirement completeness Architectural integrity Component design precision Implementation correctness

But all too often we wait until the end to “validate” what we have already done.

1

CSE 4317 3

System VerificationSystem Verification

SRD: System Requirements

ADS: ArchitectureSpecification

DDS: Detailed Design Specification

Implementation

Integration Test

Component Test (a.k.a.

Function Test)Unit Test

System Validation

Test

System Verification

System Definition

MAP:All SpecifiedRequirements

MAP:All ModuleInterfaces &Interactions

MAP:All Modules

MAP:All Subsystem & LayerInterfaces &Interactions

1 System VerificationSystem VerificationUnit Test: verifies that EVERY module

(Hardware & Software) specified in the DDS operates as specified.

Component/Function Test: verifies integrity of ALL inter-module interfaces and interactions.

Integration Test: verifies integrity of ALL inter-subsystem/layer interfaces and interactions.

System Verification Test: verifies ALL requirements are met.

CSE 4317 4

1 System Test Plan ContentsSystem Test Plan Contents IntroductionExecutive Summary (purpose, scope, etc.)References

SRD (requirements drive the system-level tests you do)

ADS (architecture drives the subsystem and integration testing)

Etc. (detailed design, standards, sample plans, etc. that are test drivers)

Test Item(s) – What will be tested

CSE 4317 5

1 System Test Plan ContentsSystem Test Plan ContentsDesign decomposition (relationship of specific

modules to test plan, test cases) Limitations of product under test (restrictions,

assumptions, caveats, etc.)Other product-level restraints on testingRisks (in test phase)

Specific risks that may affect testing/test outcome Impact assessment and management plan as it

relates to these risks

CSE 4317 6

1 System Test Plan ContentsSystem Test Plan ContentsFeatures to be tested

List of specific features (from the USERS’ viewpoint) that you will test

Features NOT to be tested List of those features (USERS’ perspective) that

cannot/will not be tested Describe rationale for not testing each item (not in

current release, insufficient test capability, low risk feature that is hard to test, not a user-visible feature)

CSE 4317 7

1 System Test Plan ContentsSystem Test Plan ContentsApproach (Strategy)

Overall test strategy (what, why, and how of your test plan)

Number of hardware and software configurations tested/not tested

Plan to deal with defects identified (re-integration, regression)

Identify metrics for overall success (number of bugs found/corrected/open, number of critical features passed, etc.)

Identify any special requirements for testing

CSE 4317 8

1 System Test Plan ContentsSystem Test Plan Contents Item Pass/Fail criteria

List specific criteria for each unit/module/feature to be tested (number and severity of defects, any specific tests that indicate system failure, etc.)

Test Deliverables List what is to be delivered by this test plan (plan

document, test cases, error logs, etc.) Incident/Bug/Problem Reports (IRs)

Test Schedule (should be consistent with your published project plan)

CSE 4317 9

1 System Test Plan - ResourcesSystem Test Plan - ResourcesIEEE Std 829-1998, IEEE Standard for

Software Test Documentation

Test Plan Outline for IEEE Format (from Gerrard Consulting)

CSE 4317 10