Ver. 1.2 1

Q

TPI at Capital One

CVSQAA Meeting11/18/03

Tom Anderson

Thomas.Anderson@capitalone.com

804-967-1688

Lamar Barnett

Lamar.Barnett@capitalone.com

804-934-8860

Ver. 1.2 2

Q

What is TPI?TPI is the Test Process Improvement model for assessing the

maturity of test processes• One of several models generally accepted by testing

profession • Prescriptive model – Tells you how to improve, as well as

describing current state• 4 Levels, A, B, C & D, where D is better than A• 20 different Key Process areas, with levels in each• Assessment is a point-in-time, a basis for improvement

For more info, read:

Test Process Improvement: A Practical Step-by-Step Guide to Structured Testing

by Tim Koomen and Martin Pol, ISBN: 0201596245

Ver. 1.2 3

Q

TPI Model Attributes

• Pinpoints strong and weak points of an organization

• Helps formulate improvement actions

• Factors in life cycle, organization, infrastructure and tools, and techniques

• Independent of development methodology

Ver. 1.2 4

Q

TPI Key Areas

Key Areas

Levels

Improvement SuggestionsCheckpoints

Test Maturity Matrix

Ver. 1.2 5

Q

Types of AssessmentsSelf-assessment• Some or all of the people doing the assessment are

performers of the work being assessedInternal assessment• Assessors are from the same organization as the

performers, but are not normally involved in the same projects

Formal Assessment• Assessment is performed by an independent firm• Assessors usually certified in assessment model and

techniques

In 4Q 2003, QS Enterprise Systems Test Teams will perform

Internal Assessments on each other

Ver. 1.2 6

Q

TPI at Capital OneQuality Services adopted TPI as the model for assessing and improving test

process maturity in 2002Many test teams were assessed by Sogeti in late 2002Used TPI Assessment on OneView program in mid-2003 as a basis to

recommend improvementsTest managers are now being trained to conduct TPI assessments internallyGoal is to do Internal Assessments on all teams by 1/31/04, use as basis for

improvement in 2004

Model Selected

Sogeti Assessments

2003 2004

ImprovementPlans

OneViewAssessment

InternalAssessorTraining

Internal AssessmentsComplete

ImprovementsBased on 1Q04Assessments

Re-assess

2005

Ver. 1.2 7

Q

My Experience in Process Audit

• ISO 9000 training TickIT Lead Auditor

• Duval County School Board HR Dept

• CIS, HTE, London Bridge process audits and improvements

• parts.com

Ver. 1.2 8

Q

Assessing & Auditing

• How can I guarantee a successful assessment?

• Commando audit techniques

• Intelligence work

• Reporting Findings

• Follow up

• OneView example

Ver. 1.2 9

Q

TPI as an Audit Model

• Audit for Compliance Findings – benchmarking, not strict adherence to regulations & policies

• Make practical recommendations to improve work life

Ver. 1.2 10

Q

Audit Interviews?

Why should internal audits involve interviews?– One viewpoint is not as robust.– “Reading between the lines”.

Adherence to achieve compliance is through consistent, uniform practice!

Ver. 1.2 11

Q

What does a successfulaudit interview look like?

Ver. 1.2 12

Q

The Steve Irwin Model

In search of the phenomenon.

Ver. 1.2 13

Q

The Stephen Covey Model

Begin with the End in Mind.

First Things First.

Think Win-Win.

Ver. 1.2 14

Q

The Tom Brokaw Model

Separate Fact From Perception & Feelings.

Ver. 1.2 15

Q

Interview Techniques

Ver. 1.2 16

Q

Interviews – Etiquette & Personalities

• Pre-interview – Provide an agenda & background material.

• Dress for success.

• Be punctual.

• Address with respect.

• Frame open-ended questions with charity.

• Avoid confrontational questioning.

Ver. 1.2 17

Q

Interviews – The Offensive Game

• The Red Herring

• Obfuscation

• Stonewalling

• Disorganized / Can’t Find It

• Interruptions

• Schedule conflicts

• Vacillation, uncertainty, indecisiveness

Ver. 1.2 18

Q

Interviews – The Defensive Game

• Prevent defensive game by preparing.

• Remain calm.

• Remember you can leave.

Ver. 1.2 19

Q

Interview Difficulties

Misunderstandings sometimes occur.– Listen carefully & actively.– Address concerns fully & repeat for feedback.– Stay grounded in facts.

Ver. 1.2 20

Q

Validation of Responses

• Review and complete your notes soon after the interview.

• Refer to documentation.

• Chat informally with others in the group.

• Set up additional interviews.

• Phone or stop by for follow-up questions.

Ver. 1.2 21

Q

Pacing the Assessment

• Two weeks includes interviews, research, and reporting.

• Total effort should be less than 40 hours.

Don’t forget scheduling delay!

Schedule all interviews and the end report up front!

Ver. 1.2 22

Q

Tracking Results

• Develop working hypotheses.

• Prove right or wrong.

• Follow-up with questions & calls.

Ver. 1.2 23

Q

Scoring the Assessment

Ver. 1.2 24

Q

Develop the Score

Key Areadescriptions

Interview questions

Answers to questions

Decisionsabout

Key Areas

AssessmentResults

Ver. 1.2 25

Q

Develop the Score

KA 1.A.1 “A motivated consideration of the product risks takes place…”

Question: “What is your test strategy, and how did you formulate it?”

Ver. 1.2 26

Q

Develop the Score

Ver. 1.2 27

Q

Develop the Score

Ver. 1.2 28

Q

Develop the Score

Ver. 1.2 29

Q

Finalizing the Presentation

• Remember your audience.

• Should be a mere formality.

• Tell them more than they know.

• Provide actionable plan

Ver. 1.2 30

Q

The Report Presentation

Ver. 1.2 31

Q

Reporting Findings

• Management team receives report

• Written as well as verbal presentation

• No surprises

Ver. 1.2 32

Q

Leveraging TPI for OneView

The 20 TPI Key Areas provided concrete suggestions for improvement– Treat testing as a project, dedicate a manager– Consolidate defect reporting with a DB tool– Dedicate professional testing resources– Involve test design resources early

Ver. 1.2 33

Q

Postscriptum

Ver. 1.2 34

Q

The Postscriptum

• After the presentation, follow up will be required. Share the formal document with participants.

• Schedule any f/u meetings.

Ver. 1.2 35

Q

The Take-Home

What’s the take-home?