Optimal Testing Strategy For Product Design

Wei WangPengbo Zhang

Apr 15 2009

4/15/2009 1

Outline

• Introduction• Pandora’s Box• Role of Learning in sequential testing• More complicated testing strategy

4/15/2009 2

Introduction• Why are new product design testing needed? – Decreased market uncertainty– Optimal candidate solution

• What are sacrificed?– Direct testing cost (Wind tunnel etc.)– Indirect opportunity cost (testing time)

• How are tests carried out ?– Sequential – Parallel – Mixed

• Optimal testing strategy needed!4/15/2009 3

Pandora’s Problem• n closed boxes• Box i contains reward xi with distribution Fi(xi)

independent of all others• x0 initial wealth to start • Cost ci to open box i with a time lag of ti

• Benefits discounted at rate r• Pandora opens one box at each stage • She can choose to stop searching and collect at

that time the maximum reward thus far uncovered• Wanted: A sequential decision rule that can

maximize the expected discounted value4/15/2009 4

Formulation

4/15/2009 5

Learning• Erat, S., Kavadias S., Sequential Testing of Product

Designs: Implications for Learning, 2008• Pandora’s Box assumes all testings (opening a box)

are independent• Interactions between design attributes and

similarities between design configurations• Outcome from testing one configuration relevant in

understanding and predicting the performance of subsequent tests

• Similar DP formulation with correlation/covariance matrix added to model transferable learning

4/15/2009 6

Formulation

4/15/2009 7

Testing Strategies• Loch et al. Parallel and sequential testing of

design alternatives, 2001

Parallel testingProceeding more rapidlyGood when lead time is important

Sequential testingLearning, lower costGood when cost per test is important

4/15/2009 8

Formulation

4/15/2009 9

Formulation

4/15/2009 10