Download - Sandra Slaughter Donald E. Harter Soon Ang Jonathan Whitaker

11/01/200511/01/2005 American UniversityAmerican University 11

Contract Choice and Product Contract Choice and Product Quality Quality

Outcomes in Outsourcing:Outcomes in Outsourcing:Empirical Evidence from Software Empirical Evidence from Software

DevelopmentDevelopment

Sandra SlaughterSandra SlaughterDonald E. HarterDonald E. Harter

Soon AngSoon AngJonathan WhitakerJonathan Whitaker


Software QualitySoftware Quality

ProblemProblem In April, a software glitch resulted in the loss of In April, a software glitch resulted in the loss of

thousands of dollars for US Airways Group Inc. thousands of dollars for US Airways Group Inc. when some tickets were mistakenly priced at when some tickets were mistakenly priced at $1.86. (ComputerWorld 7/25/05)$1.86. (ComputerWorld 7/25/05)

A software bug apparently caused the largest A software bug apparently caused the largest power outage in North America, the Northeast power outage in North America, the Northeast blackout of August 2003, which threw millions of blackout of August 2003, which threw millions of people into darkness (ComputerWorld 7/25/05)people into darkness (ComputerWorld 7/25/05)

Flawed software cost the U.S. economy $60 Flawed software cost the U.S. economy $60 billion in 2002 (NIST 2002)billion in 2002 (NIST 2002)


Research MotivationResearch Motivation

SolutionsSolutions Process maturity is key to higher quality, lower costs, Process maturity is key to higher quality, lower costs,

shorter development timeshorter development time Harter, Krishnan, Slaughter, Harter, Krishnan, Slaughter, Management ScienceManagement Science 2000 2000

Questions remain:Questions remain: How to encourage higher quality?How to encourage higher quality?

Research questionResearch question Can contract selection be a vehicle to encourage Can contract selection be a vehicle to encourage

software quality?software quality? If so, what factors drive contract selection?If so, what factors drive contract selection?


Contract SelectionContract Selection

Contract theory & agency theoryContract theory & agency theory Choice of contract structure is crucial to ensure Choice of contract structure is crucial to ensure

that agent’s goals are aligned with principal that agent’s goals are aligned with principal (Crocker & Reynolds, 1993; Grossman & Hart, (Crocker & Reynolds, 1993; Grossman & Hart, 1983; Milgrom & Roberts, 1992)1983; Milgrom & Roberts, 1992)

Issues:Issues: Hidden information leads to adverse selectionHidden information leads to adverse selection Information asymmetry leads to moral hazardInformation asymmetry leads to moral hazard Type of contract can serve as an effective Type of contract can serve as an effective

governance mechanismgovernance mechanism


Information AsymmetryInformation Asymmetry

Sources of information asymmetrySources of information asymmetry uncertainty in product specifications and uncertainty in product specifications and

uncertainty about the vendor’s ability to uncertainty about the vendor’s ability to develop quality productsdevelop quality products

Artz and Norman 2002Artz and Norman 2002 Stump and Heide 1996Stump and Heide 1996 Kalnins and Mayer 2004Kalnins and Mayer 2004

High uncertainty increases costs of High uncertainty increases costs of writing specific contract termswriting specific contract terms


Contract ChoiceTime & Material,

Hybrid, Fixed Price

Research ModelResearch Model

SpecificationUncertainty

DesignComplexity

Prior ContractingExperience

SoftwareProcess Maturity

Uncertainty of Product Specifications:

Uncertainty of Vendor Quality: Verification &Validation Quality

H1, H2

H3, H4

H5a, H5b


Product UncertaintyProduct Uncertainty

IssuesIssues Client requirements can be ambiguous (Nidumolu Client requirements can be ambiguous (Nidumolu

1995)1995) Software products are innovations and Software products are innovations and

innovations embody uncertainties (Ang & Beath innovations embody uncertainties (Ang & Beath 1993)1993)

Software development is frequently exploratory Software development is frequently exploratory (MacCormack 2001)(MacCormack 2001)

Client understanding is evolutionary (Richmond Client understanding is evolutionary (Richmond 1992)1992)

HypothesisHypothesis H1: Time & Materials contracts more likely when H1: Time & Materials contracts more likely when

specification uncertainty is highspecification uncertainty is high


Product ComplexityProduct Complexity

IssuesIssues Complex designs are more difficult to develop Complex designs are more difficult to develop

(Brooks 1995)(Brooks 1995) Effort required for testing complex designs is Effort required for testing complex designs is

highly variable (Banker 2002)highly variable (Banker 2002) Higher software complexity increases technical Higher software complexity increases technical

risk (Barki 1993)risk (Barki 1993) Development cost estimation is more uncertainDevelopment cost estimation is more uncertain

HypothesisHypothesis H2: Time & Materials contracts more likely H2: Time & Materials contracts more likely

when design complexity is highwhen design complexity is high


Vendor UncertaintyVendor Uncertainty

IssuesIssues Inability to determine vendor quality can create problems of Inability to determine vendor quality can create problems of

adverse selection and moral hazard (Artz & Norman 2002)adverse selection and moral hazard (Artz & Norman 2002) Repeated interaction and long-term relationships mitigate Repeated interaction and long-term relationships mitigate

adverse selection and moral hazard (Baker 1994)adverse selection and moral hazard (Baker 1994) Repeated transactions provide incentives that decrease Repeated transactions provide incentives that decrease

likelihood of opportunism (MacNeil 1978; Granovetter 1985)likelihood of opportunism (MacNeil 1978; Granovetter 1985) Corts & Singh (2004)Corts & Singh (2004)

Repeated interactions reduce contracting costs, leading to fixed Repeated interactions reduce contracting costs, leading to fixed priceprice

Interaction reduces opportunism, leading to time & materialInteraction reduces opportunism, leading to time & material Variance of these costs affects contract choice (Kalnins & Variance of these costs affects contract choice (Kalnins &

Mayer 2004)Mayer 2004) HypothesisHypothesis

H3: Hybrid contracts more likely when contracting experience H3: Hybrid contracts more likely when contracting experience between vendor and client is lowbetween vendor and client is low


Vendor UncertaintyVendor Uncertainty

IssuesIssues Adverse selection can be addressed using Adverse selection can be addressed using

signals designed to reveal private information signals designed to reveal private information (Milgrom & Roberts 1992; Mishra 1998)(Milgrom & Roberts 1992; Mishra 1998)

Qualification process can identify vendors with Qualification process can identify vendors with necessary skills (Stump & Heide 1996)necessary skills (Stump & Heide 1996)

Process maturity can be used to signal quality Process maturity can be used to signal quality (Arora & Asundi 1999)(Arora & Asundi 1999)

HypothesisHypothesis H4: Fixed price contracts more likely when H4: Fixed price contracts more likely when

software process maturity is highsoftware process maturity is high


Effect of Contract ChoiceEffect of Contract Choiceon Quality Outcomeson Quality Outcomes

IssuesIssues Opportunity for ex post opportunism by both parties Opportunity for ex post opportunism by both parties

(Williamson 1979)(Williamson 1979) Vendor has financial incentive to freeze the specification Vendor has financial incentive to freeze the specification

in fixed price contractin fixed price contract Incentives are to develop the software right the first time, Incentives are to develop the software right the first time,

according to the specificationaccording to the specification Clients may change are articulate new requirements as Clients may change are articulate new requirements as

they discover what they truly needthey discover what they truly need Vendor profits from new requirements under Time & Vendor profits from new requirements under Time &

Materials, and may accommodate client’s requirementsMaterials, and may accommodate client’s requirements HypothesesHypotheses

H5a: Fixed price contracts have higher development and production H5a: Fixed price contracts have higher development and production verification qualityverification quality

H5b: Time & Materials contracts have higher acceptance validation H5b: Time & Materials contracts have higher acceptance validation qualityquality


Research Site &Research Site &Data CollectionData Collection

Data collected on software projects Data collected on software projects developed from 1987 to 2004developed from 1987 to 2004

78 contracts were negotiated78 contracts were negotiated 26 time and material26 time and material 38 fixed price38 fixed price 14 hybrid14 hybrid


Contract TypesContract Types

Time & MaterialTime & Material Vendor reimbursed through hourly rateVendor reimbursed through hourly rate Technical and financial risks on clientTechnical and financial risks on client

Fixed PriceFixed Price Vendor agrees to fixed contract valueVendor agrees to fixed contract value Technical and financial risks on vendorTechnical and financial risks on vendor Costly to negotiate – requires detailed specifications ex Costly to negotiate – requires detailed specifications ex

anteante HybridHybrid

Agreement on cost estimate, but client pays all costs; Agreement on cost estimate, but client pays all costs; profit based on initial estimate and performanceprofit based on initial estimate and performance

Financial risk primarily on clientFinancial risk primarily on client


MeasuresMeasures

Contract choiceContract choice Categorical variablesCategorical variables

1-T&M, 2-hybrid, 3-Fixed price1-T&M, 2-hybrid, 3-Fixed price QualityQuality

Verification (development & production) – technical issues of Verification (development & production) – technical issues of whether the software has been developed correctly and performs whether the software has been developed correctly and performs correctlycorrectly

Validation (acceptance) – whether the right software has been Validation (acceptance) – whether the right software has been developed that satisfies the usersdeveloped that satisfies the users

Antecedents of contract choiceAntecedents of contract choice Specification uncertaintySpecification uncertainty Design complexityDesign complexity Prior contracting experiencePrior contracting experience Software process maturitySoftware process maturity

ControlsControls Product sizeProduct size


Regression Models: ChoiceRegression Models: Choice

Stage 1: Contract ChoiceStage 1: Contract Choice multinomial regression using Newton-Raphson maximum multinomial regression using Newton-Raphson maximum

likelihood estimationlikelihood estimation Prob(yProb(yii=j) = e =j) = e jXijXi / / ΣΣe e kXikXi

CorrectionsCorrections Non-independence of disturbances across different Non-independence of disturbances across different

contract segmentscontract segments Huber (1967)/White (1982) sandwich estimatorHuber (1967)/White (1982) sandwich estimator

ResultsResults Antecedents significant in predicting choice (p<.001)Antecedents significant in predicting choice (p<.001) Explain significant variance (pseudo RExplain significant variance (pseudo R22 = 0.751) = 0.751) Correlation between predicted and actual contract is Correlation between predicted and actual contract is

88.5%88.5%


Uncertainty & ComplexityUncertainty & Complexity

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Fixed Price Hybrid Time & Materials

contract choice

like

liho

od

of

ch

oo

sin

g

lowhigh

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1


contract choice

like

liho

od

of

ch

oo

sin

g

lowhigh

Likelihood of Contract ChoiceGiven Levels of Specification Uncertainty

Likelihood of Contract ChoiceGiven Levels of Design Complexity


Hypotheses SummaryHypotheses Summary

H1: Time & Materials preferred over fixed H1: Time & Materials preferred over fixed price for higher levels of specification price for higher levels of specification uncertaintyuncertainty 66% likelihood for high specification uncertainty66% likelihood for high specification uncertainty 10% likelihood for low specification uncertainty10% likelihood for low specification uncertainty

H2: Time & Materials preferred over fixed H2: Time & Materials preferred over fixed price and hybrid when there is higher design price and hybrid when there is higher design complexitycomplexity 71% likelihood for high design complexity71% likelihood for high design complexity 8% likelihood for low design complexity8% likelihood for low design complexity


Experience & ProcessExperience & Process

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1


contract choice

like

liho

od

of

ch

oo

sin

g

low

high

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1


contract choice

like

liho

od

of

ch

oo

sin

g

low

high

Likelihood of Contract ChoiceGiven Levels of Prior Contracting Experience

Likelihood of Contract ChoiceGiven Levels of Software Process Maturity


Hypotheses SummaryHypotheses Summary

H3: hybrid contracts preferred over H3: hybrid contracts preferred over fixed price when prior contracting fixed price when prior contracting experience is lowerexperience is lower 84% likelihood of hybrid for low experience84% likelihood of hybrid for low experience 88% likelihood of fixed price for high 88% likelihood of fixed price for high

experienceexperience H4: fixed price preferred for higher H4: fixed price preferred for higher

levels of process maturitylevels of process maturity


Regression Models: QualityRegression Models: Quality

Stage 2: Quality Outcomes of Contract ChoiceStage 2: Quality Outcomes of Contract Choice Multivariate general linear modeling (GLM)Multivariate general linear modeling (GLM) Two-step multinomial selection bias correction method of Lee (1983)Two-step multinomial selection bias correction method of Lee (1983)

ModelsModels Development Verification Quality = f(Contract-Choice, Specification-Development Verification Quality = f(Contract-Choice, Specification-

Uncertainty, Design-Complexity, Prior-Contracting-Experience, Uncertainty, Design-Complexity, Prior-Contracting-Experience, Software-Process-Maturity, Product-Size)Software-Process-Maturity, Product-Size)

Production Verification Quality = f(Contract-Choice, Specification-Production Verification Quality = f(Contract-Choice, Specification-Uncertainty, Design-Complexity, Prior-Contracting-Experience, Uncertainty, Design-Complexity, Prior-Contracting-Experience, Software-Process-Maturity, Product-Size)Software-Process-Maturity, Product-Size)

Acceptance Validation Quality = f(Contract-Choice, Specification-Acceptance Validation Quality = f(Contract-Choice, Specification-Uncertainty, Design-Complexity, Prior-Contracting-Experience, Uncertainty, Design-Complexity, Prior-Contracting-Experience, Software-Process-Maturity, Product-Size)Software-Process-Maturity, Product-Size)

ResultsResults Hotelling’s THotelling’s T22 test of contract choice significant (p<.001) test of contract choice significant (p<.001) Post hoc calculation of power is 0.97Post hoc calculation of power is 0.97


Quality OutcomesQuality Outcomes

0

0.25

0.5

0.75

1

1.25

1.5

1.75

2

2.25

Time & Materials Hybrid Fixed Price

Contract Type

Err

ors

pe

r K

LO

C

Devel-Verif

Prod-Verif

Accept-Val


DiscussionDiscussion

Information asymmetry arising from Information asymmetry arising from product uncertainties (specification product uncertainties (specification uncertainty and design complexity) shifts uncertainty and design complexity) shifts contract choice to Time & Materialcontract choice to Time & Material

Uncertainty of vendor quality is a strong Uncertainty of vendor quality is a strong motivator of contract choicemotivator of contract choice Vendor quality (30.2%) explains eight times Vendor quality (30.2%) explains eight times

the variance of product uncertainty (3.7%)the variance of product uncertainty (3.7%)


DiscussionDiscussion

Prior contracting experience is a critical Prior contracting experience is a critical mitigator of information asymmetrymitigator of information asymmetry Hybrid contracts more likely when experience Hybrid contracts more likely when experience

between client and vendor is lowbetween client and vendor is low Reducing contracting and shirking costsReducing contracting and shirking costs

Vendor quality certification explains Vendor quality certification explains highest variance in contract choice (20%)highest variance in contract choice (20%) Quality certification engenders greater Quality certification engenders greater

confidence in the vendor’s abilities to estimate confidence in the vendor’s abilities to estimate and deliver software products to specificationsand deliver software products to specifications


Thank You!Thank You!

Questions?Questions?