Post on 03-Jan-2016
What is Computer Science?
Three paradigms (CACM 1/89) Theory (math): definitions, theorems,
proofs, interpretations Abstraction (science): hypothesize,
model, experiment, analyze data, results
Design (engineering): state requirements, design, implement, test
How to choose a method, tool, technique?
Common sense Consultants Theoretical proof Benchmark Case study Survey Experimentation in the Small Experimentation in the Large?
Experiment Design – Define/Describe
Subjects Independent Variables and Levels Dependent Variables Confounding Variables (noise) and
how controlled Group Selection Treatment vs Measurement
Analyzing Results
Descriptive statistics averages ranges standard deviations correlations
Inferential statistics t-test ANOVA
Correlations
from -1 to +1 “explains” variance should include statistical
significance NOT causality see examples …
Useful Experimentation in the Small
The Magic Number Seven, Plus or Minus Two; Miller 1956
Exploratory Experimental Studies Comparing Online and Offline Programming Performance; Sackman 1968
A Controlled Experiment in Program Testing and Code Walkthroughs / Inspections; Myers 1978
Reliability of Function Points Measurement – A Field Experiment; Kemerer 1993
To read a paper with empirical contributions ask the following:
What was the author’s primary interest?
How rigorous is the work? Internal validity External validity
What was the STRENGTH of the statistical results reported?
What serendipitous results were reported?
Lederer and Prasad Article
Discuss methodology questionnaire sent to 400 computing
managers and professionals 115 responded responses analyzed for correlations
between approaches used and success Careful of cause and effect
see footnote 2, page 53
Some interesting guidelines:
assign initial estimating task to the developers
delay finalizing initial estimate until after thorough study
monitor progress (and use external auditors)
use the estimate to evaluate personnel don’t rely on cost estimating software!