What is Computer Science?

Three paradigms (CACM 1/89) Theory (math): definitions, theorems,

proofs, interpretations Abstraction (science): hypothesize,

model, experiment, analyze data, results

Design (engineering): state requirements, design, implement, test

How to choose a method, tool, technique?

Common sense Consultants Theoretical proof Benchmark Case study Survey Experimentation in the Small Experimentation in the Large?

Experiment Design – Define/Describe

Subjects Independent Variables and Levels Dependent Variables Confounding Variables (noise) and

how controlled Group Selection Treatment vs Measurement

Analyzing Results

Descriptive statistics averages ranges standard deviations correlations

Inferential statistics t-test ANOVA

Correlations

from -1 to +1 “explains” variance should include statistical

significance NOT causality see examples …

Useful Experimentation in the Small

The Magic Number Seven, Plus or Minus Two; Miller 1956

Exploratory Experimental Studies Comparing Online and Offline Programming Performance; Sackman 1968

A Controlled Experiment in Program Testing and Code Walkthroughs / Inspections; Myers 1978

Reliability of Function Points Measurement – A Field Experiment; Kemerer 1993

To read a paper with empirical contributions ask the following:

What was the author’s primary interest?

How rigorous is the work? Internal validity External validity

What was the STRENGTH of the statistical results reported?

What serendipitous results were reported?

Lederer and Prasad Article

Discuss methodology questionnaire sent to 400 computing

managers and professionals 115 responded responses analyzed for correlations

between approaches used and success Careful of cause and effect

see footnote 2, page 53

Some interesting guidelines:

assign initial estimating task to the developers

delay finalizing initial estimate until after thorough study

monitor progress (and use external auditors)

use the estimate to evaluate personnel don’t rely on cost estimating software!