P.B. Medawar. Advice to a young scientist. Basic Books. 1979.

Medawar’s Experimentation Models & Computer Science

Peter Medawar

Nobel Prize for Medicine 1960• 1915- 1987, born in Rio de Janeiro, son of a

Lebanese business man who was a naturalized British subject.

• Bachelor’s degree from Oxford in 1932. • Worked on tissue grafts and transplants

Solutions

Solving a problem simply means representing it so as to make the solution transparent.

Herbert Simon

Research is the art of the soluble. Peter Medawar

How Do we look at things?

El Greco test

Medawar’s Experiments and Discovery

Four kinds– Baconian (observe)– Aristotelian (effect)– Galilean (hypothesis)– Kantian (thought)

What does it mean to do experiments in CS?

1. Baconian Experimentation

Find truths by careful examination of things as they are Compilation of facts Contrived performance rather than natural occurrance No control group, no theory Examples

– Magnetising nails– Static electricity in silk

Trying things out or mucking about

Baconian experimentation in CS

Early IR KWIC/ KWOC indices Zipf distribution Counting word occurrences and distributions

2. Aristotelian Experimentation(John Glanville, Royal Soc. 1636-84)

Demonstrate some preconceived idea– Ring a bell before giving the dog his dinner

Effect without theory Examples of X CS??

Aristolelian Experiments in CS

Eliza

Bob

Pop up ads

IR data visualizations

Post hoc, ergo prompter hoc

Psych: Why do you flail your arms around like that?

Patient: Keeps the wild elephants at bay.

Psych: But there aren’t any wild elephants here.

Patient: That’s right. Effective, isn’t it!

4. Kantian Experiment

Thought experiments Examples

– non-Euclidian spaces– Parallel lines that meet

Let’s look at that differently

Kant meets CS

N-dimensional vector spaces

Shneiderman data walls

Hypercube Web graph Data visualization

3. Galilean Experimentation

Expose hypothesis to a test Dropping of canon balls off Pisa tower to test his

hypothesis of gravitational acceleration Leads to the null hypothesis

– Experiments can not really prove anything!– Best you can do is refute the null hypothesis– I.e., that you have done better than wild good luck– Looking at results of differences of observations

Be prepared to take “no difference” as an answer

Hypotheses

I cannot give any scientist of any age better advice than this: the intensity of the conviction that a hypothesis is true has no bearing on whether it is true or not.

Medawar

Galilean Experimentation in CS

Algorithm efficiency Algorithm effectiveness User preference Etc

Not withstanding

Simpson’s Paradox– 2 data sets -> separately support a conclusion– BUT the union supports the opposite conclusion

Will Roger’s Phenomena– In a patient study, it is possible to transfer a patient from one

group to another and improve the statistics of both groups

Mark Twain’s Observation– Lies, damned lies and statistics!

How to be prepared to do research I

Mastering the literature– Too much

Confine the imagination Psychological substitute for research

– Too little Make an idiot of yourself

– Mix some eclectic breadth with selected depth Eg. ACM Communications and IJHCI

How to prepare II

Get on with it– Get results

Repeat others work Try variations Try other data

– Join the discussion When I tried that … I got exactly the same results when I… I agree, for this purpose x is better then y

How to prepare III

Follow the art of the soluble Start with a “soft underbelly problem”

– Quantification of vague phenomenon– Isolating factors– Selecting feature sets

To quantify is not to be a scientist, but it does help. (Medawar)

Also part of the Scientific Process

Devising hypotheses that can be tested in a practical manner

Imaginative guesswork Exercise of common sense All experimentation is a form of criticism Having the right slot in your mind to put a new

observation or idea Good luck counts Accept flux. Science as a Maoist microcosm of

continuing revolution.