1

The design of animal

experiments

Michael FW Festing c/o Understanding Animal Research, 25 Shaftsbury

Av. London, UK.

michaelfesting@aol.com

2

Replacement

e.g. in-vitro methods, less sentient animals Refinement

e.g. anaesthesia and analgesia, environmental

enrichment Reduction

Research strategy

Controlling variability

Experimental design and statistics

Principles of Humane Experimental

Technique

(Russell and Burch 1959)

3

A well designed experiment

Absence of bias Experimental unit, randomisation, blinding

High power Low noise (uniform material, blocking, covariance)

High signal (sensitive subjects, high dose)

Large sample size

Wide range of applicability Replicate over other factors (e.g. sex, strain): factorial

designs

Simplicity

Amenable to a statistical analysis

4

The animal as the experimental unit

Animals individually treated. May be individually housed or grouped

N=8 n=4

5

A cage as the Experimental

Unit.

Treatment in water or diet.

N=4 n=2

Treated Treated Control Control

6

An animal for a period of time: repeated

measures or crossover design

Animal 1 2 3

Treatment 1 Treatment 2

N 4 4 4

N=12 n= 6

7

Teratology: mother treated,

young measured

Mother is the experimental unit.

N=2 n=1

8

Failure to identify the experimental unit

correctly in a 2(strains) x 3(treatments) x

6(times) factorial design

ELD group ELD group

Single cage of 8 mice killed at each time point (288 mice in total)

9

Experimental units must be

randomised to treatments

Physical: numbers on cards. Shuffle and take

one

Tables of random numbers in most text

books

Use computer. e.g. EXCEL or a statistical

package such as MINITAB

10

Randomisation

Original Randomised 1 2 1 3 1 3 1 1 2 2 2 1 2 2 2 1 3 3 3 2 3 3 3 1

NB Randomisation should include housing and order in which observations are made

11

Failure to randomise and/or blind

leads to more “positive” results

Blind/not blind odds ratio 3.4 (95% CI 1.7-6.9) Random/not random odds ratio 3.2 (95% CI 1.3-7.7) Blind Random/ odds ratio 5.2 (95% CI 2.0-13.5) not blind random 290 animal studies scored for blinding, randomisation and positive/negative outcome, as defined by authors Babasta et al 2003 Acad. emerg. med. 10:684-687

12

Some factors (e.g. strain, sex) can not be

randomised so special care is needed to ensure

comparability

Outbred TO (8-12 weeks

commercial)

Inbred CBA (12-16

weeks Home bred)

Six cages of 7-9 mice of each strain: error bars are SEMs

"CBA mice showed greater

variability in body weights than

TO mice..."

13

A well designed experiment

Absence of bias Experimental unit, randomisation, blinding

High power Low noise (uniform material, blocking, covariance)

High signal (sensitive subjects, high dose)

Large sample size

Wide range of applicability Replicate over other factors (e.g. sex, strain): factorial

designs

Simplicity

Amenable to a statistical analysis

14

High power: (good chance of detecting the effect

of a treatment, if there is one)

High Signal/Noise ratio

= High Standardized effect size = High d=|m1-m2|/s

= High (Difference between means)/SD

Student’s t =( X1-X2)/Sqrt (2S2/n)

15

Power Analysis for sample size and

effects of variation

A mathematical relationship between six variables

Needs subjective estimate of effect size to be detected (signal)

Has to be done separately for each character

Not easy to apply to complex designs

Essential for expensive, simple, large experiments (clinical trials)

Useful for exploring effect of variability

A second method “The Resource Equation” is described later

16

Power analysis: the variables

Sample size

Signal a) Effect size of scientific interest

or b) actual response

Chance of a false positive result. Significance level

(0.05)

Sidedness of statistical test (usually 2-sided)

Power of the Experiment (80-90%?)

Noise Variability of the

experimental material

17

Group size and Signal/noise

ratio

0

20

40

60

80

100

120

140

0 0.5 1 1.5 2 2.5 3

Effect size (Std. Devs.)

Gro

up

siz

e

90%

80%

Assuming 2-sample, 2 sided t-test and 5% significance level

Signal/noise ratio

Power

Neutral

Bad

Good

18

Comparison of two anaesthetics for dogs

under clinical conditions (Vet. Anaesthes. Analges.)

Unsexed healthy clinic dogs, • Weight 3.8 to 42.6 kg. • Systolic BP 141 (SD 36) mm Hg

Assume: • a 20 mmHg difference between anaesthetics is of clinical importance, • a significance level of a=0.05 • a power=90% • a 2-sided t-test

Signal/Noise ratio 20/36 = 0.56 Required sample size 68/group

19

Power and sample size

calculations using nQuery Advisor

20

A second paper described:

• Male Beagles weight 17-23 kg • mean BP 108 (SD 9) mm Hg. • Want to detect 20mm difference between groups (as before)

With the same assumptions as previous slide:

Signal/noise ratio = 20/9 = 2.22 Required sample size 6/group

21

Summary for two sources of dogs: aim is to

be able to detect a 20mmHg change in blood

pressure

Type of dog SDev Signal/noise Sample %Power (n=8) size/gp(1) (2)

Random dogs 36 0.56 68 18 Male beagles 9 2.22 6 98 (1) Sample size: 90% power (2) Power, Sample size 8/group

Assumes a=5%, 2-sided t-test and effect size 20mmHg

The scientific dilemma: With small sample sizes we can not detect an important effect in genetically heterogeneous animals. We can detect the effect in genetically homogeneous animals, but are they representative?

22

Variation in kidney weight in

58 groups of rats

0

10

20

30

40

50

60

70

80

90

1 5 9 13 17 21 25 29 33 37 41 45 49 53 57

Sample number

Va

ria

bil

ity Mycoplasma

Outbred

F1

F2

Gartner,K. (1990), Laboratory Animals, 24:71-77.

23

Required sample sizes

Factor Type Std.Dev Signal/

noise*

Sample

size

Power**

Genetics F1 hybrid 13.5 0.74 30 80

F2 hybrid 18.4 0.54 55 53

Outbred 20.1 0.49 67 46

Disease Mycoplasma

free

18.6 0.54 55 53

With

Mycoplasma

43.3 0.23 298 14

*signal is 10 units, two sided t-test, a=0.05, power = 80% ** Assuming fixed sample size of 30/group

24

The randomised block design: another

method of controlling noise

B C A

A C B

B A C

A C B

B C A B1 B2 B3 B4 B5

Treaments A, B & C

• Randomisation is within-block • Can be multiple differences

between blocks • Heterogeneous age/weight • Different shelves/rooms • Natural structure (litters) • Split experiment in time

25

A randomised block

experiment

0

50

100

150

200

250

300

350

400

450

500

1 2 3

Week

Ap

op

tosis

sco

re

Control

CGP

STAU

365 398 421 423 432 459 308 320 329

Treatment effect p=0.023 (2-way ANOVA)

26

Analysis of apoptosis data

Analysis of Variance for Score

Source DF SS MS F P

Block 2 21764.2 10882.1 114.82 0.000

Treatmen 2 2129.6 1064.8 11.23 0.023

Error 4 379.1 94.8

Total 8 24272.9

27

-10.0 -7.5 -5.0 -2.5 0.0 2.5 5.0 7.5

0

1

2

3

Residual

Fre

quency

Histogram of Residuals

0 1 2 3 4 5 6 7 8 9

-20

-10

0

10

20

Observation Number

Resid

ual

I Chart of Residuals

Mean=3.16E-14

UCL=20.17

LCL=-20.17

300 350 400 450

-10

0

10

Fit

Resid

ual

Residuals vs. Fits

-1.5 -1.0 -0.5 0.0 0.5 1.0 1.5

-10

0

10

Normal Plot of Residuals

Normal Score

Re

sid

ual

Residual Model Diagnostics

28

Another method of determining sample size:

The Resource Equation

Depends on the law of diminishing returns

Simple. No subjective parameters

Useful for complex designs and/or multiple outcomes

(characters)

Does not require estimate of Standard Deviation

Crude compared with Power Analysis

E= (Total number of animals)-(number of groups) 10<E<20 (but give some tolerance)

29

0 5 10 15 20 25 30 35

2.0

4.5

7.0

9.5

12.0

Degrees of freedom

Stu

den

t's t

, 5

% c

ritica

l valu

e

E= (total numbers)-(number of groups)

10<E<20

The Resource Equation & Sample Size

But if experimental subjects are cheap (e.g. multi-well plates, E can be much higher

30

A well designed experiment

Absence of bias Experimental unit, randomisation, blinding

High power Low noise (uniform material, blocking, covariance)

High signal (sensitive subjects, high dose)

Large sample size

Wide range of applicability Replicate over other factors to (e.g. sex, strain) to increase

generality: factorial designs

Simplicity

Amenable to a statistical analysis

31

Factorial designs

Single factor design

Treated Control

E=16-2 = 14

One variable at a time (OVAT)

Treated Control Treated Control

E=16-2 = 14 E=16-2 = 14

Factorial design

Treated Control

E=16-4 = 12

32

Factorial designs

(By using a factorial design)”.... an experimental investigation, at the same time as it is made more comprehensive, may also be made more efficient if by more efficient we mean that more knowledge and a higher degree of precision are obtainable by the same number of observations.” R.A. Fisher, 1960

33

A 4x2 factorial design

Analysed with Student’s t-test: This is not appropriate because: 1. Each test is based on too few animals (n=3-4), so lacks power 2. It does not indicate whether there are strain differences in protein thiol status 3. It does not indicate whether dose/response differs between strains 4. A two-way design should be analysed using a 2-way ANOVA

34

Incorrect statistical analysis leading to

excessive numbers of animals

8 mice per group 8 groups = 64 mice. E= 64-8 =56

Alternative 3 mice per group: 8 groups E=24-8 = 16 Saving:40 mice Formal test of interaction

One experiment or 4 separate experiments?

35

2 (strains) x 4 (Animal units)

factorial

36

Effect of chloramphenicol

(2000mg/kg) on RBC count

Strain Control Treated C3H 7.85 7.81 8.77 7.21 8.48 6.96 8.22 7.10 CD-1 9.01 9.18 7.76 8.31 8.42 8.47 8.83 8.67

Tests: Use a two-way ANOVA with interaction

1. Do the treatment means averaged across strains differ?

2. Do the strains differ, averaged across treatments

3. Do the two strains respond to the same extent?

Should not be analysed using two t-tests 1. Each test lacks power due to small sample size 2. Will not give a test of whether strains differ in response

37

A 2x2 factorial design with

interaction

Source DF SS MS F P

strain 1 2.4414 2.4414 13.13 0.003

Treatment 1 0.8236 0.8236 4.43 0.057

strain*treat. 1 1.4702 1.4702 7.91 0.016

Error 12 2.2308 0.1859

Total 15 6.9659

6.5

7

7.5

8

8.5

9

Control Treated Control Treated

Strain and treatment

Red

blo

od

cell

co

un

t

C3H CD-1

Pooled variance

38

Use of several inbred strains to reduce

noise, increase signal and explore

generality

500 1000 1500 2000 2500

CD-1 8 8 8 8 8 8

CBA 2 2 2 2 2

C3H 2 2 2 2 2

BALB/c 2 2 2 2 2

C57BL 2 2 2 2 2

2

2

2

2

Inbred

0

Outbred

Dose of chloramphenicol (mg/kg)

Festing et al (2001) Fd. Chem.Tox. 39:375

Effect of chloramphenicol on mouse haematology

39

WBC Strain Control Treated CBA 1.90 0.40 CBA 2.60 0.20 C3H 2.10 0.40 C3H 2.20 0.40 BALB/c 1.60 1.30 BALB/c 0.50 1.40 C57BL 2.30 0.80 C57BL 2.20 1.10 CD-1 3.00 1.90 CD-1 1.70 1.90 CD-1 1.50 3.50 CD-1 2.00 1.20 CD-1 3.80 2.30 CD-1 0.90 1.00 CD-1 2.60 1.30 CD-1 2.30 1.60

Example of a factorial compared with a single factor design

Four inbred strains One outbred stock

40

Signal Noise

Strain N 0 2500 (Difference) (SD) Signal/noise p

CBA 4 2.25 0.30 1.95 0.34 5.73

C3H 4 2.15 0.40 1.85 0.34 5.44

BALB/c 4 1.05 1.35 (-0.30) 0.34 (-0.88)

C57BL 4 2.25 0.95 1.30 0.34 3.82

Mean 16 1.93 1.20 0.73 0.34 2.15 <0.001

Dose * strain <0.001

WBC counts following chloramphenicol at

2500mg/kg

Signal Noise

Strain N 0 2500 (Difference) (SD) Signal/noise p

CD-1 16 2.23 1.83 0.40 0.86 0.47 0.38

White blood cell counts

41

Genetics is important: Twenty two Nobel Prizes since 1960

for work depending on inbred strains

Cancer

mmTV

Transmissable

encephalopathacies/prions

Pruisner

Retroviruses, Oncogenes & growth factors

Cohen, Levi-montalcini, Varmus, Bishop, Baltimore, Temin

Humoral immunity/antibodies

T-cell receptor

Tonegawa, Jerne

Cell mediated immunity

Immunological tolerance

H2 restriction, immune responses

Medawar, Burnet, Doherty, Zinkanagel

Benacerraf (G.pigs)

Genetics

Snell C.C. Little, DBA, 1909

Inbred Strains and derivatives

Jackson Laboratory

monoclonal antibodies

BALB/c mice

Kohler and Millstein

Smell Axel & Buck

ES cells & “knockouts” Evans, Capecchi, Smithies

42

18th Annual Short Course on Experimental

Models of Human Cancer

August 21-30, 2009

Bar Harbor, ME

courses.jax.org

43

Conclusions

Five requirements for a good design Unbiased (randomisation, blinding)

Powerful (signal/noise ratio: control variability)

Wide range of applicability (factorial designs, common but frequently analysed incorrectly)

Simple

Amenable to statistical analysis

Mistakes in design and analysis are common

Better training in experimental design would improve the quality of research, save money, time and animals

44

45