1

Application of DOE Methodology to SNP Assay Development

Geetha RajaveluGretchen Kiser

2002 Quality & Productivity Research ConferenceTempe, AZ

June 5-7, 2002

2

Presentation Outline

• Background Info

• Overview of SNP Assay

• Identifying Variables for Screening Experiment

• Screening Experiment Results

• Optimization Work

• Overall Performance Improvements

• Concluding Remarks

3

Background Info

• In Q3 2000, CodeLink slides

were affected by unexplained

background.

• All aspects of slide manufacturing

and assay processes were

examined for root causes.

• An effort was launched to

thoroughly characterize all

processes (with no apriori

assumptions).

4

SNP Assay Modules

Target Amplification

Target Pooling & Fragmentation

SBE Thermal Cycling SBE Slide Washing, SA-Alexa Labeling & Post-labeling Slide Washing

Drying

Scanning

5

S = major alleles = minor allele

PCR clean-up &

Fragmentation

Add reaction mixture to Bioarray

Allele-Specific Extension

Washing &Secondary Labeling

SNP

PCR

DNA genomicprimersprimers

SNP

SNP

Whole target DNA

SNP

Fragmented target

SS

SNP

S s S

Target hybridizes to probes on Bioarray

S s S

Extension reaction on Bioarray

Mediated by a DNA polymerase and labeled nucleotides

Allele on Bioarrayready to scan

SNP Assay Overview

6

Screening Experiment Info

Objective:

– To evaluate the contribution of any one variable, and the interactive contribution of any two variables of the SBE Thermal Cycling module, to the assay performance, as measured by the Signal/Noise ratio, the Call Rate and Accuracy, the Mean Pad IOD, and the Mean Blank Pad IOD.

Preliminary Info:– Factor ranges based on single-variable experiments

– Nuisance Factors: Dispense Slide Batches, Thermal Cyclers

Design Details:– 212-6 fractional factorial design

– Involved 65 runs, 2 slides/run

– Took around 9 days to complete experiment

7

Identifying Variables for the Allele-Specific Extension Reaction

Thermal Cycler Number of Cycles (A)

Extension Time (B)

Denaturation Time (C)

Extension Temperature (D)

Denaturation Temperature (E)

Reaction Mixture Target Concentration (F)

Enzyme Concentration (G)

tNTP Concentration (H)

Salt1 Concentration (J)

Salt2 Concentration (K)

Buffer Concentration (L)

Buffer pH (M)

8

Selecting Significant Factor Effects

9

Screening Experiment Analysis Results

Response Significant Factor Effects Adj R-Squared Curvature?D, G, K, C, E, DHB, H, AK, DG, HKF, H, G, DEDJ, DH, AK, J, DAF, K, EJA, D, G, B, E, K, AG, DGC, CGA, B, D, G, H, AD, DGE, KJ, DJA, H, E, G, BJ, JKAF, AK, DFA, HG, AF, BJ, DF, AE, AKJK, CF, AL, JM, CE

Call Rate 78% Yes

Call Accuracy 23% No

S/N 80% Yes

Mean IOD 85% Yes

Mean Blank Pad IOD 75% No

Stdev Blank Pad IOD 57% Yes

10

Interaction Effects

11

Interaction Effects

12

Interaction Effects

13

Parameters shaded in green represent subset chosen for

inclusion in the optimization experiment.

Screening out Variables for Optimization Phase

Mean IOD Mean Blnk IOD Signal/Noise Call Rate Call Accuracytotal

A # of Cycles 3 3 3 0.5 9.5B Extn Time 3 1.5 2 1 7.5C Dent Time 1 2 3

D Extn Temp 3 0.5 3 3 2 11.5E Dent Temp 2 2 2 2 8F Target Conc 0.5 3 3.5G Enzyme Conc 3 2 3 3 3 14H tNTP Conc 3 3 1 3 10J Salt1 Conc 1 1.5 2 4.5K Salt2 Conc 2 1.5 2 3 1 9.5L Buffer Conc 0M Buffer pH 0

Assay Parameter(Weight Values: hi=3, med=2, lo=1; (2 factor=50%))

Influence of Different Assay Parameters on Various Outputs

14

Response Surface Experiment InfoObjective:

– To determine factor settings which optimize the assay performance, as measured by the Signal/Noise ratio, the Call Rate and Accuracy, the Mean Pad IOD, and the Mean Blank Pad IOD.

Variables:

-- # of Cycles (A) -- Enzyme Conc (E)

-- Extension Time (B) -- tNTP Conc (F)

-- Extension Temp (C) -- Salt2 Conc (G)

-- Denaturation Temp (D)

Design Details:– Screening design augmented with axial runs using face-centered cube & D-Optimality

criterion (24 runs total)

15

Response Surface Modeling Results

Response Model Terms Adj R-Squared

C, E, G, C2

CF, D, F, AG, BFG, CE, AB, BGE, FC, AG, GBC, CE, EFC, E

A, C2, B

G, DF, D, AE, BE, AC, A2, DE, F

C, E, F, A, B, G

DE, D, C2, G

2,

AB, A2, AE, AG, CG

F, A

E, CG, G2, AG, F

2, D, B, CF

AC, C2, BF, EF, BG, D

2

F, A

E, CG, AG, B, G2

AD, BG, EF, AC, FG, B2, D

2, CD, F

2

Call Rate 73%

Call Accuracy 17%

S/N 81%

Mean IOD 84%

Mean Blank Pad IOD 63%

Stdev Blank Pad IOD 46%

16

Numerical Optimization Criteria

Lower Upper Lower Upper

Name Goal Limit Limit Weight Weight Importance# of Cycles is in range 2 14 1 1 5Extn Time is in range 5 15 1 1 5Extn Temp is in range 50 70 1 1 5Dentrn Temp is in range 80 90 1 1 5Enzyme Conc is in range 3.1 11.1 1 1 5tNTP Conc is in range 0.25 2.25 1 1 5Salt2 Conc is in range 1 3 1 1 5Call Rate maximize 89.9999989 99.999999 0.1 1 5Call Accuracy maximize 0 100 0.1 1 3S/N maximize 6 17.6702 0.1 1 4Ave Pad IOD maximize 200000 622987 0.1 1 4Ave Blnk Pad IOD minimize 535 8000 1 0.1 5

Constraints

17

Numerical Optimization Results

No.# of Cycles

Extn Time

Extn Temp

Dentrn Temp

Enzyme Conc

tNTP Conc

Salt2 Conc

Call Rate

Call Accuracy S/N

Ave Pad IOD

Ave Blnk Pad IOD Desirability

1 6.85 14.45 56.42 82.61 11.10 2.25 3.00 102.226 93.4249 15.7688 672581 1007.04 0.980484722 6.83 13.19 56.94 83.00 11.10 2.25 3.00 102.023 93.7322 14.741 628146 959.96 0.980454923 8.39 11.54 56.65 81.97 11.10 2.24 3.00 101.829 93.8326 14.8812 638299 976.299 0.980217384 8.99 11.77 55.76 81.43 11.10 2.25 3.00 101.708 93.5732 15.398 661269 1008.08 0.980013995 8.27 13.77 54.85 83.39 11.09 2.25 2.97 100.962 93.0696 15.7982 725328 1025.48 0.979982366 7.17 14.21 57.69 81.12 11.04 2.25 3.00 102.776 93.833 15.8991 644298 1037.99 0.979894157 8.45 12.84 56.34 84.53 10.64 2.25 3.00 100.778 93.5463 14.4109 652428 982.111 0.979344998 9.06 14.81 55.68 81.55 11.10 2.20 2.95 101.812 93.1557 17.5359 769929 1142.76 0.978984429 9.48 10.72 55.46 80.89 11.10 2.23 3.00 101.576 93.644 14.9877 637577 1030.23 0.97890117

10 6.07 14.94 59.06 85.43 11.10 2.09 2.93 101.307 94.1715 14.3735 623063 1074.38 0.9769832411 7.21 14.90 57.98 83.76 11.10 2.25 2.39 100.731 94.2739 17.6699 955383 1247.23 0.9768885612 8.48 9.36 56.46 83.34 11.10 2.25 2.60 99.9952 94.3395 14.5201 761095 1098.61 0.97661037

18

Optimal Region

19

Optimal Region

20

Optimal Region

21

SNP Assay Modules – Characterized & Optimized

Target Amplification

Target Pooling & Fragmentation(23 factorial; face-centered design in 3 factors)

SBE Thermal Cycling(2 12-6 fractional factorial; face-centered design in 7 factors)

SBE Slide Washing, SA-Alexa Labeling & Post-labeling Slide Washing

(27-2 fractional factorial; face-centered design in 5 factors)

Drying

Scanning

22

Overall Performance Improvements

Group 1DOE specified master mixDOE specified thermocycle programLow salt TNT mixLow SA-Alexa 532 dilutionOptimized Staining/Washing conditions

Group 2DOE specified master mixDOE specified thermocycle programHigh salt TNT mixHigh SA-Alexa 532 dilutionOptimized Staining/Washing conditions

Current Protocol Group 3Current master mixCurrent thermocycle programCurrent TNT mixCurrent SA-Alexa 532 dilutionCurrent Staining/Washing conditions

23

CR By Condition

96.0

96.5

97.0

97.5

98.0

98.5

99.0

99.5

100.0

100.5

1 2 3

Condition

With Control

Dunnett's

0.05

All Pairs

Tukey-Kramer

0.05

Higher Call Rate with DOE-derived Conditions

24

Higher Call Accuracy with DOE-derived Conditions

ACC By Condition

97.0

97.5

98.0

98.5

99.0

99.5

100.0

100.5

1 2 3

Condition

With Control

Dunnett's

0.05

All Pairs

Tukey-Kramer

0.05

25

IOD By Condition

100000

200000

300000

400000

500000

600000

700000

1 2 3

Condi tion

With Control

Dunnett' s

0.05

All Pairs

Tukey-Kramer

0.05

3X Improvement in Probe Intensity

26

S/N By Condition

5

10

15

20

25

30

1 2 3

Condi tion

With Control

Dunnett' s

0.05

All Pairs

Tukey-Kramer

0.05

3X Improvement in Signal/Noise

27

Lower Background with DOE-derived Conditions

Blk. IOD By Condition

800

900

1000

1100

1200

1300

1400

1500

1600

1 2 3

Condition

With Control

Dunnett's

0.05

All Pairs

Tukey-Kramer

0.05

28

Group 1 Group 2 Current Protocol Group 3

29

Concluding Remarks

• All experiments were planned and executed using Project

Management approach (risks/assumptions, scheduling, etc.)

• Sequential approach to experimentation is critical to

studying complex processes.

• Able to beat all odds!!

The probability of successfully completing an experiment is

inversely proportional to the number of runs, i.e.,1/65.

30

Acknowledgements

• SNP Assay Development Team

• Project Management Team