Genome in a Bottle Working GroupReference Material (RM) Selection and Design

NIST WorkshopAugust 15, 2013

Main Meetings – NIST Reference Materials

• April 13, 2012 (NIST)– Genome in a Bottle consortium initiation

• August 16, 2012 (NIST)– Intended uses of RMs– RM selection strategies

• November 7, 2012 (ASHG)– Status updates

• December 6, 2012 – Selection of initial RMs

• March 21, 2013 (XGEN Congress)• August 15, 2016 (NIST)

| 2

Workgroup AttendeesNIST Workshop – Aug 16, 2012

• Approximately 25 attendees– Federal, incl. FDA, CDA, NIST– Lab accreditation– Clinical reference labs– Platform technologies– Reference material / reagent providers– Research

| 3

Reference Materials Needed For• Clinical platform validation

– Sequencing System– Bioinformatics/Analysis Pipeline

• Clinical test development and validation– Whole genome– Targeted– Germline vs. tumor– Other?

• Research– Process development and QC

• Product development– Sequencing Systems– Software development | 4

NY State Guidelines – Oncology NGSMinimum Data Requirement - Validation

• Accuracy: Sequence a well-characterized reference sample (e.g. HapMap DNA GM12878) to determine error rate across all amplicons.

• Analytical sensitivity: Establish the analytical sensitivity of the assay by interrogating all variants in the 3 amplicons with the consistently poorest coverage, and all variants in 3 amplicons with consistently good coverage. This can initially be established with defined mixtures of cell line DNAs (not plasmids), but needs to be verified with 3-5 patient samples.

• Analytical specificity: Establish the analytical specificity of the assay by interrogating all variants in the 3 amplicons with the consistently poorest coverage, and all variants in 3 amplicons with consistently good coverage. This can initially be established with defined mixtures of cell line DNAs (not plasmids), but needs to be verified with 3-5 patient samples.

| 5

Reference Material – Intended Uses

• Characterize Platforms & Methods – DNA sequencing– Existing & upcoming NGS technologies– Research applications– Clinical diagnostics applications

• Not intended as reference material for– Validation of specific mutations in a panel

| 6

Preferred Reference Standard CriteriaReference Lab Perspective

• Commercially available (renewable and constant)

• Cost-effective

• Similar in performance to anticipated test samples

• Contain representative type and number of mutations tested

• Ability to adjust mutation frequency to monitor assay sensitivity

• Recognized by agencies responsible for recommending standards

Status Summary – RM Working Group

• Defined required RM characteristics• Initial set of RM samples selected

– NA12878• Many existing public and proprietary datasets• Listed in clinical guidelines to establish validation parameters• Consent limitations

– Commercial use, re-identification through sequence

• Under legal and IRB review by NIST

– Three PGP trios• One trio already available at Coriell

• Consent without withdrawal option may not meet ethical review standards

| 8

Reference Material Portfolio

Synthetic DNA• What types are most useful

– Variant types, sequence context, phasing• Assess strengths & weaknesses of platform• Test validation for existing and medically relevant variants

• What size• How many synthetic RMs

– Platform validation– Test validation

• Which existing sources should be considered– Available for research and for profit

• Source of commercial development and distribution– Manufactured under quality system for diagnostic applications

| 9 Discussion was postponed

AgendaRM Selection and Design Working Group

• Presentations:– NCI Cancer spike-ins – Jason Lih– TGen-Illumina tumor-normal pair study – Stephanie Pond– HorizonDx cell lines – Jonathan Frampton– Acrometrix FFPE controls – Mona Shahbazian

• What role should synthetic spike-ins play in GIAB? – Spike-ins could include NCI cancer spike-ins, TGen cancer rearrangement spike-ins, or new synthetic

DNA designs

• Should GIAB start to develop tumor-normal cell line pairs as Reference Materials• What is the relative importance of having genomic Reference Materials from a

variety of ancestry groups– What are the relative priorities of ancestral diversity, trios, and the broad consent of the

PGP for commercial redistribution– Trios in PGP are currently all Caucasian except for one Asian (Chinese) trio. If PGP

cannot have trios from other ancestry groups, should we use PGP Caucasian trios, PGP individuals without consented parents, or trios from other projects like 1000 Genomes

Workgroup AttendeesNIST Workshop – Aug 15, 2013

• Approximately 25 attendees– Federal, incl. FDA, CDA, NIST– Clinical reference lab– Platform technologies– Reference material / reagent providers– Research

| 11

SummaryJason Lih / NCI

• 53 plasmids with engineered cancer mutations– 1kb fragments– Mutation in center– Alien barcode in vicinity for differentiation from human DNA– Sequence is Sanger confirmed

• Low level mutations not observed in NGS verification

• Quantitation by qPCR– Observed concentrations in dilution series are linear, similar to cell

line DNA controls• Spike in controls for reference material or clinical samples• Linearized, no chromatin structure or FFPE exposure• No blocking I.P. for plasmid

| 12

Proposal: Interlab comparison of plasmid controls to assess performance and resemblance of FFPE derived DNAPossibility of QC Standard?

SummaryStephanie Pond / Illumina

• Tumor / Normal Cell Line Pair– COLO829B / COLO829– Pleasance et al, Nature 2010, 463:191-196– SNVs, InDels, CNVs, SVs (37)

• Confirmed by PCR & sequencing

– HiSeq 90x WGS at Illumina & TGEN– Complete Genomics WGS– Multiple dilution series analyzed– Consent needs investigation

| 13

Monitor: Offers no clear advantage over existing selected normal cell line trios

SummaryStephanie Pond / Illumina

• 9 synthetic fusion constructs– Spiked into COLO829 DNA– One Spike-In not detected by any fusion caller

• Confirmed to be present in sequence

| 14

Opportunity: Assess and optimize fusion caller softwareRecommendation: Recommend to make results available in data repository for software optimizationEarly status for fusion constructs, continue to monitor

SummaryJonathan Frampton / Horizon Diagnostics

• Large set of engineered cancer mutations in 3 cancer cell lines– Proprietary engineered cell lines– Confirmed identities of parental cell lines– Digital PCR confirmed mutational frequencies– FFPE embedded cells / DNA available– Linear serial dilution response to 0.05%– 11 core mutations are multiplexed with frequencies between

1% - 25%• digital PCR confirmed

– 19 additional mutations planned for multiplex with similar range of frequencies in mixture

| 15

Assess performance in interlab comparisonCombine with assessment of synthetic plasmids and Acrometrix controls?Opportunity for independently supplied reference materials

SummaryKara Norman / AcroMetrix

• Engineered cell line controls– 8 different multi mix controls– 12-26 variants per control

• 1-7 COSMIC variants / control

– Standards produced under cGMP / Quality System– Proprietary cell lines– FFPE embedded cells / DNA available

| 16

Assess performance in interlab comparison Combine with assessment of synthetic plasmids and Horizon Dx controls?Opportunity for independently supplied reference materials

Next Steps

• Working group to define testing protocol(s) and assess plasmid controls in interlab study– Combine with assessment of Horizon &

Acrometrix cell line controls?

• Confirm that performance is similar to clinical FFPE samples

| 17