Assay Development Breakout (red)

Who was in the room?• About half of attendees are active NGS

users• N=1 doing whole genome analyses• Everyone else doing “targeted” assays• Ion Torrent > Illumina > 454

Assay Development Breakout (red)

1a. Best practices for Analytical ValidationConsensus (strongest recommendation):• Not feasible to validate every possible variant• Different variants do have different analytical detection

characteristics• Precedent for Sanger sequencing remains valid (ie, no

need to validate every variant)• Statistical “quality” metrics can/should be applied

globally and individual “suboptimal” targets annotated accordingly

• Each unique sample type requires specific validation

1a. Best practices for Analytical ValidationMajority Opinion (not consensus):• Perhaps a tiered (risk-based) approach with stringent requirements

(sensitivity, specificity, accuracy) for a core set of actionable (common) variants that would each need individual controls.

• “other” variants perhaps validated with representative “classes” of controls/standards with acceptable performance metrics:– SNV, Indel (small and large), CNVs, complex, etc

• Controls could be well-characterized cell lines, synthetic DNA , pt samples (with dilutions to establish LOD)

• Who defines the “actionable” list of variants?• A consensus expert-vetted “actionable” list would be valuable• However, “actionable” likely varies between institutions and is highly labile• A proficiency/control panel containing all of the consensus “actionable”

variants would be valued

1b. What is the gold standard comparator assay?

Consensus• Given lack of a true “best” method, “reference

method” is perhaps a better term than “gold standard”

• Any clinically-validated assay can be used as a comparator for NGS, including:– Sanger sequencing (if allele burden sufficiently high)– Other single gene assays– Other multi-analyte assays

1c. How do we validate rare variants?

???

? validate the method itself (for “classes” of analogous variants), not each possible genotype

Universal application of statistical quality control metrics to each target

1d. Quality Control

Consensus• Minimally acceptable quality metrics must

be defined for:– Tumor content /cellularity (no consensus on

best method)–DNA quality and/or quantity– Limit of detection for common actionable

variants

Persistent Theme

Clinical validation of NGS assays is essentially no different than validation

of any other complex assay in the clinical molecular diagnostic lab

ie, good lab practices apply

Assessing comparability across labs. Are proficiency panels adequate?

• This issue is not substantially different in NGS compared to other assays .

• A well designed proficiency panel would generally be sufficient to establish accuracy.

• Mechanisms for sharing samples between labs, and central sources of proficiency panels should both be encouraged.

What is the complete molecular work-up for a tumor specimen?

• We really don’t know yet. Data integration challenges are huge. There is a long way to go before a broad approach could be recommended for general use.

• NGS of matched normal samples for gene panels is beneficial but not required. Adding normals might raise cost/reimbursement issues.

• No consensus on whether mutations of unknown significance should be reported.

• Multiple report formats are considered/in use. It may be desirable to report variants in a tiered fashion corresponding to their clinical significance.

Quantitation of Variant Frequencies

• Although the NGS data can generate quantitative data from a DNA sample, the clinical value of this information is substantially reduced by the lack of accurate measures of tumor fraction in most routine samples and further compounded by tumor heterogeneity.

• Strategies for tumor enrichment or independent tumor fraction measurement could potentially be developed in some clinical settings.

What is the role of whole genome sequencing?

• There was a strong consensus that in most centers WGS, while appropriate for discovery studies is not yet ready for routine clinical application to tumor characterization.

• Limitations include inadequate tumor read depth for dilute samples, high computational hardware and bioinformatics requirement.

• Enthusiasm would increase when sequence acquisition and data analysis could be routinely completed within a 6 week timeline.

What are the largest hurdles to overcome in implementing NGS?• Education of clinicians is a major issue. There

is a significant lag between progress in the lab and clinician awareness. Even in academic centers many/most clinicians are unprepared to deal with this information.

• Obtaining adequate specimens representative of the patient’s current tumor burden.