Analytical Evaluation Standards for NGS

Zivana Tezak, PhDOIR/ CDRH/FDA

August 28, 2015NIST, Gaithersburg, MD

Genome in a Bottle Consortium--August 2015 Workshop

2

Disclaimer• NGS is raising new policy / regulatory issues; thoughts

presented here are preliminary and do not represent proposed or finalized FDA policy – these are just preliminary ideas or approaches we are considering

Background (“Vision”)• February 2015 - “Optimizing FDA’s Regulatory Oversight

of Next Generation Sequencing Diagnostic Tests” http://www.fda.gov/MedicalDevices/NewsEvents/WorkshopsConferences/ucm427296.htm

Precision Medicine Initiative - FDA

Vision: Implement new regulatory policies to promote research and accelerate the translation of precision medicine technologies into treatments that benefit patients.•Near Term: Implement standards and shared resources that will enable the development of knowledge for research and patient decision making•Longer Term: Implement standards-based regulation of NGS diagnostic tests that will ensure that the tests patients receive provide accurate, reproducible, and meaningful results

3

FDA – Modernizing Regulation for NGS tests

Adopt a flexible, dynamic regulatory system(knowledgebase /

databases)Develop and implement standards

to assure quality

Develop cloud-based open-source tools

to help test developers meet

standards

Analytical evaluation

Bioinformatics

Clinical significance

Build growing suite of standards

– Performance standard documentse.g., ACMG, CLSI, NYS DOH, CAP

– Material standards e.g., NIST/GIAB

5

General Overview / Framework

• Analytical: Performance Standards as a Regulatory Tool for NGS Testing – Analytical performance standard elements – Uses / areas in need of separate analytical standards – Overall architecture / workflow of clinical NGS test (from sample to

EHR)– Example of a possible performance standard (single use)– Existing guidelines and standards for NGS tests– Stakeholder communities building some of the standards – Cloud based, open source bioinformatics tools repository to facilitate

standard development

6

Performance standard elements• Intended use of the test (e.g., carrier screening, diagnosis of

inherited disease, cancer risk prediction)• Transparency • Test description (methodology, instrumentation, reagents,

software, etc)• Metrics (quality scores, read depth, sensitivity/specificity, etc)

– what to measure• Protocols and studies to perform (specimen type, sample

numbers, comparator methods, etc) – approaches on how to measure analytical performance

• Acceptance criteria, thresholds for measurements• Performance data / result presentation

7

Major areas in need of separate standardsInitial assessments; will develop and change. Where to start?

– Targeted clinical assays • Germline targeted assays • Somatic targeted assays (cancer panels)

– Exome-based clinical assays (heritable, rare undiagnosed disease)– WGS-based clinical assays

• For germline uses• For somatic uses

– Method or analytic quality-based standards• for methodology, that would include different classes of variants, genomic regions,

allele ratios, different technology• for platform (use of standard materials, panels) • for test building blocks (e.g., preanalytical, library, software, quality system

approach, etc)– Run quality standards – including computational solutions (WGS only?)– Terminology (language agreement and definitions)– Agreement on appropriate comparators and “gold standards”– Material standards, reference samples, disease-specific sample sets, and

controls– Data formats and interchangeability / interoperability 8

By intended use

Overall architecture of NGS test• High-level NGS test steps, from patient sample through variant

call and interpretation: – Preanalytical (from tissue -> DNA); includes specimen quality

assessment, DNA preparation, library preparation– Sequencing (from DNA -> raw sequence data) – Sequence bioinformatics (from raw sequence data -> VCF); includes

pipeline from aligning, filtering, read mapping, annotation, contamination, through call determination

– Interpretation (including knowledgebases) – Reporting – EHR and archival

• Will differ for different platforms• Overall standards architecture?

9

NGS Workflow Standards and Tools

Examples of Intended Uses

Hereditary Diseases (targeted panels)

Hereditary Diseases (exomes)

Cancer panels Methodology

Preanalytical

Sequencing

Data Processing (informatics)

Knowledgebase (databases)

Interpretation

Reporting

NGS Workflow

Performance standard example

• Single use• Possible analytical approach • Heritable monogenic disorders

– Diagnosis– Targeted panels– Exomes?

• Additional examples - – Special controls for NGS platform and autosomal recessive carrier

screening

11

Need flexibility for evolving technology and knowledge!

Existing standards and guidelines

• Currently existing or ongoing standard development, how they correspond to FDA needs:– Material standards – e.g., NIST, sample panels– Methodological standards – e.g., NYSDOH, CAP, MolDx, GA4GH, Nex-

StoCT, ACMG, HGVS, CPIC, CLSI • What can be used, for what purpose• Identify gaps • Questions:

– Who should fill identified gaps – What is acceptable timeline?

• Need built-in flexibility12

Stakeholder communities

• Stakeholders involved for each area• Ways to work together • Possible conveners

– SEQC Consortium– NIST / GIAB Consortium– CERSI– CDC– CLSI– GA4GH– others?

13

Bioinformatics - General Landscape

In the NGS-based assay space -

•Some labs offer the test by sequencing the samples but outsourcing bioinformatics analysis to other companies

•Some labs require support from 3rd party companies to host their bioinformatics pipeline, but the labs perform the analysis

•Some labs offer the test by sequencing the samples AND have deep bioinformatics and analytic capability

A

B

C

4

FastQ

CommunityNGS-Based Test Developers (large and small), NIST, FDA Scientists, Standards Bodies, Academic Centers, Patient-Facing Providers, Consortiums

Analysis and Pipeline(s)

Security and Privacy HIPAA/HITECH, CAP, ISO27001Uniquely identified and immutable dataVersion-controlled applications

Reference Data

VCF

Quality Threshold

Approve

Reject

precisionFDA

Courtesy of Taha Kass-Hout, FDA Chief Health Informatics Officer

Advancing the accuracy and reproducibility of NGS•Crowd-sourced, cloud-based platform•Will provide tools and open access resources•Will allow the community to test, pilot, and validate approaches to NGS

Community

Development and Use of Analytical Standards

• Current practices• Technical knowledge

GA4GH

NIST Others

FDA

Regulatory proposalCompanies

NGS labs

Stakeholder input Discussion paper

Public workshop

Comment analysis

Standards bodiesProfessional

Societies

Success of Precision Medicine Requires:

• Safe and accurate diagnostic tests that reliably identify individual variation

• Learning health systems that enable researchers and clinicians to learn from and inform the patient experience

• Development of targeted therapies that are more efficacious or have less deleterious side effects for specific individuals

• Updated research and regulatory policies that catalyze the development of new treatments while protecting patients