Univ. California San Francisco Medical Center...
Transcript of Univ. California San Francisco Medical Center...
Validation for Histology
Tim Morken, B.A. HTL(ASCP)
Univ. California San Francisco Medical Center
2
Wall Street Journal, January 2008
3
Presentation Goals
1. Explain the term “Validation"
2. Explain how proper validation leads to quality
3. Discuss validation in the context of a "Quality
System”
Upon completion of this presentation,
participants will be able to:
4
Presentation Outline
Quality Management System
CAP / JC / CLIA Requirements
Validation Requirements
Quality Control follows from Validation
5
Key References
Quality Assurance in Anatomic Pathology (book) • Nakhleh and Fitzgibbons, Ed., College of American Pathologists, 2005
College of American Pathologists (CAP) Checklists
• Common
• Lab General
• Anatomic Pathology
Quality Assurance for Design Control and Implementation of Immunohistochemistry Assays; Approved Guideline—Second Edition
• Clinical Laboratory Standards Institute publication I/LA28-A2
Reference handout
6
Quality Management
Quality Management
• Definitions
• Interaction
7
CAP Checklists: Quality Management
General Laboratory Checklist:
• GEN.13806 - 20371
• Quality Management Program
• Quality Monitoring
• GEN.30070 Validation of Accuracy
Anatomic Pathology Checklist
• ANP.10000
• Is Quality Management Program Defined and Documented?
• Procedure Manual: no specific checklist number, but details
what is to be in the procedure manual for each method or process
8
College of American Pathologists
Three Definitions concerning Quality Management:
• 1) Quality Assurance (QA)
• 2) Quality Control (QC)
• 3) Quality Improvement (QI)
Together these constitute a "Quality System"
• Interaction of these elements leads to better quality
• System must be "worked" to succeed.
9
CAP Quality Assurance Definition
1) Quality Assurance
• The practice of assessing performance in
all steps of the laboratory testing cycle
including pre-analytic, analytic and post-
analytic phases to promote excellent
outcomes in medical care.
10
… Deconstructed
"…assessing performance…"
• Requires a set of performance standards.
• Requires comparison of results against those
standards.
"…to promote excellent outcomes…"
• Requires action if standards are not met
11
CAP Quality Control Definition
2) Quality Control
• An integral component of quality assurance
consisting of the aggregate of processes and
techniques to detect, reduce, and correct
deficiencies in an analytical process.
12
… Deconstructed
"…processes and techniques to detect, reduce,
and correct deficiencies…"
• "…Detect…":
• Requires procedures that anticipate possible non-conformances.
• "…Reduce and Correct…":
• Leads to procedures that reduce or eliminate confounding results.
13
CAP Quality Improvement Definition
3) Quality Improvement
• The practice of continuously assessing and
adjusting performance using statistically and
scientifically accepted procedures.
14
… Deconstructed
"…continuously assessing and adjusting…"
• Requires regular review of results data
• Investigation of, and correction of root cause
"..statistically and scientifically…"
• Data trends over a time period
• Hypothesis of root cause
• Eliminating variables
• Testing solutions
15
Quality Management System Summary
Quality Assurance • Describe the quality process • Set standards
Quality Control
• Institute methods to identify problems • Record and report results
Quality Improvement • Track problem trends identified by QC results • Use trend information to correct problems
16
Validation Process
Validation
• Definitions
• Processes
17
Verification vs Validation
Verification:
• The process by which a laboratory determines that an
FDA-cleared/approved test performs according to the
specifications set forth by the manufacturer.
Validation:
• A defined process by which a laboratory confirms that a
laboratory-developed (LDT) or modified FDA-
cleared/approved test performs as intended or claimed.
18
Validation Definition (1)
Establishing documented evidence which
provides a high degree of assurance that
a specific process will consistently
produce a result or product meeting it's
predetermined specifications and quality
attributes
19
Validation Definition (2)
"Establishing documented evidence…"
• Documentation of validation testing is readily available
“…which Provides a high degree of assurance…"
• Studying an adequate number of samples to give
confidence that the new or changed process will work in
your laboratory
20
Validation Definition (3)
"…that a specific process will consistently produce…"
• Identifying areas of actual or potential weaknesses so
improvements can be made prior to implementation.
"…a result or product meeting it's predetermined
specifications and quality attributes."
• Establishing acceptance criteria before initiating the
validation study.
21
Validation and Quality Control
Validation
• Assay Design Specification
• Validation
Verification
Optimization
Standardization
Quality Control
• Controls identified in Validation process
• Standardized procedures defined by Validation
• Identification of process deficiencies
22
Validation Flow Chart
Validation
23
Validation – Optimization - Standardization
Validation
Optimization
Standardization
24
Validation Process
Infinity, with Validation
25
Validation, Analytic Phase Terms (1)
CAP General Validation
CAP GEN.42020-42163 Test Method Validation
• Follows CLIA CFR Sec 493.1253
• Does not apply well to Histology (Histology is usually qualitative)
• But the general principle applies:
• The laboratory must have data on each test's accuracy, precision,
analytic sensitivity, interferences and reportable range
• Unmodified FDA-cleared or approved tests: the lab may use
manufacturer information or published reports but lab must verify outside
data.
• Non-FDA cleared: Lab MUST verify or establish analytic accuracy,
precision, sensitivity, specificity and reportable range.
26
Validation Analytic Terms Applied to Histology
Accuracy:
• Compare results with New test to a previously validated test
Precision:
• Test samples with varying expression
• Intra-run, Inter-run tests, 10 slides each
Sensitivity:
• Lowest level at which the test will demonstrate the target
Interferences [Specificity]:
• What could interfere to give a false positive or negative result?
Reportable Range
• Establish a scoring system
• Definition of a positive result
• Criteria for rejection of the test
27
Validation: Analytical
Validate Accuracy with typical cases
• Sensitivity:
• Expression range of Positive cases, low to high (10-15 cases)
• Specificity
• Positive verses expected negative cases/tissues (10-15 cases)
• Determine best controls
• Range of expression, similar to expected cases (normal or disease?)
• Preferably acquired and processed in your institution
Determine Precision (Reproducibility) • Intra-run: 10 slides in one run
• Inter-run: 10 slides, Ten different runs with one slide each
• Should have similar staining pattern and intensity on all slides
• Test on each instrument (at each site) used to run that test/process
28
Precision and Accuracy
29
Sensitivity
Analytic Sensitivity: • Lowest amount of substance detectable by the test
• Can only be done with controls of known concentration
Diagnostic Sensitivity: • Ability of the test to determine true diagnostic positive verses
false negative (higher % FN = less sensitive)
• Requires comparison to a previously validated test
Histology-related Sensitivity: • Extent to which an test can be modified and still achieve target
recognition.
• NOTE: for antibodies, This is determined by antibody AND detection system!
(adapted from: Theoretical and Practical Aspects of Test Performance, in Immunomicroscopy, Taylor & Cote, 2005)
30
Specificity
Analytic Specificity • Accuracy on tests of known positive and negative controls
• Controls of known concentration
• Determine what could “Interfere" to confound the result
Diagnostic Specificity • Ability of a test to determine true diagnostic negative verses false
positives (Higher % FP = less specific)
• Requires comparison to a previously validated antibody
Histology-related Specificity • Ability of an test procedure to demonstrate its’ particular target in
the absence of staining of other molecules • Or, staining of other structures in addition to target structures/cells
(adapted from: Theoretical and Practical Aspects of Test Performance, in Immunomicroscopy, Taylor & Cote, 2005)
31
Introducing a New Test or Process
We want to introduce or modify a test, instrument,
procedure…
• Who wants it?
• What is the desired result?
• When will we know we achieved the desired result?
• Where will it be used?
• Why do we need it?
• How will it be implemented?
32
Initial Design Specification
Write a Specification:
• The need for the assay
• The "why's"
• Identify special requirements
• Identify suppliers
• Determine the expected results
• Develop the validation procedure
• Optimize the test/process
• Develop the standard protocol
33
Validation: Pre-Analytic
Pre-Analytic example:
• Acquisition: Range of time before sample is put in fixative
• Fixation: Type and Range of time in fixative
• Processing: Several processing schedules
• Sectioning: Section thickness, drying, heating temperature
• Storage: Range of time periods for unstained slide
storage (retention of antigenicity), temperature
• Range of block storage conditions
34
Optimization Overview
Verify Vendor Specifications
• “to ascertain the truth or correctness of, as by examination, research, or comparison”
Vendor literature is a starting point
• Review the datasheet
• Review references given by vendor
• Literature search adds to knowledge
• Compare with other vendors, users
Verify/Optimize:
• Test procedure
• Timing
• Temperature
• Expected results
35
Final Design Specification
Final outcome:
• Validated test/process
• Controls specified
• Specify the expected results
• Specify rejection criteria (tolerance)
• "QC Data Sheet" to guide interpretation
36
Total Test Validation
Pre-Analytic
• Acquisition, Fixation, Processing, Sectioning, Storage
Analytic
• Optimization
• Technologist training
• Instruments
Post-Analytic
• Controls
• Interpretation
• Reporting
• Pathologist performance
(Adapted from: Goldstein NS, et..al., Appl Immunohistochem Mol Morph 15(2):124-133)
37
CAP AP QA/QC and Validation, Routine Histo, SS
ANP.
• .11713 Histologic Prep Quality
• 11734 Slide Quality
• 21395 Special Stains/Studies
• Includes special stains and IHC
• 21450 Special Stain Quality
• 23004 – 23402 Digital Imaging
• Extensive validation and QC requirements
38
Verification or Validation for General Histology
H&E:
• NEW model stainer: Validate for all types of tissue to be stained
• After moving instrument to a new location (verify)
• When changing staining reagent (H or E) (validate)
• When changing timing protocol per reagent (validate)
• Any substantial change in process
Special Stains
• When changing anything in protocol (validate)
• Reagents, Timing, etc)
• When introducing new ancillary equipment (waterbath, heating plate, etc)
(verify)
• When moving from manual to an instrument or from one instrument
to another (validate)
• After instrument move (verify)
39
Control Blocks
Control blocks, Specials or IHC
• Validation of suitable tissue
• Signed by Medical Director/Technical Director
• Target in all slides?
• Stain last slide to be sure
• Multi-tissue necessary (range of expression)?
• Validate storage conditions
• Temperature
• Time period target is viable is mounted sections
• Can you trace-back to original case?
40
Graduated Expression Arrays
Low Medium High
ER
PR
HER2
Courtesy: Pantomics, Ltd, www.pantomics.com
41
Verifcation or Validation for General Histology
Tissue processor
• NEW model: Validate for all types of tissue to be processed
• Move: Verify performance in new site
• Second instrument of same type: Verify with parallel testing on
original instrument.
• Changing protocol (reagents or timing): Validate
Microtome
• NEW model: Validate it cuts with good results
• Move: verify it is functional and cuts same as before move (pre-
move test for comparison!)
• Second instrument of same type: verify with parallel testing of
existing instrument
42
Optimization Method
Optimization
• Vendor-supplied protocols are recommendations
• Ideally work it into your regular protocol
• Test variations in timing, concentration,
temperature
• Variety of positive and negative samples to
determine sensitivity and specificity
• Staining method (Manual or Automated)
43
Optimization Result: General Histology
Write an optimized procedure for each
test/process, including:
• Materials
• Reagents
• Equipment
• Procedure
• Possible pitfalls
• Expected results
44
CAP AP Checklist, IHC
ANP.
• 12425 ASR disclaimer for report
• 21850 Positive and Negative controls for immunofluorescence
• 22250 Procedure for each antibody
• 22300 Documented modifications for fixation other than formalin
• 22500 Buffer pH monitoring
• 22550 Positive control use
• 22570 Negative control use (tissue, reagent)
• 22615 Avidin/Biotin blocking/controls
• 22660 Review/Recording of IHC results
• 22750 Validation of new antibody (except ER, PR, Her2)
• 22760 New lot validation for Antibody and Detection system
• 22800 Automated IHC staining instrument maintenance records
• 22900 Quality of IHC stains
• 22997 Her2 validation (IHC and ISH)
45
Optimization Result: IHC
Write an optimized procedure for each
antibody, including:
• .22250: Test Procedure
• reagents, dilution, HIER, etc
• .22300: Fixation
• .22550: Positive Control
• .22570: Negative Control
• .22625: Avidin-biotin Block (if necessary)
46
Key Take-away
Validation Drives Quality
• Ensures test works as intended
• Develops Standardized Procedures
• Identifies Quality Control methods/materials
Questions?
Thank you.