H62 Validation of Assays Performed by Flow Cytometry · associated with this use. This guideline is...

H62 Validation of Assays Performed by Flow Cytometry

Proposed Draft Please Review and Comment

This Proposed Draft document is provided for broad, thorough review in the Clinical and Laboratory Standards Institute (CLSI) consensus review process. The Proposed Draft document is undergoing document development committee, expert panel, and active member vote, as well as concurrent Consensus Council, Board of Directors, and public review. A 60-day period is being allocated for this voting and comment period. Actions requested: Review this draft for scientific accuracy, practicality, comprehensibility, and overall quality and utility. Submit your vote, as applicable. Submit comments by the close of the review period. Upon approval, with the resolution of comments on the Proposed Draft, this document will be distributed as a Final Draft to the Consensus Council for approval to publish. All voting members of the committee(s) involved in the development of this Proposed Draft document have completed the CLSI Disclosure of Interests form; these forms are on file at the CLSI office.

Voting and comment period ends 18 November 2019

Submit comments via e-mail to [email protected].

This guideline includes validation strategies for cell-based assays. This guideline also includes recommendations for instrument qualification and standardization, assay optimization, and examination phase and postexamination phase best practices.

“DRAFT DOCUMENT. This draft CLSI document is not to be reproduced or circulated for any purpose other than review and comment. It is not to be considered either final or published and may not be quoted or referenced. 18 September 2019.”

Appeal Process When it is believed that an objection has not been adequately considered and responded to, the process for appeal, documented in the CLSI Standards Development Policies and Processes, is followed. All comments and responses submitted on draft and published documents are retained on file at CLSI and are available upon request. For additional information on committee participation or to submit comments, contact CLSI. Clinical and Laboratory Standards Institute 950 West Valley Road, Suite 2500 Wayne, PA 19087 USA P: +1.610.688.0100 F: +1.610.688.0700 www.clsi.org [email protected]

H62


Virginia Litwin, PhD Teri Oldaker, BS, SCYM(ASCP)CM Elena Afonina, PhD Antony C. Bakke, PhD, D(ABMLI), CLS David Barnett, PhD Jacqueline M. Cleary, BS, MT(ASCP) Fiona E. Craig, MD Lorella Di Donato, PhD Bruce H. Davis, MD Thomas N. Denny, MSc Steven Eck, PhD Nancy Fine, MT(ASCP) Alessandra Vitaliti Garami, PhD Cherie Green, BS Benjamin Hedley, PhD, SCYM(ASCP) Shuguang Huang, PhD Andrea J. Illingworth, MS

Chunjie (Cassie) Jiang, MS(ASCPi) Mike Keeney, ART Wolfgang Kern, MD Raul Louzao, MPA, BS, MT(ASCP) Natalia Kokorina, MD Sarah Maremont, MLS, SCYM(ASCP)CM Laura Marszalek, BS, MT(ASCP) Katharine A. Muirhead Ahmad Al Samman, BSc John L. Schmitz, PhD Alan M. Stall, PhD Maryalice Stetler-Stevenson, MD, PhD Jennifer J. Stewart, PhD Horacio Vall, CLS, QCYM(ASCP) Paul K. Wallace, BS, MS, PhD Lili Wang, PhD Brent L. Wood, MD, PhD Yuanxin Xu, MD, PhD

Abstract Clinical and Laboratory Standards Institute guideline H62—Validation of Assays Performed by Flow Cytometry focuses primarily on analytic method validation. There are currently no official guidance documents for the validation of assays performed by flow cytometry. The existing guidance documents for the validation of biochemical methods for quantifying soluble analytes found in plasma, serum, and urine are not fully applicable for quantification and characterization of cellular measurands. As will be discussed in-depth within this guideline, the primary challenges in the validation of flow cytometry arise from the fact that the data generated are not derived from a calibration curve and there is a lack of true reference standards. Additional topics covered include instrument qualification and standardization, assay optimization, as well as examination phase and postexamination phase best practices. The recommendations presented herein are applicable to a wide range of flow cytometry laboratories, including basic research facilities, biopharmaceutical companies, medical laboratories, and manufacturers. H62 provides specific guidelines for the appropriate analytical method validation approach based on the intended use of the data and regulatory and accreditation requirements, if any, associated with this use. This guideline is designed to assist any laboratory using flow cytometry, including manufacturers, in developing, validating, verifying, controlling, analyzing, and implementing fluorescent cell-based assays. Clinical and Laboratory Standards Institute (CLSI). Validation of Assays Performed by Flow Cytometry. 1st ed. CLSI guideline H62. Clinical and Laboratory Standards Institute, 950 West Valley Road, Suite 2500, Wayne, Pennsylvania 19087 USA, 2019.

H62

ii


Committee Membership Consensus Council Dennis J. Ernst, MT(ASCP), NCPT(NCCT) Chairholder Center for Phlebotomy Education USA Mary Lou Gantzer, PhD, FACB Vice-Chairholder USA J. Rex Astles, PhD, FACB, DABCC Centers for Disease Control and Prevention USA Thomas R. Fritsche, MD, PhD, FCAP, FIDSA Marshfield Clinic USA

Loralie J. Langman, PhD, DABCC, FACB, F-ABFT Mayo Clinic USA Michelle McLean, MS, MT(ASCP) Greiner Bio-One, Inc. USA Tania Motschman, MS, MT(ASCP)SBB Laboratory Corporation of America USA James R. Petisce, PhD BD Diagnostic Systems USA

Andrew Quintenz Bio-Rad Laboratories, Inc. USA Robert Rej, PhD New York State Department of Health – Wadsworth Center USA Zivana Tezak, PhD FDA Center for Devices and Radiological Health USA

Document Development Committee on Validation of Assays Performed by Flow Cytometry Virginia Litwin, PhD Chairholder Caprion Biosciences Canada Teri Oldaker, BS, SCYM(ASCP)CM Vice-Chairholder Independent Consultant USA Raul Louzao, MPA, BS, MT(ASCP) Committee Secretary Duke University USA David Barnett, PhD Sheffield Teaching Hospital NHS Foundation Trust United Kingdom

Jacqueline M. Cleary, BS, MT(ASCP) FDA Center for Devices and Radiological Health USA Thomas N. Denny, MSc Duke University Medical Center USA Cherie Green, BS Genentech, Inc. USA Mike Keeney, ART College of American Pathologists Canada

Wolfgang Kern, MD MLL Münchner Leukämielabor GmbH Germany Jennifer J. Stewart, PhD Flow Contract Site Laboratory, LLC USA Lili Wang, PhD National Institute of Standards and Technology USA

H62

iii


Expert Panel on Hematology and Immunology and Ligand Assay Kathleen M. Curran, BSMT Chairholder New York State Department of Health USA Donna D. Castellone, MS Vice-Chairholder New York Presbyterian Hospital USA Richard J. Baltero, MD, PhD, FCAP East Carolina University USA Francois Depasse, PharmD Diagnostica Stago France

Mary J. Doyle, PhD, MS, MT(ASCP) Instrumentation Laboratory USA Ian A. Giles, MD Sysmex American, Inc. USA Paula Keating, PhD, BSc Canterbury Health Laboratories New Zealand Yutaka Nagai, PhD Nihon Kohden Corporation Japan

Elizabeth Sheppard, HT(ASCP), MBA Ventana Medical Systems, Inc. USA Elizabeth Stafford, PhD FDA Center for Devices and Radiologic Health USA Elizabeth M. Van Cott, MD Massachusetts General Hospital USA

Staff Clinical and Laboratory Standards Institute USA David E. Sterry, MT(ASCP) Project Manager

Megan L. Tertel, MA, ELS Editorial Manager Catherine E.M. Jenkins Editor

Kristy L. Leirer, MS Editor Laura Martin Editor

H62

iv


Acknowledgment CLSI, the Consensus Council, and the Document Development Committee on Validation of Assays Performed by Flow Cytometry gratefully acknowledge the following volunteers for their important contributions to the development of this guideline: Elena Afonina, PhD USA Antony C. Bakke, PhD, D(ABMLI), CLS Independent Consultant USA Bruce H. Davis, MD USA Fiona E. Craig, MD Mayo Clinic Hospital Phoenix USA Lorella Di Donato, PhD Caprion Biosciences Inc. Canada Steven Eck, PhD Medimmune Inc USA Nancy Fine, MT(ASCP) Beaumont Health System USA Alessandra Vitaliti Garami, PhD Novartis Pharma AG Switzerland Benjamin Hedley, PhD, SCYM(ASCP) London Health Sciences Centre Canada

Shuguang Huang, PhD Stat4ward LLC USA Andrea J. Illingworth, MS Dahl-Chase Pathology Associates PA USA Chunjie (Cassie) Jiang, MS(ASCPi) National Institute of Health USA Natalia Kokorina, MD Beckman Coulter USA Raul Louzao, MPA, BS, MT(ASCP) Duke University USA Sarah Maremont, MLS, SCYM(ASCP)CM Sonora Quest Laboratories/Laboratory Sciences of Arizona USA Laura Marszalek, BS, MT(ASCP) Northwestern Memorial Hospital USA Katharine A. Muirhead SciGro, Inc USA

Ahmad Al Samman, BSc Hamad Medical Corporation Qatar John L. Schmitz, PhD UNC Chapel Hill USA Alan M. Stall, PhD BD Biosciences USA Maryalice Stetler-Stevenson, MD, PhD National Institutes of Health, Clinical Center USA Horacio Vall, CLS, QCYM(ASCP) Integrated Oncology USA Paul K. Wallace, BS, MS, PhD Roswell Park Cancer Institute USA Brent L. Wood, MD, PhD University of Washington Medical Center USA Yuanxin Xu, MD, PhD Alnylam Pharmaceuticals USA

H62

v


Contents

Abstract .................................................................................................................................................... i

Committee Membership ......................................................................................................................... ii

Foreword .............................................................................................................................................. vii

Chapter 1: Introduction ....................................................................................................................... 1

1.1 Scope ............................................................................................................................. 1 1.2 Standard Precautions ..................................................................................................... 2 1.3 Terminology .................................................................................................................. 2

Chapter 2: Path of Workflow and Quality System Essentials ............................................................. 9

2.1 Process Flow Chart ....................................................................................................... 9 2.2 Introduction to Quality System Essentials .................................................................. 11 2.3 Personnel Management ............................................................................................... 11 2.4 Equipment Management ............................................................................................. 12 2.5 Process Management .................................................................................................. 12 2.6 Documents and Records Management ........................................................................ 13

Chapter 3: Fit-for-Purpose Approach to Analytical Method Validation for Flow Cytometric Methods ................................................................................................................................... 14

3.1 Considerations for the Validation of Flow Cytometric Methods ................................ 14 3.2 Application of Standard Validation Parameters for Flow Cytometric Methods ......... 16 3.3 Regulatory Requirements for Validation .................................................................... 20

Chapter 4: Instrument Qualification, Setup, and Standardization ..................................................... 23

4.1 Installation Qualification and Operational Qualification ............................................ 23 4.2 Performance Qualification .......................................................................................... 26 4.3 Cross-Instrument, Cross-Site Standardization ............................................................ 33 4.4 Compensation ............................................................................................................. 38 4.5 Longitudinal Performance .......................................................................................... 42 4.6 Qualification and Verification of Instruments for the Intended Purpose .................... 42

Chapter 5: Assay Development and Optimization ............................................................................ 44

5.1 Assay Development .................................................................................................... 48 5.2 Assay Evaluation ........................................................................................................ 63 5.3 Assay Optimization ..................................................................................................... 64 5.4 Assay Characterization ............................................................................................... 66 5.5 Documentation ............................................................................................................ 67

Chapter 6: Analytical Method Validation ......................................................................................... 68

6.1 Validation Planning Phase (Say It!) ............................................................................ 68 6.2 Validation Implementation (Do It!) ............................................................................ 85 6.3 Validation Reports (Prove It!) .................................................................................... 85 6.4 Validation Plans Based on Intended Use and Regulatory Requirements ................... 86

Chapter 7: Examination and Postexamination Phase ........................................................................ 87

7.1 Instrumentation ........................................................................................................... 89 7.2 Reagents ...................................................................................................................... 89 7.3 Examination Phase ...................................................................................................... 91 7.4 Postexamination Phase ............................................................................................... 93 7.5 File Storage and Retention .......................................................................................... 96

H62

vi


Contents (Continued)

7.6 Training Records and Competency ............................................................................. 98 7.7 Accreditation ............................................................................................................... 98 7.8 External Quality Assessment/Proficiency Testing ...................................................... 99

Chapter 8: Conclusion ..................................................................................................................... 100

Chapter 9: Supplemental Information ............................................................................................. 100

References .............................................................................................................................. 101

Additional Resources ............................................................................................................. 107

Appendix A. Detailed Validation Menus ............................................................................... 108

Appendix B. Critical Reagents Beyond Antibodies ............................................................... 143

Appendix C. Temperature Monitoring Options ..................................................................... 145

Appendix D. Precision Statistical Discussion ........................................................................ 146

Appendix E. Evaluation of Cocktail Stability ........................................................................ 149

Appendix F. Additional Tables for Qualitative Validation .................................................... 150

The Quality Management System Approach ......................................................................... 152

Related CLSI Reference Materials ........................................................................................ 154

H62

vii


Foreword 1 2 Multiparametric flow cytometry is one of the leading technologies for cellular analysis because it allows 3 for the simultaneous detection of numerous characteristics of individual cells with relatively high 4 throughput. Although this technology has been a critical component in medical laboratories and drug 5 development for many years, its importance has increased dramatically in the past few years. 6 7 In medical laboratories, flow cytometry became an important platform in the mid-1980s when CD4 T-cell 8 counts became critical measurements in the diagnosis and treatment of AIDS. The importance of flow 9 cytometry continued to grow as flow cytometric methods were used for CD34+ cell counting for 10 hematopoietic stem cell transplantation, diagnosis, and to guide treatment decisions for leukemia and 11 lymphoma as well as other diseases of the blood, such as paroxysmal nocturnal hemoglobinuria. More 12 recently, flow cytometry has become a critical tool in the assessment of minimal residual disease in 13 leukemia and lymphoma. 14 15 In the biopharmaceutical industry, this flexible and powerful platform has been important in supporting 16 biomarkers in all phases of the drug development pipeline for nearly 20 years. Yet with the introduction of 17 immunotherapeutic agents, novel vaccines, and cell-based therapies, flow cytometry has become a critical 18 tool supporting every aspect from manufacturing to primary end point determinations in medical 19 development. 20 21 This heightened role of flow cytometry in both laboratory medicine and drug development results in an 22 increased need for high quality and validated methods, which, in turn, creates a need for official guidance 23 from regulatory agencies and accreditation organizations regarding the validation of assays performed by 24 flow cytometry. Because no official guidance documents exist for validation of assays used in flow 25 cytometry, H62 seeks to fill the need for consensus recommendations. 26 27 Because data should be reliable, no matter the intended use, the intended target audience for this guideline 28 is broad, and includes both regulated and nonregulated laboratories, such as basic research laboratories. As 29 such, a one-size-fits-all approach to analytical method validation is not appropriate. Therefore, this 30 guideline presents a fit-for-purpose (FFP) approach to validation, as described in Chapters 3 and 6. Briefly, 31 the concept of an FFP method validation was introduced in 2005 in a publication from the American 32 Association of Pharmaceutical Scientists. This publication conveyed the message that some degree of 33 validation should always be conducted to generate reliable data, but that the level of validation should be 34 tailored to the intended use of this data. If the intended use of the data changes, additional validation should 35 be conducted to meet the new intended use and the regulatory requirements associated with this use. This 36 term—FFP method validation—appears in numerous publications including the Bioanalytical Method 37 Validation: Guidance for Industry, which was published by the US Food and Drug Administration in 2018.1 38 Nonetheless, the term is controversial because it is interpreted by some as “cheating” and indeed, it is 39 sometimes used as a justification for inadequate validation. This guideline presents the minimal standards 40 for FFP as well as full validation for a wide variety of intended uses (see Table 21 and Appendix A for 41 more information). 42 43 Flow cytometric methods pose particular validation challenges due to the complexity of cellular 44 measurands, the lack of reference materials, and the fact that data are not derived from a calibration curve. 45 Thus, the existing recommendations for validation of biochemical methods for quantifying soluble analytes 46 found in plasma, serum, and urine cannot be fully applied in the validation of flow cytometric methods for 47 quantifying cellular measurands. 48 49 50 51

H62

viii


In addition to discussing analytical method validation, this guideline provides guidance for instrument 52 characterization and standardization; assay development and optimization; and examination phase and 53 postexamination phase best practices. The content is designed to assist laboratories and manufacturers in 54 developing, validating, verifying, controlling, analyzing, and implementing cell-based assays performed by 55 flow cytometry. 56 57 NOTE: The content of this guideline is supported by the CLSI consensus process and does not necessarily 58 reflect the views of any single individual or organization. 59 60 Key words 61 62 Cell-based assay, fit-for-purpose, flow cytometry, laboratory-developed tests, standardization, validation, 63 verification 64 65

Use of ECD® (PE/Texas Red® or sulforhodamine 101 sulfonyl chloride), Alexa Fluor 488® (succinimidyl ester free acid), Brilliant Violet™, and SRM® 1934 in this guideline is not an endorsement on the part of CLSI. With each use of the trade name, the words “or the equivalent” are added to indicate that this guideline also applies to any equivalent products.

66 67

H62

1


Validation of Assays Performed by Flow Cytometry 68 69 Chapter 1: Introduction 70 71

This chapter includes: Guideline’s scope and applicable exclusions Standard precautions information “Note on Terminology” that highlights particular use and/or variation in use of terms and/or

definitions Terms and definitions used in the guideline Abbreviations and acronyms used in the guideline

72 1.1 Scope 73 74 This guideline focuses on the unique requirements for the validation of cell-based assays performed by flow 75 cytometry, which are not covered in other CLSI documents. Recommendations and practical instructions 76 are provided for preexamination phase activities, such as sample requirements; reagent optimization 77 evaluation; instrument qualification (IQ) and standardization; and, finally, assay optimization and 78 validation. Guidance for examination phase activities (instrument monitoring and QC) are described, as are 79 best practices for postexamination activities including data review, reporting, storage, and retention. 80 Situations in which the validation requirements, or quality monitoring practices, are the same for flow 81 cytometric methods are not discussed in this guideline; rather, the reader is referred to the appropriate CLSI 82 documents for more information. 83 84 This guideline is intended for use in a flow cytometry environment in which preclinical and clinical 85 assessments are conducted. For example, this includes, but is not limited to: 86 87 Medical and research laboratories (academic and nonacademic) 88 Regulatory agencies 89 Reagent, assay, and instrument manufacturers 90 Drug discovery, development, and manufacturing companies 91 92 Cell type–specific assay development (eg, immunophenotyping, neoplastic cell, or red blood cell analysis) 93 is not discussed in this guideline, because these topics are covered in CLSI documents H42,2 H43,3 and 94 H52.4 The validation of flow cytometric assays for noncellular measurands or soluble analytes and software 95 validation are also outside the scope of this guideline. For more information about software validation, see 96 CLSI document AUTO13.5 97

H62

2


1.2 Standard Precautions 98 99 Because it is often impossible to know what isolates or specimens might be infectious, all patient and 100 laboratory specimens are treated as infectious and handled according to “standard precautions.” Standard 101 precautions are guidelines that combine the major features of “universal precautions and body substance 102 isolation” practices. Standard precautions cover the transmission of all known infectious agents and thus 103 are more comprehensive than universal precautions, which are intended to apply only to transmission of 104 bloodborne pathogens. Published guidelines are available that discuss the daily operations of diagnostic 105 medicine in humans and animals while encouraging a culture of safety in the laboratory.6 For specific 106 precautions required for preventing the laboratory transmission of all known infectious agents from 107 laboratory instruments and materials and for recommendations for the management of exposure to all 108 known infectious diseases, refer to CLSI document M29.7 109 110 1.3 Terminology 111 112 1.3.1 A Note on Terminology 113 114 CLSI, as a global leader in standardization, is firmly committed to achieving global harmonization 115 whenever possible. Harmonization is a process of recognizing, understanding, and explaining differences 116 while taking steps to achieve worldwide uniformity. CLSI recognizes that medical conventions in the global 117 metrological community have evolved differently in different countries and regions and that legally 118 required use of terms, regional usage, and different consensus timelines are important considerations in the 119 harmonization process. CLSI recognizes its important role in these efforts, and its consensus process 120 focuses on harmonization of terms to facilitate the global application of standards and guidelines. Table 1 121 is provided to clarify the intended interpretations of the following terms. 122 123 H62 uses the term “laboratorian” rather than various other terms used to describe an individual who works 124 in a laboratory (eg, laboratory technician, laboratory technologist, medical laboratory scientist, medical 125 technician, and medical technologist). H62 uses the term “staff” to refer to the collective group of 126 employees who work for an organization. 127 128 Table 1. Common Terms or Phrases With Intended Interpretations 129

Term or Phrase Intended Interpretation “Needs to” or “must”

Explains an action directly related to fulfilling a regulatory and/or accreditation requirement or is indicative of a necessary step to ensure patient safety or proper fulfillment of a procedure

“Require” Represents a statement that directly reflects a regulatory, accreditation, performance, product, or organizational requirement or a requirement or specification identified in an approved documentary standard

“Should” Describes a recommendation provided in laboratory literature, a statement of good laboratory practice, or a suggestion for how to meet a requirement

130 1.3.2 Definitions 131 132 analytical specificity – ability of a measurement procedure to measure solely the measurand8; NOTE 1: 133 Lack of specificity may be called analytical interference; NOTE 2: A type of interference in 134 immunochemistry measurement procedures may be cross-reactivity; NOTE 3: Specificity of a 135 measurement procedure should not be confused with diagnostic specificity; NOTE 4: Specificity is the 136 ability of an analytical method to determine only the component it purports to measure; the extent to which 137 the assay responds only to (all subsets of) a specified analyte and not to other substances present in the 138 sample; NOTE 5: For qualitative or semiquantitative assays, it is the method’s ability to obtain negative 139 results in concordance with negative results obtained by the reference method; NOTE 6: Analytical 140

H62

3


specificity of an immunohistochemical assay is largely dependent on the characteristics of the primary 141 antibody in the total test system. 142 143 accuracy – (of measurement) closeness of agreement between a measured quantity value and a true quantity 144 value of a measurand.9 145 146 application setting//assay-specific setting – user-specified instrument settings based on mean or median 147 fluorescence intensity (MFI or MdFI) target values using fluorescence reference particles used to set-up an 148 instrument’s detector voltages for a defined assay or application. 149 150 background noise – signal present when no particles are flowing in the sample stream; NOTE: 151 Background noise is one factor that limits the sensitivity of fluorescence detection. 152 153 calibration material – a material or device of known or assigned quantitative characteristics (eg, 154 concentration, activity, intensity, reactivity, responsiveness) used to adjust the output of a measurement 155 procedure or to compare the response obtained with the response of a test specimen and/or sample; NOTE 156 1: The quantities of the analytes of interest in the calibration material are known within limits ascertained 157 during its preparation and may be used to establish the relationship of an analytical method’s response to 158 the characteristic measured for all methods or restricted to some; NOTE 2: Calibration materials with 159 different amounts of analytes may be used to establish a calibration or response “curve” over a range of 160 interest; NOTE 3: The terms “primary” and “secondary standard” are used by WHO and ISO to refer to 161 calibration materials; NOTE 4: The calibration material must be traceable to a national or international 162 reference preparation or reference material when these are available. 163 164 calibrator – measurement standard used in calibration9; NOTE 1: The term “calibrator” is only used in 165 certain fields9; NOTE 2: In NBS05,10 a material or device of known or assigned quantitative characteristics 166 (eg, concentration, activity, intensity, reactivity, responsiveness) used to adjust the output of a measurement 167 procedure or to compare the response obtained with the response of a test specimen and/or sample; NOTE 168 3: The quantities of the analytes of interest in the calibration material are known; NOTE 4: Calibration 169 materials with different amounts of analytes may be used to establish a calibration or response “curve” over 170 a range of interest; NOTE 5: An instance in which the testing procedure is performed by a health care 171 provider for an individual; NOTE 6: Calibrators with different quantities or analytes may be used to 172 establish a quantity/response “curve” over a range of interest. 173 174 coefficient of variation (CV) – a measure of relative precision; NOTE 1: For a non-negative characteristic, 175 the ratio of the standard deviation to the average; NOTE 2: It is often multiplied by 100 and expressed as 176 a percentage. 177 178 context of use – is a complete and precise statement that describes the appropriate use of the biomarker 179 and how the qualified biomarker is applied in drug development and regulatory review. The context of use 180 statement also describes important criteria regarding the circumstances under which the biomarker is 181 qualified; NOTE: Context of use is central to biomarker qualification.11 182 183 equivalent reference fluorophores (ERF) – unit of fluorescence intensity; the number of reference 184 fluorophores in solution that produce the same fluorescence signal as a single dyed microsphere or bead. 185 The ERF value assignment is provided by the US National Institute of Standards and Technology (NIST) 186 and is traceable to NIST SRM® 1934a (or the equivalent) standard reference material. 187 188 189 190 191

a NIST SRM® is a registered trademark of the US National Institute of Standards and Technology.

H62

4


expression – the conversion of the genetic instructions present in a DNA sequence into a unit of biological 192 function in a living cell; NOTE: Expression typically involves the process of transcription of a DNA 193 sequence into an RNA sequence followed by translation of the RNA into protein; the RNA may be spliced 194 before translation to remove introns. 195 196 fit-for-purpose (FFP) – the level of validation should be appropriate for the intended purpose of the data. 197 198 flow cytometry standard (FCS) – data file standard for the reading and writing of data from flow 199 cytometry experiments. 200 201 fluorescence compensation – mathematical or electronic correction applied to flow cytometric data to 202 account for the overlapping fluorescence emission from one fluorochrome in the wavelength region in 203 which a second fluorochrome is measured. 204 205 fluorescence intensity – a measurement of the amount of fluorochrome bound to a particle or cell; NOTE 206 1: Increasing intensity is reflected in a fluorescence signal appearing in a higher channel number. Under 207 appropriate conditions, fluorescence intensity can be related to the number of binding sites a cell has for a 208 particular fluorochrome-conjugated reagent; NOTE 2: Expressed as mean fluorescence intensity (MFI), 209 median fluorescence intensity (MdFI), or geometric mean fluorescence intensity (GeoMFI). 210 211 fluorescence minus one (FMO) – a staining tube that contains all but one of the antibodies/fluorophores 212 in a multicolor panel; it is used to properly interpret flow cytometry data and to identify and gate cells in 213 the context of data spread in a given panel; NOTE: For some applications, it is useful to omit more than 214 one antibody/fluorophore; when this is done, the staining tube is often referred to as FMx (fluorescence 215 minus X number of antibodies/fluorophores). 216 217 good clinical laboratory practice (GCLP) – a set of standards that provide guidance on implementing 218 good laboratory practice (GLP) and good clinical practice principles to the analysis of samples from a 219 clinical trial. 220 221 good laboratory practice (GLP) – a quality system concerned with the organizational process and the 222 conditions under which nonclinical health and environmental safety studies are planned, performed, 223 monitored, recorded, reported, and archived. 224 225 installation qualification (IQ) – a set of formal checks and records that confirms the equipment or process 226 and its components, including any integral hardware or software, were supplied as ordered and properly 227 installed in the laboratory or other environment; NOTE: IQ can be performed by the manufacturer’s 228 technical service engineer. 229 230 linearity – the ability (within a given range) to provide results that are directly proportional to the 231 concentration (amount) of the analyte in the test sample; NOTE 1: Linearity typically refers to overall 232 system response (ie, the final analytical answer rather than the raw instrument output); NOTE 2: The 233 linearity of a system is measured by testing levels of an analyte that are known by formulation or known 234 relative to each other (not necessarily known absolutely); when the system results are plotted against these 235 values, the degree to which the plotted curve conforms to a straight line is a measure of system linearity; 236 NOTE 3: Linearity tests are conducted over the system’s analytical measurement range to validate the 237 system’s overall response to varying levels of the analyte in question. 238 239 lymphosum – in an immunophenotyping assay for the major lymphocyte subsets (T cell, B cell, natural 240 killer [NK] cell), the lymphosum is the sum of % CD3+ (T cells), % CD19+ (B cells), and % CD3− (CD56+

241 ± CD16+) (NK cells); NOTE: In a medical laboratory, if the assay uses side-scatter/CD45 gating, the 242 lymphosum target range is within 90% to105%. 243 244

H62

5


measurand – quantity intended to be measured9; NOTE: see analyte. 245 246 molecules of equivalent soluble fluorochrome (MESF) – this concept indicates that a sample labeled 247 with a fluorochrome has the same fluorescence intensity as an equivalent number of molecules of the free 248 fluorochrome in a solution under the same environmental conditions. 249 250 operational qualification (OQ) – process and records to confirm that the equipment or process is 251 operational for its intended use and operation; NOTE: Operational qualification can be performed by the 252 manufacturer’s technical service engineer. 253 254 performance qualification (PQ) – process to confirm that the equipment or process will perform to the 255 organization’s own specified needs and intended use, producing acceptable results under normal operating 256 conditions; NOTE: Performance qualification needs to be performed by laboratory personnel before use in 257 testing and examinations. 258 259 precision – (of measurement) closeness of agreement between indications or measured quantity values 260 obtained by replicate measurements on the same or similar objects under specified conditions.9 261 262 qualitative – a characterization applied to laboratory tests that detect the presence or absence of a particular 263 analyte, constituent, or condition; NOTE: When used to describe a test, means a test that produces a result 264 that is descriptive rather than numerical. 265 266 quasiquantitative – the prefix “quasi-” means, having some resemblance to, or possession of 267 certain attributes of; NOTE: Quasiquantitative data are continuous numeric results expressed in terms of 268 the test sample, but data are not derived from a calibration curve or reference standard.12 269 270 reference material – material or substance, one or more of whose property values are sufficiently 271 homogeneous and well established to be used for the calibration of a measuring system, the assessment of 272 a measurement procedure, or for assigning values to materials13; NOTE 1: Reference material is a generic 273 term14; NOTE 2: Properties can be quantitative or qualitative, eg, identity of substances or species14; NOTE 274 3: Uses may include the calibration of a measurement system, assessment of a measurement procedure, 275 assigning values to other materials, and quality control14; NOTE 4: A reference material can only be used 276 for a single purpose in a given measurement. 277 278 ratiometric – describing any system in which an output is directly proportional to an input. 279 280 repeatability – (measurement) measurement precision under a set of repeatability conditions of 281 measurement9; NOTE 1: For the purpose of immunohematological testing, this is typically confined to 282 challenging the automated system’s ability to render the same interpretation consistently; NOTE 2: 283 Repeatability conditions include the same measurement procedure, the same observer, and the same 284 measuring instrument, all used under the same conditions, in the same location, and repeated over a short 285 period of time; NOTE 3: It is often termed in the laboratory as “within-assay,” “within-run,” “intra-assay,” 286 and “intrarun” precision. 287 288 reproducibility – (measurement) measurement precision under reproducibility conditions of 289 measurement9; NOTE 1: For the purpose of immunohematological testing, this is typically confined to 290 challenging the automated system’s ability to render the same interpretation consistently; NOTE 2: More 291 broadly defined as the agreement between repeated measurements on the same specimen under different 292 conditions; NOTE 3: The closeness of agreement between the results of the measurements of the same 293 measurand carried out under changed conditions of measurement; NOTE 4: Conditions include different 294 locations, operators, measuring systems, and replicate measurements on the same or similar objects9; 295 NOTE 5: The ability of a product, within lots and across multiple lots, to produce the same results, when 296 used in different settings (eg, different sites, different operators, different instruments); NOTE 6: Several 297

H62

6


types of reproducibility are often used in laboratory medicine: “between-assay,” “between-run,” 298 “interassay,” “inter-run,” and “interlaboratory” precision. The operative conditions should always be 299 specified. 300 301 resolution – smallest change in a quantity being measured that causes a perceptible change in the 302 corresponding indication9; NOTE: Resolution can depend on, for example, noise (internal or external) or 303 friction. It may also depend on the value of a quantity being measured.9 304 305 sample – one or more parts taken from a specimen.13 306 307 selectivity – (of a measuring system) property of a measuring system, used with a specified measurement 308 procedure, whereby it provides measured quantity values for one or more measurands such that the values 309 of each measurand are independent of other measurands or other quantities in the phenomenon, body, or 310 substance being investigated9; NOTE 1: In chemistry, the measured quantities often involve different 311 components in the system undergoing measurement and these quantities are not necessarily of the same 312 kind9; NOTE 2: In chemistry, selectivity of a measuring system is usually obtained for quantities with 313 selected components in concentrations within stated intervals9; NOTE 3: Selectivity is a concept similar to 314 analytical specificity.9 315 316 semiquantitative – tests that yield results in an approximate range of values (eg, trace, moderate); or a test 317 that has a dose-response gradient that may be included in the reported result, but for which no authoritative 318 calibration scale exists to determine inaccuracy and imprecision; NOTE 1: This definition includes tests 319 with subjective readout of quantification such as immunofluorescence–antinuclear antibody titers, and it 320 includes tests with an instrumental readout of quantification such as ELISA–antinuclear antibody when the 321 instrument scale cannot be referenced to an authoritative calibration scale; NOTE 2: An alternative usage 322 of this term is similar to that described above for quasiquantitative. 323 324 sensitivity – (of a measuring system) quotient of the change in an indication of a measuring system and the 325 corresponding change in a value of a quantity being measured9; NOTE 1: Sensitivity of a measuring system 326 can depend on the value of the quantity being measured; NOTE 2: The change considered in a value of a 327 quantity being measured must be large compared with the resolution9; NOTE 3: The sensitivity depends 328 on the imprecision of the measurements of the sample; NOTE 4: In qualitative testing, the test method’s 329 ability to obtain positive results in concordance with positive results obtained by the reference method. 330 331 specimen – discrete portion of a body fluid, breath, hair, or tissue taken for examination, study, or analysis 332 of one or more quantities or properties assumed to apply for the whole.13 333 334 spread – refers to the distribution of a given population for a given parameter and is measured as robust 335 standard deviation (rSD). The spread of a population may be due to natural biological variation, and/or to 336 other sources of measurement imprecision such as fluorescence spillover in a multicolor analysis. 337 338 standard deviation (SD) – a statistical measurement of the distance from the mean of a series of 339 measurements; NOTE: It is a measure of precision, reproducibility, or dispersion of a frequency 340 distribution. 341 342 standard reference material (SRM) – certified reference materials; NOTE 1: These are well-343 characterized materials produced in quantity to improve measurement science; NOTE 2: Standard 344 reference materials are certified for specific chemical or physical properties and are issued with certificates 345 that report the results of the characterization and indicate the intended use of the material. 346 347 titer – the concentration of a solution as determined by titration. The minimum volume of a solution needed 348 to reach the end point in a titration. The concentration of an antibody, as determined by finding the highest 349 dilution at which it is still able to maintain the intended result. 350

H62

7


traceability – property of the result of a measurement or the value of a standard whereby it can be related 351 to stated reference materials, usually national or International Standards, through an unbroken chain of 352 comparisons all having stated uncertainties.13 353 354 viability – refers to the percentage of a cellular population that are viable. 355 356 1.3.3 Abbreviations and Acronyms 357 358 % CV coefficient of variation expressed as a percentage 359 ABC antibodies bound per cell 360 ANOVA analysis of variance 361 APC allophycocyanin 362 BMMC bone marrow mononuclear cells 363 CD cluster of differentiation 364 CDx companion diagnostic 365 CE Conformité Européenne 366 CV coefficient of variation 367 DNA deoxyribonucleic acid 368 EQA external quality assessment 369 ERF equivalent reference fluorophores 370 FcR Fc receptor(s) 371 FCS flow cytometry standard 372 FDA US Food and Drug Administration 373 FFP fit-for-purpose 374 FITC fluorescein isothiocyanate 375 FMO fluorescence minus one 376 FSC forward scatter 377 HLA human leukocyte antigen 378 IQ installation qualification 379 ISO International Organization for Standardization 380 IVD in vitro diagnostic 381 LED light-emitting diode 382 LDT laboratory-developed test 383 LLOQ lower limit of quantification 384 LOB limit of blank 385 LOD limit of detection 386 MFI mean fluorescence intensity 387 MdFI median fluorescence intensity 388 MESF molecules of equivalent soluble fluorochrome 389 MRD minimal residual disease 390 NIST National Institute of Standards and Technology 391 OQ operational qualification 392 PBMC peripheral blood mononuclear cell(s) 393 PD pharmacodynamic(s) 394 PE phycoerythrin 395 PK pharmacokinetic(s) 396 PMN polymorphonuclear neutrophils 397 PMT photomultiplier tube 398 PNH paroxysmal nocturnal hemoglobinuria 399 PQ performance qualification 400 PT proficiency testing 401 QA quality assurance 402 QC quality control 403

H62

8


QMS quality management system 404 QSE quality system essential 405 rSD robust standard deviation 406 RNA ribonucleic acid 407 RO receptor occupancy 408 SD standard deviation 409 SI staining index 410 SOP standard operating procedure(s) 411 SRM standard reference material 412 SSC side scatter 413 SSM spillover spread matrix 414 S/N signal-to-noise 415 SW staining window 416 ULOQ upper limit of quantification 417 418

H62

9


Chapter 2: Path of Workflow and Quality System Essentials 419 420

This chapter includes: A flow chart that depicts the recommended steps in the flow cytometry assay validation process Overview of selected quality system essentials (QSEs) Information on the following QSEs as they relate to flow cytometry:

- Personnel Management - Equipment Management - Process Management - Documents and Records Management

421 2.1 Process Flow Chart 422 423 Figure 1 depicts the recommended flow cytometry assay validation process. These steps include IQ (see 424 Chapter 4), assay development and optimization (see Chapter 5), validation planning and implementation 425 (see Subchapters 6.1 and 6.2), and QC verification (see Subchapter 7.3.3). Subchapter 7.3.5 describes 426 sample processing and acquisition. 427 428

H62

10


Instrument is qualified, set up, and standardized

Assay is developed and optimized

Validation is planned

Performance qualifications are conducted

Compensation is established

Validation is implemented

Instrument is aligned and calibrated

QC is verified

Sample is processed and acquired

Chapter 4

Subchapter 4.2

Subchapter 4.4

Chapter 5

Subchapter 6.1

Subchapter 6.2

Subchapters 7.1.1, 7.1.2

Subchapter 7.3.3

Subchapter 7.3.5

Need for a flow cytometric assay is identified

Postexamination phase is started Subchapter 7.4

Flow cytometric assay is implemented

429 430 Abbreviation: QC, quality control. 431 a Five basic symbols are used in process flow charts: oval (signifies the beginning or end of a process), arrow (connects process 432 activities), box (designates process activities), diamond (includes a question with alternative “Yes” and “No” responses), pentagon 433 (signifies another process). 434 Figure 1. Flow Cytometry Assay Validation Processesa 435 436

H62

11


2.2 Introduction to Quality System Essentials 437 438 Regulated laboratories and manufacturers of in vitro diagnostic (IVD) devices and reagents utilize a 439 foundation of a quality management system (QMS) to reduce or eliminate errors, meet customer 440 expectations, improve operational efficiency, meet regulatory and accreditation assessments, and meet 441 institutional quality objectives. These processes are aligned with the twelve quality building blocks or 442 QSEs: 443 444 Organization and Leadership 445 Process Management 446 Assessments 447 Continual Improvement 448 Customer Focus 449 Document and Records Management 450 Equipment Management 451 Facilities and Safety Management 452 Information Management 453 Nonconforming Event Management 454 Personnel Management 455 Supplier and Inventory Management 456 457 QMS and the corresponding QSEs are described in detail in CLSI document QMS01.15 458 459 Flow cytometry is one of the many technologies used in laboratory testing but is unique in that it analyzes 460 individual cells instead of soluble analytes; therefore, some aspects of the QMS may be unique to this 461 technology. This chapter provides information on four of the 12 QSEs that may need different or unique 462 information for flow cytometry, namely Personnel Management, Equipment Management, Process 463 Management, and Documents and Records Management. The details provided for each topic are not meant 464 to be all-inclusive; they are intended to add technology-specific information that is not covered in 465 CLSI document QMS01.15 466 467 2.3 Personnel Management 468 469 The QSE of Personnel Management is about hiring and retaining qualified, well-trained, competent 470 laboratorians to perform and manage the activities within the flow cytometry laboratory to include sample 471 processing, data analysis, and interpretation. Depending on the scope of the laboratory and the regulatory 472 requirements, a documented job description may be necessary. The qualifications of the personnel 473 necessary to undertake the role should be included in this job description. The level of education, specific 474 credentials or certifications needed, formal training, and essential competencies should also be outlined in 475 the description. Additionally, the specific duties and responsibilities for each job level should be included 476 in the job description. 477 478 The laboratory must ensure that each new employee is familiar with the laboratory policies, processes, and 479 procedures in addition to any applicable regulatory requirements. This should facilitate standardization of 480 processes across staff to ensure alignment and conformity. Detailed and comprehensive training of 481 laboratorians is essential for flow cytometric methods due to the highly complex nature of the technology. 482 This training may include learning through formal education, passing a certification examination, and on-483 the-job training. A thorough and documented training program should be in place with minimum 484 expectations of performance, which is described in detail in CLSI document QMS03.16,17 485 486 Flow cytometry technology continues to evolve and, as such, necessitates continued professional 487 development for laboratorians through ongoing continuing education and evaluation by attending relevant 488

H62

12


congresses. Furthermore, this continuing education and professional development should be coupled with 489 ongoing competency assessments of the individual and should encompass (at a minimum) direct 490 observation, assessing the individual’s problem-solving skills, and conducting records review. Specific 491 performance criteria should be established, and failure to maintain a laboratory-defined competency level 492 must result in remedial training and corrective action, and this information should be documented in the 493 individual’s competency record.16 This periodic evaluation of laboratorians is not only essential for ongoing 494 staff alignment, but it also promotes communication, contributes to employee professional development, 495 and meets regulatory compliance. 496 497 2.4 Equipment Management 498 499 Flow cytometry includes the use of complex and user-configurable equipment. It is critical to define the 500 intended use of the equipment before procurement. Once procured, conducting a detailed and documented 501 qualification of all equipment is considered a best practice and a regulatory requirement. The process of 502 setting up the equipment should include a formal installation qualification (IQ), operational qualification 503 (OQ), and performance qualification (PQ). These topics are covered in detail in Chapter 4. Calibration and 504 verification of the equipment are necessary to document ongoing operational standardization together with 505 a well-designed and documented equipment maintenance program. All qualification, calibration, and 506 maintenance records should be documented and stored with the equipment. Any equipment that is 507 decommissioned should be documented appropriately. 508 509 2.5 Process Management 510 511 As defined in CLSI document QMS01,15 processes for preexamination, examination, and postexamination 512 must be designed with the intended use and level of regulatory compliance in mind. These processes include 513 specimen collection and handling; specimen transportation to the laboratory; sample preparation and 514 analysis; result interpretation; and sample storage, and they should be based on process mapping and 515 standard operating procedures (SOPs). The documentation should follow the entire path of workflow. See 516 Chapters 5 and 7 for more information. 517 518 In addition to mapping the current processes of flow cytometric assays, it is critical to establish the planning, 519 design, and validation process of the test before its implementation. Validation is one of the main areas of 520 focus of this guideline. Cell-based assays have unique challenges because of the nature (ie, live cells) and 521 limited stability of the sample. Defining acceptability criteria and appropriate sample size for each assay 522 performance specification are vital components of cell-based validation. See Chapter 6 for more 523 information. 524 525 Once an assay has been validated to include a comprehensive QC process, it is necessary to maintain and 526 monitor drift from the performance specifications established at validation. Because flow cytometric cell-527 based assays have unique challenges, it is necessary to ensure that a process control has similar properties 528 to the cells being measured. Lastly, processes that evolve throughout the lifecycle of the assay and changes 529 should be tracked and documented through a change control process. This documentation should include 530 the reason(s) for the change, the date the change(s) is implemented, and a description of the method 531 modification as well as evaluation after the change has been implemented. 532 533

H62

13


2.6 Documents and Records Management 534 535 Documentation and records are necessary to re-create the procedures, ensure traceability and compliance, 536 and retain records of all samples, reagents, and staff associated with the processes. For flow cytometry, this 537 also includes the analysis layouts/templates (raw data in addition to gated data), IQ plans, and reports 538 together with validation plans and summaries. More details about the recommended content for validation 539 documentation are provided in Chapter 6. 540 541 Documents and records can be paper-based or electronic. A complete document management system is 542 necessary to ensure traceability. Each process, SOP, and attached form(s) must have a unique identification 543 code. The documentation must include creation and review dates, names and signatures of those approving 544 the document (together with any revisions), and whether the procedure was archived and, if so, on which 545 date. According to QSE principles, laboratory processes and procedures require the completion of 546 respective forms that become permanent records. More details about the requirements for Documents and 547 Records Management are included in CLSI document QMS01.15 548 549 550

H62

14


Chapter 3: Fit-for-Purpose Approach to Analytical Method Validation for Flow 551 Cytometric Methods 552 553

This chapter includes: Considerations for cell-based assay validation Application of standard validation parameters for flow cytometric methods Regulatory requirements for validations

554 Analytical method validation is the confirmation, by examination and the provision of objective evidence, 555 that the particular specifications for an intended use are fulfilled. Method validation is not necessarily a 556 one-time activity but a process that is revised over the life cycle of the assay. Using a fit-for-purpose (FFP), 557 or assay context of use approach, the initial validation should meet the requirements of the intended use of 558 the data and the regulatory requirements associated with this use; because these aspects change (laboratory-559 initiated assay revision), supplemental validation studies should be performed to confirm test suitability. 560 The scope of the supplemental validation depends on the extent of the change.12 561 562 Clinical validation or biomarker qualification demonstrates whether the validated analytical method has 563 clinical utility in that the test results are of benefit to the patient. Clinical validation and biomarker 564 qualification are beyond the scope of this guideline and are not covered. Accuracy, linearity, 565 specificity/selectivity, sensitivity, precision, stability, and reference intervals are parameters that can be 566 evaluated during analytical method validation. Because of the lack of a reference standard with true values, 567 the complexity of cellular measurements, and the type of data generated, some validation parameters cannot 568 always be evaluated in cell-based assays. Specificity, precision, sensitivity, reference ranges, and stability 569 are the parameters that can be validated for cell-based assays.18 570 571 3.1 Considerations for the Validation of Flow Cytometric Methods 572 573 Several factors contribute to the challenges and complexity associated with the validation of cell-based 574 methods by flow cytometry: 575 576 The complexity of cellular measurands includes sample matrices and disease state 577 The lack of reference materials 578 The fact that data are not derived from a calibration curve 579 580 The lack of reference materials and the absence of calibration curves result in the inability to validate 581 accuracy and linearity in the traditional sense. This point has become one of the most difficult and 582 controversial aspects of cell-based assay validation. The following subchapters elaborate specifically on 583 how reference materials, calibration curves, and bioanalytical data categories influence the ability to 584 validate accuracy and linearity parameters. 585 586 3.1.1 Bioanalytical Data Categories 587 588 The type of bioanalytical data generated by a given method dictates which validation parameters can and 589 cannot be evaluated. Four categories of bioanalytical data have been described (ie, definitive quantitative, 590 relative quantitative, quasiquantitative, and qualitative).12 Because of a lack of reference standards and the 591 nature of the determinations, many flow cytometric assays fall into the quasiquantitative and qualitative 592 categories. 593

H62

15


3.1.1.1 Definitive Quantitative Data 594 595 Definitive quantitative data are used to determine the absolute quantitative values for unknown samples 596 using well-defined reference standards that are fully representative of the endogenous measurand. Examples 597 of assays that generate definitive quantitative data include pharmacokinetic (PK) and liquid 598 chromatography–mass spectrometry assays. Although common in other areas of the medical laboratory, 599 most flow cytometric assays are not considered definitive quantitative because of the lack of true 600 traceability to a primary standard.18,19,20 601 602 3.1.1.2 Relative Quantitative Data 603 604 Relative quantitative data are derived from assays in which the calibration curve or reference standards are 605 not fully representative of the endogenous measurand. Relative quantitative data should be used to estimate 606 the value of the measurand and track temporal changes but not to determine absolute quantitative values. 607 The key difference between definitive and relative quantitative data are the type of calibrators or standards 608 and lack of traceability to primary standards. Quantitation of relative antigen expression levels using 609 fluorescence quantitation beads are considered relative quantitative. Although a calibrator is used in the 610 form of antigen binding beads or with beads having predefined levels of fluorescent molecules, it is not 611 fully representative of the cellular measurand. Examples include flow cytometric assays for CD64 612 expression on myeloid leukocytes and human leukocyte antigen, DR isotype (HLA-DR) expression on 613 monocytes, both expressed in terms of the fluorophore detection relative to stabilized bead calibrators.21 614 615 3.1.1.3 Quasiquantitative Data 616 617 Quasiquantitative data are derived from assays that lack a calibration curve and reference standards. The 618 data are numeric and expressed in terms that are characteristic of the test samples. Similar to relative 619 quantitative data, quasiquantitative data can be used to estimate the values of the measurand in the samples 620 and track temporal changes but not to determine absolute quantitative values. Without reference standards, 621 accuracy for quasiquantitative assays cannot be directly assessed. The term “semiquantitative” has also 622 been used to describe data generated from flow cytometric methods in a manner synonymous with the term 623 quasiquantitative, but this term is also used to report ordinal data, as described in Subchapter 3.1.1.4.22,23 624 Thus, in this guideline, the term quasiquantitative is used to avoid confusion. 625 626 Flow cytometric methods, in which the readout is the relative percentage of a given population of cells or 627 the cellular concentration (number of cells/unit volume), are considered quasiquantitative methods, eg, 628 enumeration of CD4+ T cells or CD34 progenitor cells. 629 630 The enumeration of CD19+ B cells provides a clear example of quasiquantitative data. CD19+ B cells are 631 typically evaluated by staining whole blood with at least CD19 and CD45. The events satisfying the 632 definition of CD19+ lymphocytes (low side scatter [SSC], low forward scatter [FSC], high levels of CD45 633 expression, CD19+) are reported as the relative percentage of events satisfying the definition of lymphocytes 634 (low SSC, low FSC, high levels of CD45 expression). During data analysis, or gating, each decision point 635 in the process that identifies the CD19+ B cells is based on properties of the test sample. Properties such as 636 the expression level of each antigen and the light scatter are numeric; however, calibrators for antigen 637 expression are not used, and the final reported result is not derived from a standard curve. Control materials 638 and reference standards are discussed in depth below. 639 640

H62

16


3.1.1.4 Qualitative Data 641 642 Qualitative data are descriptive as opposed to numeric and are used to determine characteristics of the test 643 sample. Data are reported in either nominal (eg, yes/no) or ordinal (eg, +, ++, +++) formats. Examples of 644 qualitative data include leukemia/lymphoma immunophenotyping by flow cytometry and genetic markers. 645 The ordinal data format is also sometimes referred to as semiquantitative.23 646 647 3.1.2 Reference Standards for Flow Cytometry 648 649 There is an ever-increasing number of fluorophores, lasers, and types of instruments in the field of flow 650 cytometry. Furthermore, biological research is progressing so rapidly that the number of cellular antigens 651 and novel cell subsets to measure are continually expanding as well. These factors are major contributors 652 to the challenges in validating flow cytometric methods for cellular measurands. Given the increase in the 653 types of reagents and cellular targets of interest, it is not surprising that there are very few suitable reference 654 standard materials. As such, significant progress has been made on this front, and there are now certified 655 reference standards for a limited number of fluorophores and cellular targets. 656 657 3.1.2.1 Equivalent Number of Reference Fluorophores 658 659 Described in more detail in Chapter 4, the National Institute of Standards and Technology (NIST) has 660 created fluorescence calibration beads with traceable numbers of equivalent reference fluorophores 661 (ERF).24 These beads are primarily intended for use by manufacturers to calibrate their fluorescent bead 662 products. The use of fluorescent beads with ERF calibration allows for accuracy, linearity, and 663 standardization to be established for measurements reported as fluorescent intensity. 664 665 3.1.2.2 Biological Standards 666 667 There are a variety of control materials for flow cytometry in the form of preserved whole blood, lyophilized 668 peripheral blood mononuclear cells (PBMCs), and dried leukocytes.25 These materials are very valuable as 669 process controls; however, they cannot be considered true reference standards, because the target ranges 670 provided are very broad and the target values derived by the manufacturer are not certified by an 671 independent metrology organization. Also, target values are not frequently provided for every subset 672 reported in a given method. For example, for an assay measuring B-cell subsets, the control material is 673 provided with target values for CD19+ and CD20+ cells but not B-cell subsets, such as regulatory B cells 674 (CD45bright, SSClow, CD19+, CD24bright, CD38bright). 675 676 There are now a limited number of biological reference materials for flow cytometry available with 677 provided certified target values and the documented associated measurement uncertainty, such as CD4 and 678 CD34 biological reference materials created by a collaboration between NIST and the National Institute of 679 Biological Standards and Control under the World Health Organization. These reference materials are 680 limited in scope and are calibrated for cell counts and not fluorescence intensity. Regardless, the creation 681 of a certified biological standard is a major advancement in flow cytometry and may be used to support 682 method validation of assays reporting CD4+ and CD34+ cell counts. 683 684 3.2 Application of Standard Validation Parameters for Flow Cytometric Methods 685 686 3.2.1 Accuracy/Trueness 687 688 According to the International Organization for Standardization (ISO), the accuracy of a method 689 incorporates both trueness and precision. Trueness is defined as the closeness of the agreement between the 690 average value obtained from a large series of results of measurements and a true value. By this definition, 691 accuracy/trueness cannot be established for most flow cytometric methods. The primary reason is centered 692 on the fact that most flow cytometric methods generate quasiquantitative or qualitative data. As described 693

H62

17


above in more detail, assays that report quasiquantitative data provide a continual numeric result but lack a 694 calibration curve or standard reference material (SRM). Biological reference material is available for a 695 limited number of cell types, but a true cellular SRM with appropriate biological matrixes is not available 696 for most cell subsets reported from multiparameter flow cytometric methods. As a result, accuracy is often 697 not included in FFP validations. For cases in which an assessment of accuracy is a regulatory requirement, 698 novel approaches may be required, as described in Subchapter 6.1. 699 700 3.2.2 Linearity 701 702 There are a number of different ways to define linearity, including but not limited to the degree to which 703 the calibration curve approximates a straight line or the results are proportional to the concentration of the 704 measurand. Linearity can be demonstrated when the analytical method contains a calibration curve and 705 generates definitive quantitative or relative quantitative data. Conversely, true linearity cannot be 706 demonstrated in the absence of a calibration curve as with quasiquantitative or qualitative data. Most flow 707 cytometric methods generate data that are considered quasiquantitative; therefore, linearity as previously 708 defined cannot be covered. Linearity assessment is not applicable for assays generating qualitative data, 709 such as immunophenotypic characterization of leukemia and lymphoma.20,26 710 711 In cases in which the population of interest is not present in either the patient or healthy populations, such 712 as in paroxysmal nocturnal hemoglobinuria (PNH) clone detection or leukemia/lymphoma 713 immunophenotyping assays, laboratories find it useful to plot the data generated from serial dilutions of a 714 spiked positive sample into a negative sample to determine assay sensitivity. This assay sensitivity should 715 not be confused with true linearity. 716 717 Receptor occupancy (RO) and receptor modulation assays represent a unique category of flow cytometric 718 methods that are increasingly common in drug development. This category of assay is unique, because the 719 intent is to describe the relationship between a drug (typically a protein-based therapeutic agent) and its 720 target (typically a cell surface protein). The analytical measurement range and linearity must be evaluated 721 for RO and receptor modulation assays.27 Specific details on assessment of linearity for RO assays are 722 covered in Chapter 6. 723 724 3.2.3 Specificity and Selectivity 725 726 The definition of analytical specificity is the ability of an assay to measure the intended measurand while 727 avoiding any cross-reactivity that leads to an incorrect assessment. For flow cytometric methods, this 728 translates to the ability of the assay to measure specifically the intended cellular populations and antigens 729 of interest, excluding events present in a particular gate that are the result of compensation errors, conjugate 730 degradation, cellular doublets, or contamination from another cell type. These aspects of specificity 731 assessment are accomplished during panel design and optimization, as described in Chapter 5. 732 733 Selectivity refers to the ability to measure the measurand in the presence of unrelated compounds in the 734 matrix, often referred to as matrix effects. In flow cytometry, assay selectivity can be affected in hemolyzed, 735 lipemic, or clotted samples.2,3,28,29 Additionally, interference can come from protein-based immunotherapies 736 targeting cell surface molecules (eg, anti-CD2, anti-CD19, anti-CD20, anti-CD38). These aspects should 737 be evaluated during panel design and optimization, as described in Chapter 5. Either noncompeting 738 detection antibodies must be identified or drug-specific antigens must be eliminated from the panel. 739 740 When interfering substances such as abnormal (or abnormally represented) cell populations or soluble 741 factors are expected to be encountered in disease-state clinical specimens, their effect on selectivity should 742 be evaluated either during the initial analytical method validation or in a supplemental validation. Examples 743 include: 744 745

H62

18


Disruption of internal gating reference populations due to differences in the prevalence of cell 746 populations between those used in testing vs those used in the assay development and initial validation. 747

748 The presence of a soluble target that specifically competes with a monoclonal antibody in the assay and 749

can affect the signal intensity independently from the cellular target’s representation. 750 751 Nonspecific interference associated with soluble components that are prevalent within certain disease 752

states (eg, rheumatoid factor, immune complex). 753 754 For qualitative tests, specificity relates more to the clinical utility or predictive value of the method rather 755 than analytical performance (see CLSI document EP1223). In this situation, specificity is determined 756 through concordance with a comparative method, such as another flow cytometric assay or 757 morphology/immunohistochemistry. 758 759 3.2.4 Sensitivity and Upper Limit of Quantification 760 761 Sensitivity can be determined in several different ways depending on the type of data generated. For 762 quasiquantitative assays, sensitivity often refers to the ability to detect and quantitate low values, such as 763 CD34+ progenitor cells or minimal residual disease (MRD). For these assays, analytical sensitivity includes 764 both the limit of detection (LOD)—the ability of the assay to distinguish signal from background—and the 765 limit of blank (LOB)—the highest signal expected in absence of the measurand. Functional sensitivity refers 766 to the lower limit of quantification (LLOQ)— the ability to precisely and accurately measure low amounts 767 of the measurand (population frequency, fluorescence intensity, or absolute counts). Accuracy is not taken 768 into consideration when establishing the LLOQ for most flow cytometric methods due to the 769 aforementioned challenges in establishing accuracy. The design of high-sensitivity assays is discussed in 770 Chapter 5. 771 772 The assessment of sensitivity for population frequency is of particular importance in the following 773 scenarios: 774 775 Monitoring rare cell populations, such as CD34+ peripheral stem cells 776 Identifying the presence of MRD in leukemia/lymphoma immunophenotyping 777 Monitoring treatment with cell-depleting therapeutics 778 779 The assessment of sensitivity for fluorescence intensity is of value in the following scenarios: 780 781 Monitoring dimly expressed antigens 782 783 Assessing poorly separated negative and positive events 784 785 Detecting small changes in intensity (loss or gain) of antigen expression as is required in leukemia and 786

lymphoma immunophenotyping 787 788 The upper limit of quantification (ULOQ) should be handled during the flow cytometric assay development. 789 Antibodies are added in excess; therefore, the possibility of depleting the antibody is not generally an issue. 790 Similarly, cell concentrations and antibody titers should also be established during assay development. This 791 topic is covered in detail in Chapter 5. ULOQ may be most relevant in cell populations that express high 792 levels of a particular antigen (eg, CD38 on plasma cells). For qualitative tests, sensitivity relates more to 793 the clinical utility or predictive value of the method rather than analytical performance (see CLSI document 794 EP1223). In this situation, sensitivity is determined through concordance with a predicate method, such as 795 another flow cytometric assay or morphology/immunohistochemistry. 796

H62

19


3.2.5 Precision 797 798 The precision of an analytical method describes the closeness of agreement between individual measured 799 values when the procedure is repeatedly applied. Measurement variability is the result of contributions from 800 a variety of sources that may include the following30: 801 802 Day 803 Run 804 Reagent lot 805 Operator 806 Instrument 807 Laboratory 808 809 The particular sources of analytical variability and the significance of their effects vary between assays. In 810 addition to understanding analytical variability, understanding the intra- and intersubject variability is 811 important when interpreting test results.31,32 Evaluating intra- and intersubject (biological) variability is 812 more related to the clinical validation than the analytical method validation and is beyond the scope of this 813 guideline. 814 815 CLSI document EP0530 states that there are three types of precision (ie, repeatability, intermediate 816 precision, and reproducibility) that should be evaluated. Not all of these sources may be relevant for a given 817 assay; however, one should be wary of dismissing a source as irrelevant. The specific challenges in assessing 818 precision for cell-based methods arise from limited sample stability, limited sample volume, and difficulty 819 in obtaining samples with different levels of each of the cellular subsets typically reported.18,32 High reagent 820 and labor costs add to the challenges. These factors limit the number of replicates and the number of days 821 of testing that can be evaluated. Thus, a factorial design strategy is recommended, as described in Chapter 822 6. In a factorial design strategy, multiple factors are evaluated within a single experimental design rather 823 than the evaluation of each factor in independent experiments.33 824 825 Repeatability 826

– Repeatability is traditionally referred to as intra-assay (within-run precision) and reflects the 827 variability among replicate measurements of a sample under experimental conditions held as 828 constant as possible. 829

830 Within-laboratory precision or intermediate precision 831

– Intermediate precision is often referred to as within-laboratory precision. Intermediate precision 832 builds on repeatability by incorporating run-to-run (interassay), between run (day-to-day), as well 833 as operator-to-operator (interoperator) variability on a single instrument within a single laboratory. 834 Day-to-day intermediate precision may not always be applicable for flow cytometry assays due to 835 low sample stability. 836

837 Reproducibility 838

– Reproducibility incorporates laboratory-to-laboratory variability and/or instrument-to-instrument 839 variability (interinstrument variability) on top of the source contributing to “within-laboratory 840 precision.” As such, reproducibility is covered for a multisite laboratory analysis. 841

842

H62

20


3.2.6 Stability 843 844 3.2.6.1 Specimen Stability 845 846 Stability is defined by ISO as the capability of a sample material to retain the initial property of a measurand 847 constituent for a period of time within specified limits when the sample is stored under defined conditions.34 848 Instability is present when there are significant and relevant changes in one or more of these measurements. 849 Stability assessments should be applied to both specimens and reagents. The specimen type and 850 anticoagulant should be selected during the assay-optimization phase, as described in Chapter 5, and they 851 should have a significant effect on specimen stability. Postcollection handling and specimen transport 852 conditions (ie, mode and temperature) also affect specimen stability. Specimen stability must be validated 853 using the final specimen type(s), anticoagulant(s) and storage conditions. If postcollection sample 854 processing procedures, such as PBMC isolation or cryopreservation of cells occur, the effect of these 855 procedures must be included in the stability-validation plan. 856 857 3.2.6.2 Processed-Sample Stability 858 859 Processed-sample stability (ie, the time between staining and acquisition) must also be validated. The time 860 and storage conditions for postprocessing stability evaluation should reflect the actual conditions likely to 861 occur in the testing laboratory. 862 863 3.2.7 Assay Carryover 864 865 Assessment of carryover is a normal component of instrument PQ that is described in more detail in 866 Subchapter 4.2.5. Additionally, the potential for carryover should be evaluated in the development of 867 individual assays and demonstrated to be adequately controlled during the assay validation. Evidence of 868 risk or lack of risk from carryover can be sensibly argued based on data collected from the instrument PQ 869 and maintenance. When no substantial risk is evident, the PQ data and procedures should be cited in support 870 of the declaration. Carryover can vary between instruments; therefore, adequacy of measures to control 871 carryover should be determined for each instrument on which the sample for that assay will be acquired. 872 873 3.2.8 Reference Intervals 874 875 Reference intervals (reference ranges/normal ranges) describe the limits of the distribution for a measurand 876 within a selected population. Although they are essential in the interpretation of most medical laboratory 877 results, they are not always mandatory for flow cytometric methods. For example, if the flow cytometric 878 method is being used as diagnostic for abnormal populations normally absent from healthy donors or a 879 pharmacodynamic (PD) biomarker in a clinical trial, reference intervals may not be needed. The process 880 for establishing reference intervals for a laboratory-developed test (LDT), when applicable or to verify the 881 manufacturer’s ranges for IVD/Conformité Européenne (CE), assays should follow existing guidance 882 documents, such as CLSI document EP28.35 883 884 3.3 Regulatory Requirements for Validation 885 886 Flow cytometric assays have a variety of intended purposes that include basic research, drug development, 887 diagnostics, or a companion diagnostic (CDx), in addition to a variety of other applications, such as marine 888 biology and bacterial testing. According to CLSI document EP23,36 in the medical laboratory testing 889 environment, various applications of flow cytometric assays are associated with different risk levels to 890 human subjects or patients (low, moderate, or high) and regulatory requirements for validation or 891 verification.37 A risk assessment must be performed for each flow cytometric assay by considering the 892 intended use of the assay, how the assay might lead to human subject or patient harm through misuse, and 893 the ability to mitigate this risk. Manufacturers of commercial assays to be used as IVDs are responsible for 894 conducting a risk assessment. Thus, if the laboratory adopts such assays without modification, it does not 895

H62

21


need to repeat the risk assessment. For LDTs or modified assays, the laboratory must assume responsibility 896 for performing the risk assessment and all associated validation studies. 897 898 A research assay designed by a laboratory is considered low clinical risk because the data generated by this 899 assay does not have an effect on clinical decision outcomes and has no associated risk to patients (see Table 900 2). Custom reagents and reagent combinations are considered low risk by the manufacturer, but a laboratory 901 using these products is responsible for determining the risk associated with the intended purpose, as well 902 as performing the indicated level of validation studies. Assays used as aids to diagnosis, such as 903 leukemia/lymphoma immunophenotyping assays, have moderate clinical risk, because the results are not 904 used alone when determining a diagnosis but rather are used together with other information. Flow 905 cytometric assays used alone to establish a diagnosis, to determine entry into a clinical trial, or for selection 906 of the optimal therapy (ie, complementary diagnostic or CDx) are associated with a high clinical risk to 907 patients due to the possibility of withholding the appropriate therapy. Table 2 highlights the intended use 908 of different assays and their associated risks. 909 910 Table 2. Assay Risk Categories 911

Clinical Risk Potential Uses for Data Low Basic research assay

Drug discovery assay Exploratory end points in clinical trials

Moderate LDT used as an aid to diagnosis Secondary end points in clinical trials

High Primary end point in clinical development Clinical trial biomarker assay (enrollment criteria) Complementary diagnostic Combination product/CDx

Abbreviations: CDx, companion diagnostic; LDT, laboratory-developed test. 912 913 Assays manufactured as IVD devices fall under applicable regulatory and accreditation requirements and 914 relevant international standards, which take into account available information, such as the generally 915 accepted state-of-the-art and known stakeholder concerns. For example, in the United States, commercial 916 manufacturers must meet US Food and Drug Administration (FDA) Quality System Regulation for current 917 good manufacturing practices that are consistent with international standards for QMSs of medical 918 devices.38,39 Submission of an assay to the FDA should be made either through a Premarket Approval, which 919 requires demonstration of safety and effectiveness, or a 510(k), which necessitates demonstration that it is 920 at least as safe and effective as a previously cleared (predicate) assay or device. In the European Union, 921 commercial manufacturers must comply with European Parliament regulations on IVD medical devices.40 922 Through the submission and review process for regulatory approval, or clearance, manufacturers have 923 already assessed risk by performing appropriate validation studies; therefore, it is necessary for a medical 924 laboratory that is adopting an IVD assay to perform a verification of assay performance but not method 925 validation. If the laboratory modifies the assay or deviates in any way from the protocol provided by the 926 manufacturer, the assay is then considered an LDT and additional validation studies are required. The 927 laboratory must assess which components of the assay might be affected by the modification and determine 928 which additional validation studies must be performed. For example, a change in anticoagulant can have a 929 major effect on the performance of the assay, and a significant additional validation will need to be 930 performed. 931 932 933

H62

22


During the life cycle of an assay, the intended use of the data might change. For instance, a research assay 934 might be adopted into clinical practice. In this case, the clinical risk changes from low to moderate or high, 935 and additional validation is required. A second example is if a laboratory modifies an IVD assay and it 936 becomes an LDT. It is prudent for laboratories to anticipate changes in the intended purpose of an assay to 937 predict the need for future validation studies and avoid repeating prior validation studies. When considering 938 a possible future submission of an assay for regulatory approval, laboratories should seek guidance from 939 the FDA through the presubmission process. The type of data generated by the method and intended use/risk 940 should be taken into consideration when a validation plan is designed (see Chapter 6 for more detailed 941 information). Additional information is also included in Appendix A. 942 943

944

H62

23


Chapter 4: Instrument Qualification, Setup, and Standardization 945 946

This chapter includes information on: IQ and OQ PQ Cross-instrument, cross-site standardization Compensation Longitudinal performance Qualification and verification of instruments for the intended purpose

947 Qualification of flow cytometers ensures that the output generated on these instruments is reproducible and 948 precise. One of the most relevant elements of IQ includes testing to verify that the equipment is installed 949 appropriately and performs as intended. This chapter, which covers the fundamental principles for IQ, 950 includes establishing controlled procedures for installation, maintenance, calibration, cross-instrument 951 standardization, and longitudinal performance monitoring. For fully automated IVD instrument systems, 952 the vendor dictates the IQ, OQ, and PQ. For such systems, subsequent performance monitoring should be 953 performed according to the vendor-specified QC system using qualified QC material cleared by a regulatory 954 agency (eg, FDA, CE), which can include beads or stabilized cellular controls. 955 956 4.1 Installation Qualification and Operational Qualification 957 958 The purpose of IQ and OQ is to provide evidence that instrument hardware and associated software are 959 installed and functioning per the manufacturer’s specifications and the user requirements.41 The following 960 subchapters describe some of the critical parameters that should be managed during IQ/OQ. Examples of 961 IQ and OQ parameters are shown in Tables 3 and 4, respectively. 962 963 IQ 964

– On installation of the cytometer, environmental and utility requirements such as space, temperature, 965 and electrical components should be verified against the vendor’s specifications. Instrument 966 vendors frequently offer IQ/OQ services, although laboratorians may perform the qualifications. 967 On instrument installation, the manufacturer should provide a Certificate of Compliance to the 968 laboratory. The certificate is proof that the instrument was qualified and passed all manufacturer 969 specifications. 970

971 OQ 972

– During the OQ phase, tests should be developed and implemented with acceptance criteria to verify 973 that the instrument functions appropriately and adheres to the vendor’s specifications and user 974 requirements. The qualification should include positive outcomes and emphasize an adverse 975 outcome assessment for the system. Additionally, the document should contain any specific actions 976 stated in the risk assessment that attempt to reduce risk. 977

978

H62

24


Table 3. Examples of IQ Parameters and Associated Testing and Documentation 979 Installation Parameter Pass/Fail Criteria Documentation Notes

Environment Do benchtop and associated laboratory space meet vendor specifications?

A checklist with vendor requirements, a conclusive notation of pass/fail, and initials and date should be used as documentation.

Space requirements for instrument/computer footprint and additional clearance for future maintenance should be considered.

Utilities Do the temperature and humidity of laboratory space meet vendor specifications?


Equipment used to perform verification should be documented in a report appended to the checklist.

Electrical Do the electrical conditions meet vendor specifications?


Equipment used to perform verification should be documented in a report appended to the checklist.

Hardware Has the installation of all components been verified?

Instrument specifications (model, serial number, manufacture date) should be documented.

All associated components, if any, including automated sample acquisition modules, uninterrupted power supplies, etc., should be documented.

Computer system and associated software

Are the workstation and associated software installed?

Workstation specifications (computer model, serial number, software version) should be documented.

External hard drives used for data backup and printers, identified as non-networked or networked, should be included in documentation.

Does the computer and instrument communication meet vendor specifications?

A checklist with vendor requirements, a conclusive notation of pass/fail, and the user’s initials and date should be used as documentation.

Minimum verification process should make the user log in and run the vendor-specified QC sample.

Abbreviations: IQ, installation qualification; QC, quality control. 980 981

H62

25


Table 4. Examples of OQ Parameters and Associated Testing and Documentation 982 Operational Parameter Pass/Fail Criteria Documentation Notes

Software functionality

Have the automated system functions (start-up, QC) been performed?

Screenshots that show the software functioning correctly or a report with a conclusive notation of pass/fail and the user’s initials and date should be used as documentation.

Automated maintenance procedures should be documented.

Operating system alerts

Has the system been stressed to demonstrate that it detects problems and displays appropriate warnings? Example: A user should attempt to acquire data with a low fluidics level or a disconnected computer cable.

Any warnings displayed should be documented with screenshots that include the user’s initials and date.

Visual cues can also be used to prompt the user to change fluids. Example: Fluidics icons change color when low levels are detected. The system should have a warning and not allow additional acquisition until the fluidics issues are corrected.

Optical precision

Have the calibration beads been run to verify whether % CV, detector sensitivity, and laser power output meets vendor specifications?

A checklist with vendor requirements, a conclusive notation of pass/fail, and the user’s initials and date should be used as documentation.

Any automated QC reports and all testing reagents should be documented in a statement attached to the checklist. For some fully automated systems, this can only be performed by the service engineer.

Automated sample acquisition

Have triplicates of testing material (beads or cells) in randomly distributed locations in an automated loader been acquired?

A checklist with a positive notation of successful sample acquisition, a conclusive notation of pass/fail, and the laboratorian’s initials and date should be used as documentation.

There is some overlap in PQ; replicate samples can also be used to demonstrate precision. OQ can be performed using beads whereas PQ requires the use of intended biological samples.

Abbreviations: % CV, coefficient of variation expressed as a percentage, QC, quality control, OQ, operational qualification; 983 PQ, performance qualification. 984

H62

26


4.2 Performance Qualification 985 986 Demonstrating PQ may be the most time-consuming part of IQ. The parameters tested during PQ depends 987 on the intended use of the instrument. Additional PQ may be required when new assays are introduced. 988 Even after the flow cytometer is installed and passes OQ, there is no guarantee that the cytometric results 989 will be adequate for all assays. The performance of flow cytometers varies among different instrument 990 models and manufacturers. Even different instruments of the same model have different levels of 991 performance, particularly regarding fluorescence. Moreover, performance can degrade over time. It is best 992 to have objective and measurable criteria when instrument performance is assessed and is particularly 993 important when the study data is obtained from multiple cytometers. Cytometer performance can be 994 assessed through a matrix of measurement parameters that include: 995 996 Optical alignment 997 Linearity and dynamic range 998 Detection efficiency (Q) 999 Electronic noise (SDen) 1000 Background signal (B) 1001 Overall resolution of the detection system, which is affected by Q, B, and SDen 1002 Carryover during sample acquisition 1003 1004 As described below, a set of good quality multi-intensity beads that includes both unstained beads and beads 1005 with defined fluorescent intensity units is adequate for characterizing cytometer performance as it pertains 1006 to linearity, dynamic range, electronic noise, and background signal in all appropriate fluorescence 1007 detection channels of the flow cytometer. NIST has created fluorescence calibration beads with a traceable 1008 ERF that is shown in Tables 5A and 5B.24,42,43,44 These calibration beads are primarily used by 1009 manufacturers to calibrate its fluorescent bead products. The use of fluorescent beads with ERF calibration 1010 allows for accuracy and linearity to be established for measurements reported as fluorescent intensity. This 1011 approach is ideal for multi-instrument and multisite studies. Many cytometers have integrated software that 1012 can be used in combination with manufacturer-specific fluorescent beads to automate the process of 1013 determining detection efficiency, electronic noise, background signal, sensitivity, and linearity for all 1014 detectors. This chapter also outlines the procedure for calculating these performance characteristics 1015 manually. 1016 1017 NIST provides a measurement service to the flow cytometry quantitation consortium members who submit 1018 flow cytometer calibration microparticles for ERF value assignment traceable to NIST SRM® 1934 (or the 1019 equivalent). For example, certified ERF values have been assigned with specific laser excitations and 1020 fluorescence wavelength ranges for two commercial products, 15 different covalently-linked fluorochrome 1021 beads, which are listed in Table 5A, and four different hard-dyed fluorescent microparticles, which are 1022 listed in Table 5B. 1023 1024

H62

27


Table 5A. ERF Assignment of Covalently Linked Fluorochrome Beads Using SRM® 1934 (or the 1025 equivalent) 1026

Covalently Linked Fluorochrome Bead Excitation Laser, nm SRM® 1934 (or the equivalent)

FITC 488 FluoresceinPE 488 FluoresceinBB515 488 FluoresceinPerCP 488 Nile RedPerCP-Cy5.5 488 Nile RedPE-Cy7 488 Nile RedAPC 633 APCAPC-R700 633 APCAPC-H7 633 APCAPC-Cy7 633 APCV450 405 Coumarin 30BV421 405 Coumarin 30V500-C 405 Coumarin 30BV510 405 Coumarin 30BV605 405 Coumarin 30

Abbreviations: APC, allophycocyanin; ERF, equivalent reference fluorophores; FITC, fluorescein isothiocyanate; 1027 PE, phycoerythrin; PerCP, peridinin-chlorophyll protein; SRM, standard reference material. 1028 1029 Table 5B. ERF Assignment of Hard-Dyed Fluorescent Microparticles Using SRM® 1934 (or the 1030 equivalent) 1031

6-Peak Hard-Dyed Fluorescent Microparticles Excitation Laser, nm SRM® 1934 (or the equivalent)

Intensity 2–6 488 Fluorescein Intensity 2–6 488 Nile Red Intensity 2–6 633 APCIntensity 2–6 405 Coumarin 30

Abbreviations: APC, allophycocyanin; ERF, equivalent reference fluorophores; SRM, standard reference material. 1032 1033 4.2.1 Linearity and Dynamic Range 1034 1035 For a given fluorescence channel (eg, the detector shown in Figure 2 [575/26 nm, PE]), the fluorescence 1036 intensities associated with the different bead populations shown in the histogram of Figure 3 can be plotted 1037 vs the ERF values of the different bead populations referenced to Coumarin 30. The resulting calibration 1038 curve yields an estimate of linearity of response and dynamic range for this fluorescence channel. The 1039 coefficient of variation (CV) of the brightest peak or second brightest peak can be used for the assessment 1040 of the laser alignment to the sample core stream of the flow cell in the cytometer. The mean fluorescence 1041 intensity (MFI) and CV of the five fluorescent populations listed in Table 6 can also be applied to 1042 characterize the performance of the cytometer detection system. Alternatively, linearity can be evaluated 1043 using multi-intensity beads and the ratiometric method shown in Figure 2. Thus, the output is directly 1044 proportional to the input. Therefore, this method is based on the principle that if detector output is linearly 1045 proportional to input sample intensity, the intensity ratio for two different beads should be independent of 1046 where the events are placed on the intensity scale.45 The linearity of the 575/26 nm (PE) detector is checked 1047 using six-peak hard-dyed beads. MFI is determined for the two brightest beads (beads five and six) as 1048 detector voltage is increased. 1049 1050

H62

28


1051 1052

1053 Abbreviation: MFI, mean fluorescence intensity. 1054 Figure 2. Ratiometric Method for Linearity Assessment 1055 1056 Figure 3 shows a histogram of six-peak hard-dyed beads. MFI of each fluorescent bead population (black 1057 squares) is plotted against its respective ERF value (right y-axis; NIST-assigned using Coumarin 30 as the 1058 primary reference fluorophore). A linear fit (solid black line; correlation coefficient 0.979) yields a 1059 fluorescence calibration curve for the 4′6-diamidino-2-phenylindol (DAPI) channel. 1060

2.00

2.50

3.00

3.50

4.00

350 400 450 500 550 600 650

Bead

6 M

FI/B

ead

5 M

FI

Detector Voltage

A

‐3.00

‐2.00

‐1.00

0.00

1.00

2.00

3.00

1,000 10,000 100,000

% Deviation

Bead 6 MFI

B

H62

29


1061 Abbreviations: A, area of the fluorescence peak; B, background signal; CV, coefficient of variation; DAPI, 4′6-diamidino-2-1062 phenylindole; ERF, equivalent number of reference fluorophores; Q, detection efficiency. 1063 Figure 3. Example of Linearity Assessment Using the ERF Method 1064 1065 Table 6. Example of ERF Data for DAPI Channel Calibration 1066

Bead Population MFI SD % CV ERF Peak 1 (blank) 62 16 25.1 N/APeak 2 487 32 6.5 0.524 × 105

Peak 3 4 250 196 4.6 3.23 × 105

Peak 4 12 385 520 4.2 9.95 × 105

Peak 5 35 060 1 472 4.2 38.3 × 105

Peak 6 88 138 3 790 4.3 71.7 × 105

Abbreviations: % CV, coefficient of variation expressed as a percentage; DAPI, 4′6-diamidino-2-phenylindole; ERF, equivalent 1067 reference fluorophores; N/A, not applicable; MFI, mean fluorescence intensity, SD, standard deviation. 1068 1069 Panel A 1070

- Percent deviation between observed intensity ratio at a given voltage and average intensity ratio for 1071 panel A data points from 400 volts to 625 volts, plotted as a function of bead six MFI. From 400 1072 volts to 625 volts, the intensity ratio remains constant (mean ± SD = 2.40 ± 0.01), which indicates 1073 excellent linearity. At < 400 volts, beads five and six are well resolved from each other and lower 1074 intensity beads are in the set, but the intensity ratio increases to 3.45, which indicates nonlinearity 1075 at the low end of the intensity scale. 1076

1077 Panel B 1078

- Percent deviation between the mean intensity ratio (across 400 volts to 625 volts) and the observed 1079 mean intensity ratio is plotted as a function of bead six MFI. At 375 volts, the deviation is +44% 1080 (data not shown). All other deviations are < 0.5% except for the data point at 625 volts. This point 1081 has a variance of –1.0%, a distortion due to a portion of the bead six distribution being off-scale 1082 high, which is not observable with MFI. 1083

1084

H62

30


4.2.2 Electronic Noise 1085 1086 Electronic noise contributes to the broadening of measured signals in populations. The CV of a population 1087 is the accumulation of the CVs intrinsic to the sample itself and the added deviation from the measurement 1088 process in the instrument. Instrument measurement variation results from a constant level of electronic 1089 noise, optical background light, statistical change in the number of photoelectrons generated by a light 1090 pulse, laser noise, and variation in how uniformly each particle is illuminated, which alters the fluorescence 1091 signal collected by the detector. As shown in equation (1), total instrument variance is calculated from the 1092 sum of the squares of individual contributions (SD2). 1093 1094

𝑆𝐷 𝑆𝐷 𝑆𝐷 𝑆𝐷 𝑆𝐷1095

𝑆𝐷 (1) 1096 1097 For bright signals, the variability of sample illumination and detection based on its position in the sample 1098 stream (grouped in SDposition) and laser noise are dominant. For dimmer signals, the statistical nature of the 1099 photon detection process adds variance along with the variation from nonsignal photoelectrons from 1100 background light. At the low end of the measurement scale, the contribution from electronic noise becomes 1101 significant. The important consideration with regards to assessing electronic noise is that, unlike other 1102 sources, it is not amplified by increases in detector gains. 1103 1104 Electronic noise can be measured in several ways.46 The SD of the electronic noise can be measured by 1105 turning the detector voltage to zero and using a pulsed light-emitting diode (LED) source to trigger data 1106 acquisition. Alternatively, one can monitor the SD of a bead with bright uniform fluorescence as the detector 1107 voltage is reduced to successively lower values. The histogram will broaden as electronic noise becomes a 1108 significant portion of the total variance at very low voltages, and the SD will tend toward a stable number 1109 no matter how bright the initial bead fluorescence. Because all contributions to measured CV are 1110 independent of detector voltage, except for SD2

ElectronicNoise, electronic noise can be quantified by plotting 1111 measured fluorescence CV2 vs 1/MFI2 of a particle over a range of detector voltages. The slope of the plot 1112 is SD2

ElectronicNoise, as shown in equation (2). 1113 1114

𝐶𝑉 ⋅

(2) 1115

1116 SD2

ElectonicNoise can also be derived by using a set of beads, each containing a different amount of 1117 fluorophore, at a fixed detector voltage and equation (2). 1118 1119 4.2.3 Sensitivity and Background Signal 1120 1121 Variance due to the limited number of signal photoelectrons is determined by the detector sensitivity. When 1122 both signal and background light contributions are considered together, the variance in photoelectron 1123 contribution is the sum of both variances. Background signal B is expressed as the number of fluorophore 1124 units that produces the background light. When measured under conditions in which detected signals are 1125 well above the SDen, Q and B are related to total signal variance (SD2

ERF), as shown in equation (3).47 1126 1127

𝑆𝐷 𝐸𝑅𝐹 ⋅ 𝐶𝑉 𝐸𝑅𝐹 ⋅1𝑄

𝐵𝑄

1128

1129 When Q is the number of statistical photoelectrons per fluorescence signal in ERF units, B is the background 1130 signal in ERF units, and CV2

Intrinsic is associated with the accumulation of all nonphotonic signals acquired 1131 while measuring the fluorescence signal. 1132 1133

(3)

H62

31


Q and B can be measured by using either hard-dyed beads with multiple fluorescence intensity levels or a 1134 pulsed LED source. Because of variations of laser noise, illumination conditions, and dye loading for hard-1135 dyed beads, use of a pulsed LED source provides a more accurate assessment of Q and B. Because the LED 1136 pulse has a near-zero intrinsic CV, equation (3) can be simplified to give equation (4). 1137

1138

𝑆𝐷 𝐸𝑅𝐹 ⋅1𝑄

𝐵𝑄

1139

1140 A linear fit of SD2

ERF vs ERF shown in equation (4) can also be applied to determine Q and B, where the 1141 slope is 1 / Q and intercept of the fit is B / Q. A quadratic fit to equation (3) and linear fit to equation (4) 1142 should give similar results for Q and B. Values for both Q and B should be obtained with flow cytometers 1143 equipped with linear amplifiers and by using a linear scale. Importantly, the use of ERF units instead of 1144 instrument-dependent MFI units in the determination of Q and B ensures that Q and B are voltage 1145 independent and can be compared across different flow cytometers. 1146 1147 4.2.4 Resolution 1148 1149 Determining values for Q and B allows one to predict a flow cytometer’s ability to resolve dim fluorescent 1150 populations by calculating a separation parameter, S, in normalized SD units. The parameter S is defined 1151 as the difference between the MFI of a stained fluorescence population and the MFI of the background 1152 population divided by the square root of the sum of the variances of the background and fluorescence 1153 populations.48,49 At low but nonzero fluorescence intensities, this becomes equation (5). 1154 1155

𝑆𝑄 ⋅ 𝑀𝐹𝐼

1 2 ⋅𝐵

𝑀𝐹𝐼

1156

1157 As shown in Figure 4, the parameter S increases with increasing Q and decreases with increasing B. A 1158 cytometer’s minimal resolvable MFI value (S > 2) can be predicted from equation (5), and the instrument’s 1159 experimentally determined Q and B values. For most flow cytometers, B is of a similar magnitude or smaller 1160 than MFI. When B is less than sample MFI (B / MFI < 1), changing Q has a much more significant effect 1161 on the separation parameter S than changing B. In this case, increasing Q should improve the ability of a 1162 cytometer to resolve dimly fluorescent populations more effectively than by reducing B. However, when B 1163 is much greater than MFI, B becomes as important as Q. 1164

(4)

(5)

H62

32


× 1165 1166 1167 1168 Abbreviations: B, background signal; MFI, mean fluorescence intensity; Q, detection efficiency. 1169 Figure 4. Example of the Relationship of S, Q, and B 1170 1171 In Figure 4, the separation parameter S is shown as numerical values on solid lines is a function of the 1172 detection sensitivity Q, background B, and MFI. As S increases, so does the ability of the cytometer to 1173 distinguish between a dimly stained but fluorochrome-positive population and a population lacking 1174 fluorochrome. 1175 1176 In practice, S is proportional to the classic staining index (SI), which should be measured using a positive 1177 and negative population, and is defined as shown in equation (6). 1178 1179

𝑆𝐼𝑀𝐹𝐼 𝑝𝑜𝑠 𝑀𝐹𝐼 𝑛𝑒𝑔

2 ⋅ 𝑆𝐷 𝑛𝑒𝑔 1180

1181 SI can be measured using antibody-stained cells or a mixture of blank beads and stained antibody-capture 1182 beads and should be used to monitor the effect of changes in Q and B on positive/negative separation. 1183 1184 4.2.5 Carryover 1185 1186 The use of automated sample loaders increases the potential for carryover between samples, mainly when 1187 samples with a high frequency of cells/particles of interest are acquired before samples with low 1188 cell/particle frequency. Determination of carryover is particularly crucial for rare event determinations. 1189 Carryover detection should be performed as part of the initial PQ and included as a regular QA monitor.20 1190 1191 Carryover assessment can be performed by a straightforward method as defined in CLSI document H52 1192 and in the literature.20 A minimum of two samples are required, including one with a high-frequency 1193 cell/particle of interest and one lacking the cell/particle or having a low target cell/particle frequency. 1194 Triplicates of each sample should be prepared according to the laboratory’s established protocol. Replicates 1195 of a sample with a high level of cell/particle target (eg, A1, A2, A3) should be followed by replicates with 1196 a low cell/target level (eg, B1, B2, B3). The carryover percent can be calculated with equation (7). 1197

(6)

Q •

MF

I

B / MFI

H62

33


Carryover % B1 B3 / A3 A1 100 (7) 1198 1199 Acceptable limits of carryover must be determined by the laboratory, considering assay-specific 1200 requirements. At a minimum, the carryover percent should be no greater than the manufacturer’s 1201 specifications. 1202 1203 4.3 Cross-Instrument, Cross-Site Standardization 1204 1205 This subchapter describes methods that should be used to standardize multiple flow cytometers that are 1206 summarized in Table 7. The primary objective of standardization in multicolor flow cytometry should be 1207 either to issue the same diagnosis for a given patient or to produce the same experimental result, regardless 1208 of where the samples are analyzed. Therefore, the analytical sensitivity and specificity of instruments must 1209 be as close as possible. A second objective should be to accurately interpret parameters that carry prognostic 1210 value, such as the intensity of CD45 expression on acute lymphoblastic leukemia cells or a biomarker end 1211 point measurement for a pharmaceutical clinical trial.50,51 To meet these objectives, any protocol to cross-1212 standardize multiple flow cytometry instruments must produce superimposable data from the same 1213 specimen. Ideally, this means that the same cell or bead sample should give equivalent MFI/median 1214 fluorescence intensity (MdFI) values on all instruments. 1215 1216 Standardization procedures for instruments are critical to meet these objectives. The literature indicates that 1217 devices of the same model or even from different manufacturers can be calibrated and cross-1218 standardized.50,52,53,54,55 The complexity of multi-instrument standardization varies depending on the number 1219 of laboratories, instruments, and the intended use. The simplest case is a single laboratory standardizing a 1220 single instrument configuration. The instrument setup should be identical, and standardization should be 1221 based on the use of calibration materials. These materials are usually beads, most commonly internally 1222 (hard) dyed fluorescent particles classified as Type IIIA beads.56 These beads should be stable for several 1223 years when properly stored and produce a fluorescent signal in all channels. However, because they are not 1224 spectrally identical to individual fluorophores, their relative intensity in different fluorescence channels 1225 depends on the specific optical configuration, lasers, and filters used in each instrument. The use of 1226 spectrally-matched calibration material is therefore recommended to achieve cross-system and cross-1227 laboratory standardization. For intended applications involving antibody labeling, this should be 1228 accomplished using beads stably coated with the fluorophore(s) of interest or antibody-capture beads and 1229 then confirmed with antibody-stained cells. Additional applications, such as functional measures (eg, 1230 oxidative burst), may need assay-specific material. Stable fluorophore-coated beads provide an advantage 1231 because staining-related variability can be avoided, but these beads are not available for all fluorophores of 1232 interest. 1233 1234 Once all instruments have been cross-standardized by adjusting the detector voltages using fluorescence 1235 controls, each cytometer should be monitored daily using hard-dyed beads. If new target values are 1236 established, it is essential to document the new MFI/MdFI target values once the detectors have been 1237 adjusted, either manually or by using automated software. If a new lot of standardizing beads are received, 1238 the old target values need to be transferred to the new lot or the cytometers need to be restandardized 1239 according to user or vendor recommendations. Compatible instruments for standardization should: 1240 1241 Be equipped with similar excitation lasers and collection optics (emission filters) to collect comparable 1242

fluorescent signals 1243 1244 Have stable fluidics 1245 1246 Be sufficiently sensitive to discriminate dim fluorescence signals to support the intended use 1247 1248 Give reasonably low background (photon and electronic noise) 1249

H62

34


Produce linear signal across dynamic range for the intended use 1250 1251 Produce data conforming to the current flow cytometry standard (FCS) data format57 1252 1253 Table 7. Summary of Scenarios Requiring Instrument Standardization 1254

Abbreviations: MFI, mean fluorescence intensity; MdFI, median fluorescence intensity; SOP, standard operating procedure. 1255 1256

Situation Requiring Standardization Standardization Procedure Verification

One laboratory, multiple “identical” instruments (ie, same make, model, filter configuration)

1. Optimize assay-specific settings (ie, application, panel or experiment settings) on primary reference instrument.

2. Set target range on primary reference instrument using fluorescence controls (eg, hard-dyed beads, antibody-capture beads, commercial control cells), and confirmed with a viable specimen.50

3. Use same lot of beads or specimen containing cells of interest on each instrument to adjust scatter and fluorescence detectors.

1. Moderate- to high-intensity beads should give similar MFI/MdFI ± 7% on all instruments55 (see Figure 5).

or 2. Cells of interest should

produce similar plots with similar positive staining MFI/MdFI ± 15%.52,55

Multiple laboratories, multiple “identical” instruments (ie, same make, model, filter configuration)

Follow steps 1–3 listed above, and then:

4. Adhere to same SOP and use same lot of control beads for instrument setup at each site.

Follow either step 1 or 2 listed above.

1 laboratory, multiple different instruments

Follow steps 1–3 listed above, and then:

5. Adjust target MFI/MdFI values for dynamic range differences or fluorophore and optical filter differences.

Follow either step 1 or 2 listed above, and then:

3. Adjust intensity scales for range used by the instruments and any differences in filters and proprietary fluorophores (see Figure 6).55

Multiple laboratories, multiple different cytometers

Follow steps 1–5 listed above. Follow either step 1 or 2 listed above, and then follow step 3 listed above.

H62

35


Figure 5 shows eight-peak hard-dyed beads run on four standardized instruments with the same 1257 configuration; blue, purple, green, and orange histograms overlaid represent individual instruments. 1258

1259

1260 Abbreviations: MFI, mean fluorescence intensity; INT LOG, instrument log. 1261 Figure 5. Example of MFI of Hard-Dyed Beads From Four Standardized Instruments 1262 1263 Figure 6 shows MFIs of CD4+ cells from five instruments with three different optical configurations after 1264 standardization, with spectrally matched fluorescence control beads coupled to each of the indicated 1265 fluorophores. 1266 1267

1268 Abbreviations: APC, allophycocyanin; FITC, fluorescein isothiocyanate; MFI, mean fluorescence intensity; PE, phycoerythrin; 1269 PerCP, peridinin-chlorophyll protein. 1270 Figure 6. Example MFIs of CD4+ Cells With Different Optical Configurations 1271 1272 When standardizing instruments from multiple manufacturers or configurations, differences in fluorophore 1273 emissions and bandpass filters must be considered. Standardization should be verified with spectrally 1274 matched beads or stained cells. 1275 1276 Hard-dyed beads are useful for instrument calibration, QC, and tracking. However, as noted above, 1277 fluorophore-matched control beads or cells are recommended to cross-standardize instruments with 1278 different optical systems as shown in Figure 6. Standardization across instrument platforms typically 1279 focuses on consistent MFI/MdFI of positive populations. However, as illustrated in Figure 7, MFI/MdFI of 1280

H62

36


negative populations and overall resolution can vary among instruments. This is an essential factor to 1281 consider when developing and running an assay across multiple platforms. In the example shown, if one 1282 designs an assay to identify a dim population (black box) in the PE detector on cytometer 1, that same assay 1283 will fail to resolve the same population on cytometer 2. In general, assays should always be developed and 1284 tested on the poorest performing instrument (eg, cytometer 1 in Figure 7) to ensure adequate sensitivity for 1285 the intended purpose. 1286 1287

1288 Abbreviations: APC, allophycocyanin; FITC, fluorescein isothiocyanate; PE, phycoerythrin. 1289 Figure 7. Example of Cross-Standardization With Spectrally Matched Control Beads 1290 1291 Figure 7 shows three different instruments from different platforms that are cross-standardized with 1292 spectrally matched fluorescence control beads. Cross-standardization is confirmed using a three-color 1293 panel. Although the MdFI of the significant populations is equivalent in every detector, the spread of the 1294 double negative populations differs significantly due to the different optical configurations. The Q value 1295 for PE is much higher on cytometer 3 because it uses a 561 nm laser, which excites PE better. This higher 1296 Q results in a lower spread of the negative population. Similarly, cytometer three uses a 50 mW red laser 1297 instead of the 18 mW laser used on the other two instruments. Again, the higher power increases Q for 1298 APC, which results in a lower spread. 1299 1300 One of the most comprehensive examples of cross-standardization is represented by large clinical 1301 consortiums. In the method described in the literature,55 standardization is performed globally across many 1302 laboratories using eight-peak beads and setting a target range based on the second brightest or seventh peak 1303 in each channel on the primary instrument. These initial target ranges should be transferred to all other 1304 cytometers, and the MFI of the seventh peak of beads should be measured on each analyzer. The same lot 1305 of beads should be set to the same relative MFI on all instruments. The seventh peak should be used because 1306 it is optimally separated and not near the upper end of the scale, like the eighth peak. However, because 1307 individual instruments differ both in dynamic range (intensity scale) and emission filters, simply 1308 recalculating the target values to a different scale is not sufficient for the final detector voltage adjustment. 1309 If the target MFI setting results in detector settings that give suboptimal resolution or off-scale results for 1310

Cytometer 2 PE: 50 mW 488 nm

APC: 18 mW 633 nm


APC: 50 mW 640 nm


APC: 18 mW 640 nm

H62

37


cells on any instrument, the target MFI values should be adjusted accordingly until consensus target MFI 1311 values can be established for all cytometers.58 1312 1313 Cell autofluorescence has also been used to standardize instruments for specific applications, although 1314 differences in autofluorescence as a function of excitation wavelength and cell type should be considered. 1315 For example, a violet-laser–excited autofluorescence is much greater than that seen with blue or red laser 1316 excitation because of intracellular components that are excited by this near-ultraviolet light; cell lines have 1317 very different autofluorescence than lymphocytes, and lymphocytes are much less autofluorescent than 1318 granulocytes. Despite these challenges, researchers50 standardized immunophenotypic data from > 20 1319 instruments from multiple manufacturers using autofluorescence. Detectors were adjusted so that > 80% of 1320 the autofluorescence signal was above the lowest channel on the intensity scale. Standard hard-dyed beads 1321 were then run on the primary instrument to establish target ranges for longitudinal performance monitoring. 1322 1323 Clinical applications have been important drivers of methods for cross-instrument and cross-laboratory 1324 standardization. Table 8 highlights a few examples of sources and acceptable limits of variability for clinical 1325 applications after standardization. In each of these examples, multiple instruments are used over an 1326 extended period. 1327 1328 Table 8. Examples of Variability of Selected Markers on Standardized Flow Cytometers 1329

Sources of Variation Marker,

Fluorophore Marker

Intensitya,31,59Observed

Variabilityb,31,59 14 cytometers (2 models,

1 manufacturer) 11 sites Single 8-color tube, 1 send

out/year Multiple beads and reagent lots 123 locally stained peripheral

blood samples over 4 years

CD45 on T cells (Pacific Orange)

0.723 0.31

Kappa light chain on B cells (PE)

3.99 1.42

CD56 on NK cells (PE)

4.26 0.74

3 cytometers (1 model/manufacturer)

Single site, multiple technologists

Eight 4-color tubes (CD45 in each) per sample

Multiple bead and reagent lots 50 pediatric AML bone marrow

samples at ≈ 1-month post chemotherapy over 3.5 years

CD45 on mature lymphocytes (PerCP)

2.79 Replicate: 0.017 Within-patient: 0.11 Between-patient: 0.085

SSC (mature lymphocytes)

1.24 Replicate: 0.02 Within-patient: 0.14 Between-patient: 0.071

Abbreviations: AML, acute myeloid leukemia; MdFI, median fluorescence intensity; NK, natural killer; PE, phycoerythrin; PerCP, 1330 peridinin-chlorophyll proteins; SSC, side scatter; SD, standard deviation. 1331 a Kalina: MdFI averaged over all samples and time points (n = 123); expressed in log units. Loken: mean intensity averaged over 1332 all tubes (n = 50 × 8 = 400); expressed in log units. 1333 b Kalina: 95th percentile MdFI − 5th percentile MdFI (all samples and time points); expressed in log units. Loken: all metrics 1334 expressed in log units. Replicate equals SD of mean intensity (or SSC) per patient (n = 8), averaged over all patients. Within-patient 1335 equals average histogram SD per patient (n = 8), averaged over all patients. Between-patient: SD (n = 50) of average geometric 1336 mean fluorescence intensity (or SSC) per patient (n = 8). 1337 1338

H62

38


4.4 Compensation 1339 1340 The theoretical basis of compensation and the procedure for establishing it are described in CLSI documents 1341 H422 and H43.3 If instrument sensitivity is inadequate, dim antigen expression may not be detected. If 1342 fluorescence compensation is excessive, a population expressing one antigen may be incorrectly considered 1343 to lack another. Conversely, if compensation is inadequate, a population expressing one antigen may be 1344 erroneously considered also to express another. The compensation setting accuracy is critical in 1345 hematopathology because neoplastic cells frequently exhibit aberrant intensities of antigen expression or 1346 abnormal patterns of antigen co-expression. It is also essential in the detection of intracellular signaling 1347 proteins, in which the signals are often weak, and background can be increased (eg, some phosphoproteins 1348 and cytokines). 1349 1350 Establishing the compensation matrix on the instrument is assay-specific when using reagent-specific 1351 fluorescence controls or different fluorophores in the same detector (refer to Subchapter 5.1.2.2 for a more 1352 detailed description). However, if the compensation matrix is calculated through integrated software on the 1353 instrument, the basic functionality of correcting spectral overlap should be confirmed during the OQ testing. 1354 Each manufacturer has a standard procedure for verifying the production of a compensation matrix on a 1355 specific instrument involving either beads or cells. Whenever a compensation matrix is established for a 1356 particular purpose and panel of fluorophores, the following factors should be considered: 1357 1358 Laser and filter configuration of the flow cytometer 1359

1360 Sensitivity and resolution of the instrument (regarding the ability to discriminate between a negative 1361

particle and one with dim antigen expression, not numbers of fluorophore molecules measured) 1362 1363 Detector linearity 1364 1365 Spectral overlap of the fluorophores for the intended assay is essential when the fluorophores are 1366

staining different classes of molecules, eg, proteins and nucleic acids (see Figure A2 in CLSI document 1367 H422) 1368

1369 Excitation of the fluorophore by more than one laser, eg, PerCP-Cy5.5, as shown in Figure 8 1370 1371 Samples with characteristics as similar as possible to those of the test samples should be used to establish 1372 instrument performance, including both sensitivity and compensation. A variety of spectral viewers are 1373 publicly available to aid in predicting potential issues with spectral overlap and cross-laser excitation. All 1374 instruments and software programs should use the same mathematical algorithm (see equation [8]) to 1375 calculate spillover between any two fluorophores. 1376

1377

%𝑆𝑝𝑖𝑙𝑙𝑜𝑣𝑒𝑟𝑀𝐹𝐼 𝑀𝐹𝐼

𝑀𝐹𝐼 𝑀𝐹𝐼 100% 1378

1379

(8)

H62

39


Figure 8 shows an example of the excitation (dotted lines) and emission spectra (filled histograms) of four 1380 common dyes. In this case, the excitation is from a 405-nm–laser line. Many of the same fluorophores are 1381 also excited by a 488-nm–laser line. 1382 1383

1384 Abbreviations: FITC, fluorescein isothiocyanate; PE, phycoerythrin; PerCP, peridinin-chlorophyll protein. 1385 Figure 8. Spectral Overlap 1386 1387 4.4.1 General Factors to Consider for Calculating Compensation 1388 1389 Errors in compensation are most frequently due to problems with the single-color compensation controls 1390 and how they are used. Five simple rules for compensation controls should be followed: 1391 1392 The fluorescence spectrum (%Spillover) of the compensation control reagent should be identical to the 1393

reagent used in the experiment. It does not have to be the same reagent, but the fluorophore should be 1394 identical (eg, Alexa Fluor 488®b [or the equivalent] is not interchangeable with FITC). 1395

1396 The negative and positive populations should have equivalent auto-fluorescence (eg, the positive 1397

population (ie, beads) should not be combined with the negative population (ie, cells); the positive 1398 population (ie, monocytes) should not be combined with the negative population (ie, lymphocytes). 1399

1400 The positive population should be as bright as possible but within the linear range of the detector to 1401

provide the highest accuracy to the spillover calculation. 1402 1403 Sufficient events should be acquired to obtain representative intensities (at least 1 000 positive and 1404

negative events). 1405 1406 When using any automated compensation system, it is important to ensure that the proper positive and 1407

negative populations are selected for the calculations and that neither population has a significant 1408 number of events off-scale, either low or high. 1409

1410

b Alexa Fluor 488® is a registered trademark of Life Technologies Corporation.

H62

40


4.4.2 Types of Compensation Controls 1411 1412 There are several types of compensation controls, each with advantages and disadvantages: 1413 1414 Cells stained with fluorophore-labeled antibody 1415

– Antibody-labeled cells have the advantage of being closest to the true spillover and should be 1416 prepared using a generic reagent like an anti-CD4, or for the best possible spectral match, with a 1417 lot-specific reagent (typically required for tandem fluorophores). 1418 1419

– The disadvantage is that this preparation may entail the use of a precious sample, and for 1420 lot-specific controls, there may be very few antibody-positive cells in the sample. 1421 1422

Antibody-capture beads or stable prelabeled beads 1423 – Both types of beads have the advantage of being easy to prepare and use. 1424

1425 The disadvantage of both types of beads is that they are surrogates for stained cells but they are not identical. 1426 There can be spectral differences between stained beads and cells that result in spillover errors for some 1427 fluorophores. Some manufacturers’ software autocorrects these differences, but in some cases, the error is 1428 large enough to invalidate its use. When using a new fluorophore or reagent, it is essential to test it on both 1429 cells and beads. If the spillover values are not equivalent, cells must be used. 1430 1431 4.4.3 Compensation and Linearity 1432 1433 Ideally, samples should be run at instrument settings that place all cells of interest within the linear range 1434 of all relevant detectors. Spectral overlap calculations assume that detector output is linearly proportional 1435 to the optical input. Detector nonlinearity can lead to significant compensation errors, especially at the low 1436 or high end of the dynamic range, which is shown in Figure 9. For this reason, the intensity of the 1437 compensation control must fall within the linear range, even if it differs in brightness from reagents used in 1438 the assay. Although it is common to use linear fitting of multi-intensity calibrator beads to check detector 1439 linearity (see Figure 3), errors associated with the assignment of either molecule of either manufacturer-1440 assigned intensities or ERF units may lead to underestimation of nonlinearity compared with the ratiometric 1441 method (see Table 9).42 As illustrated in Figure 10, final decisions on what level of detector nonlinearity is 1442 acceptable must be made in the context of each particular assay and the type of measurement being reported 1443 (eg, % antibody positive vs staining intensity/relative antigen expression). 1444 1445

H62

41


1446 Abbreviations: FITC, fluorescein isothiocyanate; PE, phycoerythrin. 1447 Figure 9. Effect of Nonlinearity on Compensation 1448 1449 Antibody-capture beads are stained with varying levels of FITC antibody, and compensation is set using 1450 samples A and C. Compensation is the ratio of MdFI in the spillover detector (in this example, PE) to the 1451 MdFI of the primary (FITC) detector. Unstained antibody-capture beads (MdFI 68, panel A) and beads 1452 labeled with a moderate-intensity antibody (MdFI 5921, panel C) were used to calculate this ratio at a single 1453 point in the dynamic range. The resulting compensation is correct for samples B and C (ie, their PE MdFI 1454 is comparable to that of sample A). However, for sample D (MdFI 73 000), the population erroneously 1455 appears to be positive in the PE detector (> four-fold brighter than unstained beads) because the detector 1456 nonlinearity of +2% causes the calculated ratio to underestimate the correction needed. 1457 1458 Table 9. Comparison of Methods for Detector Linearity Assessment 1459

Linear Fit Methoda Ratiometric Method

Measured MFI % Difference

From Fit Bead A Median Bead B Median

% Difference From Average

Ratio

26 65 −1.75

304 4.06 77 200 2.07

788 2.84 203 514 -0.5

2 375 −2.15 1 033 2 636 0.28

6 024 −0.69 4 040 10 309 0.28

16 793 −4.15 12 730 32 474 0.25

48 441 −4.18 36 441 92 438 −0.31

104 057 3.87 98 468 24 5714 −1.94

Average = 2.46% Average = −0.202Abbreviation: MFI, mean fluorescence intensity. 1460 a Slope = 1.0 (ie, assuming perfect linearity46). 1461 1462

H62

42


1463 Abbreviations: BV, Brilliant Violet™c; CD, cluster of differentiation; SOV, spillover value. 1464 Figure 10. Compensation Error 1465 1466 The level of acceptable compensation error, including any related to detector nonlinearity, should depend 1467 on the brightness of the reagents used in a given application. Choice of compensation control should depend 1468 on the level of desired precision for the compensation matrix, with generic fluorescence control beads or 1469 antibody-capture beads providing somewhat less precise results than cells stained with an antibody bearing 1470 the same fluorophore or cells stained with the same antibody and lot used in the assay. 1471 1472 4.5 Longitudinal Performance 1473 1474 Once the IQ, OQ, initial PQ, and cross-standardization have been performed, it is essential to monitor and 1475 maintain consistent instrument performance over time. Longitudinal performance is vital for all assays in 1476 which data are tracked over extended periods of time (eg, clinical trials). The operation can be assessed 1477 through monitoring the established MFI/MdFI targets in each fluorescent channel with hard-dyed beads or 1478 antibody-capture beads. Secondly, a compensation check sample should be acquired and analyzed on each 1479 instrument to show that there has been no change in the compensation matrix. If the MFI/MdFI targets have 1480 been reproducibly set (eg, within 5%), there should also be little change in the spillover values. An 1481 appropriate proficiency testing (PT) or external quality assessment (EQA) program can be useful for 1482 monitoring performance over time. Not all programs adhere to international testing standards (eg, ISO 1483 17043), and different programs monitor different QC parameters based on the nature of the flow cytometric 1484 application. Still, published data from multiple programs show improved interlaboratory consistency over 1485 time.59,60,61 Interinstrument differences in scatter optics can contribute to overall variability, where scatter 1486 parameters are used for gating purposes. Scatter parameters should also be included as part of instrument 1487 performance monitoring. 1488 1489 4.6 Qualification and Verification of Instruments for the Intended Purpose 1490 1491 In some cases, additional PQ is necessary to determine whether a given flow cytometer is suitable for the 1492 intended use, such as analysis of small cells or particles (eg, bacteria, extracellular vesicles) or very low 1493 abundance CD markers. For low-abundance CD markers, careful optimization of sensitivity, resolution, 1494 and consideration of the effect of detector nonlinearity on fluorescence compensation are critical aspects of 1495 IQ and performance verification. Objective assessment of instrument performance characteristics—optical 1496 alignment, linearity, electronic noise, background signal, detector sensitivity, and resolution—is 1497 straightforward. The results can be used to predict performance in biological applications in whatever 1498 fluorescence intensity units the measurements are made. For dimly expressed biomarkers (eg, ZAP-70, 1499 CD38, or CD25 on T cells; microRNA) or small biological entities (eg, bacteria, extracellular vesicles), 1500 these performance characteristics are vital for the success of the cytometric measurements. When Q and B 1501

c Brilliant Violet™ is a registered trademark of Sirigen Group Ltd.

H62

43


are standardized with traceable ERF fluorescence units, users should take into account differences in 1502 performance among various flow cytometer platforms to design the most sensitive assays for use in 1503 multisite studies. Careful characterization of how scatter optics affect the ability to discriminate among 1504 small particles of different sizes and refractive indices (eg, nanobeads vs extracellular vesicles) is also vital 1505 for some intended purposes. 1506 1507

H62

44


Chapter 5: Assay Development and Optimization 1508 1509

This chapter includes information on: Assay development, assay evaluation, assay optimization, and assay characterization Documenting assay development and optimization processes

1510 Flow cytometry is a high-complexity technology that includes several components that are user-1511 configurable. The majority of flow cytometric assays are LDTs; thus, the laboratory must develop and 1512 optimize the assays to meet the intended use and quality requirements. Several factors are important when 1513 developing and optimizing flow cytometric methods. The basic steps for assay development are outlined in 1514 Figure 11. Defining the assay objective and assessing the instrument configuration are the first key steps 1515 that should be completed. Next, the assay panel and staining conditions should be defined followed by the 1516 data analysis (gating) strategy. 1517 1518 A properly developed flow cytometric method should specifically measure the intended cellular populations 1519 and antigens of interest. The gating process should exclude events present in a particular gate that are the 1520 result of nonspecific events due to fluidics instability, cell doublets, debris, compensation errors, conjugate 1521 degradation, or contamination by another cell type expressing shared antigens. For this reason, the assay 1522 specificity, defined as the ability of an assay to measure the intended measurand, should be established 1523 during the panel design and optimization phase, as illustrated in Figure 12. 1524 1525 Figure 13 highlights assay development, optimization, and validation that are iterative processes that should 1526 start with the design of the assay, followed by optimization and characterization preceding validation. If 1527 any performance issues are revealed, more optimization is required. A robust, well-characterized assay is 1528 essential before embarking on any validation activities. 1529

H

62

45

“DR

AF

T D

OC

UM

EN

T. T

his

draf

t CL

SI d

ocum

ent i

s no

t to

be re

prod

uced

or c

ircu

late

d fo

r an

y pu

rpos

e ot

her

than

rev

iew

and

com

men

t. It

is n

ot to

be

cons

ider

ed e

ithe

r fi

nal o

r pu

blis

hed

and

may

not

be

quot

ed o

r re

fere

nced

. 18

Sept

embe

r 20

19.”

1530

A

bbre

viat

ions

: mA

b, m

onoc

lona

l ant

ibod

y; M

FI,

mea

n fl

uore

scen

ce in

tens

ity.

15

31

Fig

ure

11.

Ove

rvie

w o

f A

ssay

Dev

elop

men

t S

tep

s 15

32

15

33

H62

46

“DR

AF

T D

OC

UM

EN

T. T

his

draf

t CL

SI d

ocum

ent i

s no

t to

be re

prod

uced

or c

ircu

late

d fo

r an

y pu

rpos

e ot

her

than

rev

iew

and

com

men

t. It

is n

ot to

be

cons

ider

ed e

ithe

r fi

nal o

r pu

blis

hed

and

may

not

be

quot

ed o

r re

fere

nced

. 18

Sep

tem

ber

201

9.”

The

des

ign

of a

flo

w c

ytom

etri

c m

etho

d di

rect

ly a

ffec

ts th

e m

etho

d sp

ecif

icit

y, a

s sh

own

in F

igur

e 12

. 15

34

15

35

15

36

Fig

ure

12.

Fac

tors

Th

at A

ffec

t A

ssay

Sp

ecif

icit

y 15

37

15

38

1539

•Are the an

tibodiesspecific?

•Are the fluorochrome conjugates robust?

•Are the an

tibodiestitrated

to optimal concentration?

Reagents

•Cellular phen

otypes defined

by literature

•Positive and negative biological controls

•Positive and negative an

alytical controls (eg, ex vivo stim

ulation, pep

tide blocking)

•Disease state sam

ples

Phenotype

•Th

e type of cells in the gate should be determined

.

Gating

•Compen

sation should be verified

to ensure it is correct.

Spillove

r an

d Compensation

•Cross‐reactivity of reagen

ts and im

munotherap

ies should be assessed

.

Interference

H

62

47

“DR

AF

T D

OC

UM

EN

T. T

his

draf

t CL

SI d

ocum

ent i

s no

t to

be re

prod

uced

or c

ircu

late

d fo

r an

y pu

rpos

e ot

her

than

rev

iew

and

com

men

t. It

is n

ot to

be

cons

ider

ed e

ithe

r fi

nal o

r pu

blis

hed

and

may

not

be

quot

ed o

r re

fere

nced

. 18

Sept

embe

r 20

19.”

15

40

Fig

ure

13.

Pro

cess

Map

: A

ssay

Dev

elop

men

t to

Val

idat

ion

1541

1542

T

he p

athw

ay f

rom

ass

ay d

esig

n to

val

idat

ed m

etho

d is

an

iter

ativ

e pr

oces

s. A

ssay

per

form

ance

sho

uld

be e

valu

ated

thro

ugho

ut th

e pr

oces

s so

that

15

43

perf

orm

ance

issu

es a

re id

enti

fied

and

cor

rect

ed b

efor

e em

bark

ing

on v

alid

atio

n ac

tivi

ties

. 15

44

H62

48


5.1 Assay Development 1545 1546 5.1.1 Assay Objectives 1547 1548 Flow cytometric methods are typically defined by the intended use of the assay and the type of data 1549 generated (eg, qualitative or quasiquantitative). When developing a new assay, the initial considerations 1550 should include what should be measured, how it should be measured, and in what matrix. Clinical, research, 1551 and drug development applications include: 1552 1553 Antipolymorphonuclear neutrophils (PMN) titer 1554 Antigen-specific T-cell detection (multimer staining) 1555 Apoptosis and viability analysis 1556 CD34 stem cell counts 1557 Chronic granulomatous disease and other genetic causes of PMN dysfunction detection 1558 Fetal maternal hemoglobin detection 1559 Functional assays such as intracellular cytokine detection 1560 HLA-B27 measurement 1561 Immunoplatelet counts (CD61, CD42, CD41) 1562 Infection/sepsis (CD64 expression levels) detection 1563 Leukemia/lymphoma/myelodysplasia detection 1564 Lymphocyte enumeration (T cells, B cells, NK cells) 1565 Lymphocyte HLA crossmatches for solid organ transplants 1566 MRD monitoring 1567 Phosphoflow assays 1568 PNH evaluation 1569 Proliferation assays 1570 RO assays 1571 Reticulated platelets or immature platelet fraction measurement 1572 Reticulocytes, including immature reticulocyte fraction measurement 1573 T- and B-cell immune profiling (eg, naïve, memory, proliferation, activation, exhaustion) 1574 1575 Additional considerations are related to the following type of data that should be generated by the assay 1576 and the specific reportable results, including: 1577 1578 Relative percentage of a parent population 1579

1580 Cell concentration (cells/unit volume) 1581 1582 Antigen expression levels expressed as: 1583

Fluorescence intensity readouts directly from the instrument (the instrument will report both the 1584 MFI and the MdFI, but MdFI is recommended because it is a more accurate term in general to 1585 describe fluorescence intensity) 1586

1587 Fluorescence intensity normalized to fluorescence beads and expressed as molecules of equivalent 1588

soluble fluorochrome (MESF), or ERF or antibodies bound per cell (ABC) 1589 1590 Percent bound, percent free for RO assays 1591 1592

H62

49


5.1.2 Panel Design 1593 1594 Flow cytometric methods involve identifying and characterizing one or more particular populations of 1595 interest. The following considerations are critical to design a panel where these populations can be 1596 optimally resolved: 1597 1598 The phenotype of primary populations of interest (key positive and negative antigens) should be 1599

identified 1600 1601

The level of expression of each antigen in the intended population, healthy or disease state, should be 1602 determined 1603

1604 The instrument configuration should be evaluated 1605 1606 5.1.2.1 Antigen Selection and Antibody Clone Evaluation 1607 1608 Flow cytometry has been used in clinical and nonclinical laboratory settings for many decades. As a result, 1609 human and nonhuman cellular development and activity has been described in numerous publications. For 1610 many immune cell subsets and leukemic populations, consensus phenotypes have been published. One 1611 consideration when using defined phenotypes to identify populations is the potential interference from 1612 antibody-based or immunomodulatory therapies (eg, anti-CD20, anti-CD19, anti-CD22, anti-CD28, anti-1613 CD38 therapies). Antibody clones should be screened for cross-reactivity to the therapeutic compound 1614 (refer to Subchapter 6.1.2.2), or alternative antigens should be used in the assay (eg, use of CD19 to identify 1615 B cells in the presence anti-CD20 therapeutics). 1616 1617 If the assay includes more than one staining tube, it is useful to include common markers across tubes 1618 (backbone). MRD assays in particular benefit from multiple common markers.62,63,64,65,66,67 The choice of 1619 backbone marker should be approached through the published evidence that includes previously reported 1620 tests for similar therapies, useful antigens, and antigen combinations, and International Clinical Cytometry 1621 Society guidelines.20 1622 1623 For new assays in which antibodies have not been well characterized, it is advisable to evaluate multiple 1624 clones for each antigen to select the optimal clone for the specific assay. References or data supporting the 1625 antibody specificity of each clone should be provided.25,62,63,64,65,68,69 It is necessary to possess a good 1626 understanding of the clone selected or to properly research the literature because some markers clustered 1627 under the same CD marker may bind to different epitopes resulting in selective binding. An example of this 1628 is CD34 antibodies, which are classified into three distinct categories with differential binding attributed to 1629 each class.70,71,72 Also, the isotype of the clone used for a particular marker may have an effect on the assay. 1630 As a general rule, IgG1 has a lower nonantigen-specific binding (ie, binding to Fc receptors [FcRs]) 1631 compared with IgG2a or IgG2b. 1632 1633 Once the populations of interest and their respective phenotypes have been established, it is important to 1634 determine which antigens are co-expressed on critical populations (eg, CD19 and CD10 on B cells) and 1635 which are expressed on unique populations (eg, CD3 on T cells and CD19 on B cells). Finally, the expected 1636 expression level of each antigen on each population (normal or abnormal) should be considered. In normal 1637 tissue, the antigen density of a given marker such as CD45RA and CD45RO can vary as much as 100- to 1638 500-fold on different subpopulations; however, the level of expression on a particular cellular subset is 1639 relatively stable. In contrast, expression for other antigens, in particular activation markers and checkpoint 1640 antigens, can vary over 100-fold on the equivalent population in disease-state samples. A general list of 1641 critical reagents beyond antibodies is provided in Appendix B. 1642 1643

H62

50


5.1.2.2 Fluorophore Selection 1644 1645 Flow cytometry assays can be highly multiplexed, with eight to 12 simultaneous antibodies in medical 1646 laboratories and > 18 antibodies in research and drug development settings. Selecting appropriate 1647 fluorophores for different antigens is well described in the literature.73,74 The pairing of antibodies with the 1648 optimal fluorophores is one of the most critical aspects of panel design. The primary factor influencing 1649 fluorophore selection is the instrument itself given that type of lasers and detectors determines which 1650 fluorophores can be used. From there, many factors will influence the choice of fluorophore for a given 1651 antibody, such as the level of antigen expression and the co-expression with other key antigens. Most 1652 reagent manufacturers provide information on the relative brightness of the fluorophores. Given that the 1653 relative brightness is affected by the instrument (lasers, laser power, and optics), it is important to verify 1654 fluorophore brightness on the instrument to be used for the assay. 1655 1656 Many vendors provide spectral viewer and panel designing interactive tools on their websites. As a general 1657 rule, the first principle is to pair fluorophore brightness to antigen expression; the brighter fluorophores 1658 should be reserved for the detection of weakly expressed antigens to maximize the resolution of negative 1659 and dim populations, and the dimmer fluorophore should be used with the highly expressed antigens. This 1660 is not the only approach to consider. The effect of spillover-induced spread must also be taken into account. 1661 In fact, this is now considered the major factor affecting the resolution of critical populations. Thus, 1662 although matching antigen expression to fluorophore brightness is a good first step in panel design, paying 1663 attention to the level of spread induced between two fluorophores on co-expressed markers should 1664 ultimately determine the robustness of the assay. 19,23,33,34 1665 1666 5.1.2.2.1 Fluorescence Spillover vs Spillover Values vs Spread 1667 1668 In flow cytometry, the terms spillover and spillover values are used interchangeably. Interchanging these 1669 terms is incorrect and can lead to confusion. 1670 1671 Fluorescence spillover refers to the actual amount of fluorescence (ie, photons) from a fluorophore into a 1672 different (nonprimary) detector. Thus, the operator should be aware of the amount of fluorescence spillover 1673 of the fluorophore FITC into the PE detector or of PerCP-Cy5.5 into the APC detector. Fluorescence 1674 spillover is a physical function of the filters and the fluorophores and is not affected by detector gains. 1675 1676 Spillover values are the mathematical values that the instrument and its associated software use to correct 1677 this spillover in a process called compensation. Importantly, spillover values are very gain dependent. For 1678 any fluorophore, changing the gain of any detector will alter the spillover value into that detector. 1679 1680 Spread is measured by the robust standard deviation (rSD) of the population. The process of compensation 1681 subtracts the MdFI due to the spillover. It does not reduce spread. As shown in Figure 14, the amount of 1682 the spread in any detector is directly proportional to the amount of fluorescence spillover but NOT the 1683 spillover value, which is gain dependent. For this reason, when making choices of fluorophores to 1684 minimize spread, it is not recommended to use spillover value tables to estimate the amount of spread a 1685 fluorophore will introduce into other detectors. A better tool is the spillover spread matrix (SSM).74 Creating 1686 an SSM is relatively easy and highly recommended as a valuable tool for good panel design. 1687 1688 Spread is cumulative. If there are four co-expressed markers on a population of interest, the spillover of 1689 every fluorophore will contribute to the spread of each. For any given fluorophore, the spread is also 1690 proportional to the fluorescence intensity of the marker, which is illustrated in Figure 14. 1691 1692

H62

51


1693 Abbreviations: FITC, fluorescein isothiocyanate; PE, phycoerythrin; PerCP, peridinin-chlorophyll protein. 1694 Figure 14. Relationship Between Spillover Values and Spread 1695 1696 PBMC are stained with anti-CD4 conjugated to PE (marker A), V450 (marker B), FITC (marker C), and 1697 PerCP-Cy5.5 (marker D). Spillover values of PE into the PerCP-Cy5.5 detector (marker A) and FITC into 1698 the PE detector (marker C) are essentially equivalent (17.5 vs 17.9) but show very different spread as 1699 measured by rSD of the positive population (462 vs 112), partly due to differences in intensity. Similarly, 1700 V450 into AmCyan/V500 (marker B) and PerCP-Cy5.5 into PE-Cy7 (marker D) have similar spillover 1701 values and different levels of spread. 1702 1703 5.1.2.2.2 Minimizing the Effect of Spread Resolution 1704 1705 Reagent combinations are usually considered and evaluated on a one-by-one basis: how does marker A 1706 affect marker B? The challenge of multicolor flow cytometry is that they must finally be assessed in an N- 1707 by-N basis in the context of the entire assay. A useful decision process for selecting antibody-fluorophore 1708 combinations to minimize their effect on the spread and maximize the resolution of the population(s) of 1709 interest is summarized below and in Figure 15. 1710 1711 1. Determine whether markers A and B are co-expressed on the same population. 1712

If no, choose any fluorophores (considering antigen expression and fluorophore brightness). 1713 If yes, determine which antigen is more highly expressed. 1714

1715 2. If marker A is more highly expressed than marker B: 1716

For marker A, use a dimmer fluorophore with low spillover into marker B. This will minimize the 1717 spread by having low intensity and low spillover; thus, maximizing the resolution of the A+ B+ 1718 population from A+ B-. 1719

1720 For the dimmer marker (ie, marker B), use a brighter fluorophore. Although less critical for this 1721

marker (due to lower fluorescence intensity), a fluorophore that has only low-to-moderate spill into 1722 marker A should be used. 1723

H62

52

“DR

AF

T D

OC

UM

EN

T. T

his

draf

t CL

SI d

ocum

ent i

s no

t to

be re

prod

uced

or c

ircu

late

d fo

r an

y pu

rpos

e ot

her

than

rev

iew

and

com

men

t. It

is n

ot to

be

cons

ider

ed e

ithe

r fi

nal o

r pu

blis

hed

and

may

not

be

quot

ed o

r re

fere

nced

. 18

Sept

embe

r 20

19.”

17

24

Fig

ure

15.

Dec

isio

n T

ree

for

Ch

oosi

ng

Rea

gen

t C

omb

inat

ion

s17

25

H62

53


These steps should be repeated for all possible combinations of markers. At this point, the panel must be 1726 tested to see if it satisfies the assay requirements. If it does not satisfy the assay requirements, the reagent 1727 combinations that are causing problems should be identified and alternatives should be selected as needed. 1728 Fluorescence minus one (FMO) controls, where all cells are stained with all but one antibody, are an 1729 invaluable tool when evaluating which reagents are introducing spread and reducing resolution. 1730 1731 As noted in Figure 13, assay development to validation, including panel design, is a process of iteration. In 1732 the end, assay and panel design is a process of compromise; there may be multiple populations to resolve. 1733 Some populations will have better resolution than others. The needs of the assay should dictate which 1734 populations demand the best resolution. 1735 1736 5.1.2.2.3 Tandem Dye Interactions 1737 1738 Fluorescent tandem dyes are fluorophores made of two fluorescent reagents conjugated together. Each 1739 tandem consists of a donor fluorophore (excited by a given laser line), which through a variety of 1740 mechanisms, transfers energy to the acceptor fluorophore that emits the light. Examples of these are R PE- 1741 sulforhodamine 101 sulfonyl chloride-Xd (the donor is PE and the acceptor is sulforhodamine 101 sulfonyl 1742 chloridee); and APC-H7 (the donor is APC and the acceptor is H7). In some rare cases, reagents in close 1743 proximity to each other can enhance or diminish fluorescence through resonance energy transfer. An 1744 example of this resonance energy transfer can be observed with PE conjugates that are quenched and 1745 tandems that are enhanced when in close proximity.75 Similar effects can also be observed with other donor 1746 fluorophores, such as APC. This effect is relatively rare for most cell surface antigens. However, if falsely 1747 high or low signals are seen when evaluating a particular panel of antibodies in combination vs singly, an 1748 FMO approach might prove useful in determining whether an interaction is the cause. The FMO is a type 1749 of control that can be used to identify and gate cells in the presence of spillover spread due to the multiple 1750 fluorophores in a given panel. For example, in a four-color panel, there are four separate FMO control 1751 tubes, each omitting one of the antibodies in that panel. This process should identify the amount of spread 1752 from the other three antibodies into each omitted channel. Finally, inappropriate storage conditions, 1753 including exposure to both heat and light, can cause the tandem dyes to become unconjugated, thereby 1754 leaving donor-labeled antibody. 1755 1756 5.1.3 Matrix 1757 1758 It is essential to select the most relevant specimen type when designing the assay to generate the most 1759 meaningful results. Considerations in matrix selection should include the measurements of mature vs 1760 immature markers and the specific disease state. Typical matrixes include: 1761 1762 Peripheral whole blood 1763 Bone marrow aspirates/cores 1764 Other body fluids (cerebral spinal fluid, sputum) 1765 Isolated PBMC or bone marrow mononuclear cells (BMMC) 1766 Tissues, tumors, tumor-infiltrating leukocytes 1767 Other (eg, cell lines, marine, bacteria, microparticles) 1768 1769 5.1.3.1 Whole Blood and Bone Marrow 1770 1771 Whole blood and bone marrow should be collected into tubes containing an anticoagulant and in some cases 1772 a preservative. The choice of anticoagulant should be driven by both the specimen stability and the type of 1773 assay (phenotyping, antigen expression level, or functional).20,76 Each anticoagulant has certain advantages 1774 and disadvantages that are illustrated in Table 10. 1775

d ECD® is a registered trademark of Invitrogen. e Texas Red® is a registered trademark of Life Technologies Corporation.

H62

54


5.1.3.2 Peripheral Blood Mononuclear Cells and Bone Marrow Mononuclear Cells 1776 1777 Functional assays and other assays that need long processing times are often performed on PBMC and 1778 BMMC. Advantages and disadvantages of using PBMC and BMMC are described in the literature.62,77 1779 1780 5.1.3.3 Tissue Specimens 1781 1782 Solid-tissue specimens must be disrupted into single-cell suspensions before flow cytometric analysis. 1783 Tissue disruption may be accomplished by manual or automated disaggregation or by enzymatic methods. 1784 Enzymatic methods are not usually necessary for immunophenotyping of lymphoid tissues. Either solid 1785 tissue specimens or cellular suspensions can be shipped before analysis; however, the logistics of when, 1786 where, and how the tissue specimens are to be prepared should be evaluated. The major challenge for this 1787 type of specimen evaluation is the limited availability of tissue specimens for experiments required to 1788 evaluate processing methods. 1789 1790 Table 10. Specimen Collection Materials 1791

Material Comments Advantages Disadvantages EDTA Preferred anticoagulant for

hematology analyzers Potassium and sodium

EDTA salts are available. Potassium solubility is

better than sodium; thus, it is more commonly used.

Refrigeration may extend stability.

Different cells are depleted at different rates.

Limited stability concerning both antigen detection and light scatter properties

Storage time appears to be ≈ 24–48 hours due to the depletion and/or degranulation of myeloid cells and granulocytes.

Chelates divalent cations, so assays sensitive to this process may be affected.

Sodium heparin Suitable specimen anticoagulant for the following: Immunophenotyping Functional assays

Intracellular cytokine detection

Internalization assays Cell signaling pathway

assays

As a lyophilized anticoagulant, it is appropriate for assessing absolute counts because there is no dilution effect.

Less likely to cause cellular damage when collection tubes are underfilled.

Considered less harsh on the cells compared with EDTA.

Loss of SSC cell complexity of granulocyte population when stored at room temperature for extended periods of time; however, refrigeration may extend stability.

Not suitable for morphology because of background staining.

1792 1793

H62

55


Table 10. (Continued) 1794 Material Comments Advantages Disadvantages

ACD (solution A or B)

The 2 solutions differ in the additive/blood ratio, with solution B providing more dextrose than solution A.

Extended stability of many cell surface markers and cell subsets, including erythrocytes, by providing dextrose for metabolism

As a liquid anticoagulant, a dilution factor must be applied when using for absolute counts.

Can act as a preservative for peripheral blood in a completely filled tube.

May be toxic to cells at higher concentrations, which can occur in partially filled tubes.

Sodium citrate Known to maintain the integrity and function of platelets postvenipuncture and is used for flow cytometric assays involving platelet enumeration or platelet activation.

May also be used as an anticoagulant for specimens analyzed for some lymphocyte functional assays.

Anticoagulant used for proof-of-concept evaluations involving phosphoprotein analysis, such as phospho-Akt, phospho-ERK 1/2, and phosphoribosomal S6

Very specialized application and not generally used for general flow cytometry testing protocols.

Should not be used with assays that use streptavidin because this medium contains biotin.

1795

H62

56


Table 10. (Continued) 1796 Material Comments Advantages Disadvantages

Whole-blood stabilization products

Commercially available products fall into 2 categories: Cell preservative solutions

that are added to blood after collection into standard anticoagulant blood collection tubes

Direct-draw blood collection tubes that include both anticoagulant and a cell preservative solution

These products improve the stability of surface marker expression and light scatter properties of lymphocytes and circulating tumor cells.

The proprietary stabilization compounds can exhibit effects similar to cells treated with fixatives, such as formaldehyde, which decreases the expression level of some markers or destroys the epitopes and can render the cells unsuitable for functional assays.

This effect can be compounded with improper blood collection procedures, such as partial draws or not inverting the tube immediately after collection.

May interfere with tandem dyes or cause false-positive results62

Cell culture media RPMI medium, other cell culture media, or serum are available

These media and serums provide nutrients to sustain tissue or CSF during shipment.78

Some dyes or components in these media can interfere with specific flow cytometry assays. Example: The use of phenol red, a pH indicator in media, may increase background autofluorescence.79

Abbreviations: ACD, acid citrate dextrose; CSF, cerebral spinal fluid; ERK, extracellular signal-regulated kinase; ETDA, 1797 ethylenediaminetetraacetic acid; RPMI, Roswell Park Memorial Institute; pH, negative logarithm of hydrogen ion concentration; 1798 SSC, side scatter. 1799 1800 5.1.4 Antibody Titration 1801 1802 The antigen-antibody binding reaction is concentration dependent. An increase in the concentration of 1803 labeled antibody should result in an increase in signal up to a certain point in which the signal plateaus, 1804 despite an increase in labeled antibody. The response should remain in the plateau phase until reaching a 1805 redissolution phase (decrease in signal) due to excessive antibodies.27,66,80,81 An antibody titration 1806 experiment must be conducted to determine the antibody amount and concentration resulting in the lowest 1807 level of nonspecific binding and the highest amount of specific binding.69,82,83 1808

H62

57


Titration is typically conducted using a fixed sample volume input and antibody concentration.68 The 1809 optimal sample input should be established to ensure enough sample is processed to obtain the required 1810 minimum number of cells in the critical gate(s). The titration experiment typically includes five to six levels 1811 of antibody concentrations as well as an unstained sample. The staining volume for the titrations should be 1812 the same as the reaction volume used in the final assay. Thus, antibody titrations are prepared by serial 1813 dilutions (typically two-fold) so that the volume is kept constant.84 1814 1815 If the antibody is provided with a recommended reagent volume, this may be used as the highest 1816 concentration. From there, the volume should be serially diluted down (1/2, 1/4, 1/8, 1/16, 1/32). When the 1817 antibody is not provided with recommendations, a useful starting point for the highest concentration is 0.2 1818 µg in addition to the unstained sample. Some reagents need much greater levels of dilution. 1819 1820 5.1.4.1 Titration Samples 1821 1822 Ideally the sample(s) used for the titration should be the same as the final matrix but if matrix samples or 1823 disease-state samples are not available, surrogate samples maybe used. For lymphoproliferative disorders, 1824 disease-state samples are required; if fresh samples are not available, cryopreserved samples may be spiked 1825 into normal donor samples for assay development and validation. When matrix samples are available, they 1826 must be used for confirmation. This approach is dependent on the intended use of the data. In some cases, 1827 it may be advantageous to evaluate several different samples to ensure that the titer is optimal over a range 1828 for the antigen expression levels. 1829 1830 5.1.4.2 Calculations 1831 1832 The current recommended method for evaluating the titration data is to calculate the SI, which is shown in 1833 equation (9), but many laboratories still calculate the signal-to-noise (S/N) ratio, which is shown in equation 1834 (10). A newer approach is the staining window (SW) that is shown in equation (11).85 A comparison of 1835 these three methods is shown in Figure 16 and Table 11. 1836 1837

Staining index𝑀𝑒𝑑𝑖𝑎𝑛 𝑀𝑒𝑑𝑖𝑎𝑛

2 𝑆𝐷 1838

1839 1840

Signal-to-noise ratio𝑀𝑒𝑑𝑖𝑎𝑛𝑀𝑒𝑑𝑖𝑎𝑛

1841

1842

Staining window

1843

1844 1845

(9)

(10)

(11)

H62

58


5.1.4.2.1 Staining Index 1846 1847 To assess the lowest level of nonspecific binding and the highest level of specific binding, the fluorescence 1848 intensity of a “negative” cell population should be compared with that of positive cells in the stained sample. 1849 Then the separation between a “negative” and “positive” population of cells (normalized to the SD of the 1850 negative population) should be determined by calculating the SI. 1851 1852 The SI from each sample in the titration series should be plotted in a graph against the antibody 1853 concentration or amount. In general, this results in a series of data points through which a curved line can 1854 be drawn with the maximum SI value as the “top,” displaying the best antibody concentration and amount 1855 to label representative cell sample. The reason for using the SI is to normalize the signal to the negative 1856 population spread, which tends to increase with higher antibody concentrations. 1857 1858 5.1.4.2.2 Signal-to-Noise Ratio 1859 1860 When calculating the S/N ratio, the relative MFI of the antigen-negative population should be divided by 1861 the MFI of the antigen-negative population. This works well for the fundamental assessment for most assays 1862 but does not consider the “spread” of either the positive or negative population, which will affect resolution 1863 between the two populations; moreover, the S/N approach can be overly sensitive to changes in the MFI of 1864 the negative population. For example, with the same positive MFI, a change in negative MFI from two to 1865 eight results in four-fold lower S/N, not wholly representing the effectiveness of the reagent. 1866 1867 5.1.4.2.3 Staining Window 1868 1869 S/N does not take into consideration the variation (the SD of either the positive or the negative cell 1870 populations). The reason for using the SI is to normalize the signal to the negative population spread, which 1871 tends to increase with higher antibody concentrations. There are two basic differences between an SI and 1872 an SW. First, the distance (numerator) for an SI is defined as the distance between the centers (medians) of 1873 the two distributions; an SW is defined as the distance of the two distributions. An SW considers the 1874 variations of both distributions and measures the distance between the 95th percentile of the negative vs the 1875 5th percentile of the positive distribution; thus, it captures how much (or how little) overlap the two 1876 distributions have. Second, the denominator for an SI is the SD of the negative cell populations, the 1877 denominator for an SW is the SD of positives. As is often the case in fluorescence-based assays, the 1878 variation is positively correlated with the mean (ie, SDpos > SDneg), dividing by the SD of the positive cell 1879 populations (which is larger) can more realistically measure the separation between the two distributions. 1880 1881 Anti-CD3 APC is titrated using a lysed whole-blood sample from a healthy individual. Varying amounts 1882 of antibody are added while keeping the staining volume constant at 100 µL. Lymphocytes are gated based 1883 on light scatter properties. The MdFI from the positive and negative cell populations (see Table 11) are 1884 used to calculate S/N (see Figure 16, panel A), SI (see Figure 16, panel B), and SW (see Figure 16, panel 1885 C). The selected titers are circled in red. 1886

H62

59


1887 Figure 16. Antibody Titration 1888 1889 1890 1891 1892 1893

H62

60


Anti-CD3 APC is titrated as described above in Figure 16. MdFIs are used to calculate the SI (see equation 1894 [9]), S/N (see equation [10]), and SW (see equation [11]). Data are calculated from MdFI and rounded to 1895 one decimal place. The selected titer is emphasized with bold text in Table 11. 1896 1897 Table 11. Antibody Titration 1898

Concentration MdFI+ MdFI− SDpos SDnega S/N SI SW % Gated

0.1250 µg/100 µL 160 2 52.1 1.22 73.5 65.0 0.5 70

0.1000 µg/100 µL 127 2 42.9 1.06 82.8 59.6 0.4 70

0.0625 µg/100 µL 92 1 32.0 0.86 86.0 53.2 0.4 70

0.0313 µg/100 µL 52 1 18.0 0.50 86.0 51.6 0.4 70

0.0125 µg/100 µL 32 0.5 11.4 0.49 65.1 32.0 0.3 70

0.0063 µg/100 µL 19 0.5 7.3 0.27 45.6 34.4 0.3 70 Abbreviations: MdFI, median fluorescence intensity; SD, standard deviation; SDneg, standard deviation of the negative population; 1899 SDpos, standard deviation of the positive population; SI, staining intensity; S/N, signal-to-noise; SW, staining window. 1900 a Average of SD negative = 0.73. 1901 1902 In the example shown above in Table 11, for anti-CD3 APC, the SD for the negative population (B cells) 1903 in the titration increases six-fold. For this reason, the SI should be used to calculate the optimal volume of 1904 antibody to staining for CD3+ cells. In this example, the S/N and SI are plotted against the concentration 1905 and give the different optimal concentration, but this may not always be the case with the little variation of 1906 the SD. When looking at dimly expressed proteins or activation markers, the use of SI is recommended as 1907 the broadening of the negative peak is accounted for, which is critical in determining the resolution of dim 1908 populations. Different antibody clones can show strikingly different titration curves. High-affinity 1909 antibodies typically reach saturation, as measured by MFI, at a low concentration. 1910 1911 5.1.5 Blocking 1912 1913 When PBMC, BMMC, or prelysed samples are used for staining, a blocking step is recommended to 1914 minimize the binding of antibodies to FcR.86 This blocking step should be performed by preincubating the 1915 sample with either commercially available blocking solution or with serum (eg, normal human, fetal bovine, 1916 fetal calf, immune rabbit). Blocking is not required for whole blood or bone marrow in which plasma 1917 concentration of immunoglobin is sufficiently high to block FcR binding. 1918 1919 5.1.6 Red Blood Cell Lysis 1920 1921 The purpose of lysing the sample should be to remove the RBCs, if not analyzed in the assay, and to simplify 1922 the analysis of the white blood cells or the population(s) of interest. RBC lysis is recommended over density 1923 gradient media because “immunophenotyping after density gradient isolation gives selective loss of 1924 different leukocyte populations and lower counts of lymphocyte subsets.”87 The effect of the lysis procedure 1925 on the final reportable results should be evaluated during assay development. Table 12 highlights the 1926 different lysing procedures. 1927 1928 1929

H62

61


Table 12. Staining Procedures88 1930

Abbreviations: DAPI, 4′6-diamidino-2-phenylindole; O2, oxygen; pH, negative logarithm of hydrogen ion concentration; RBC, red 1931 blood cell; SSC, side scatter; WBC, white blood cell. 1932 1933

Method Advantages (+)/Disadvantages (−) Comments Lyse/no wash Stain, lyse, no-wash

Increased background noise or debris (+) Potential for decreased resolution of dim antigens

and rare populations (−) Potential for increased debris (depending on lysing

agent) (−)

Required for single platform enumeration of cell counts

A lysing reagent with fixative may be used

Lyse/wash Stain, lyse, wash

Good signal discrimination (+) Potential for an inconsistent amount of lysed vs

unlysed material across tubes (−)

Not suitable for all antigens (eg, surface immunoglobulins)


Good separation of populations on SSC/CD45 for WBC differential (+)

Potential for increased debris (depending on lysing reagent) (−)

Potential for cell loss due to washing (−) Wash/lyse/wash Wash, stain, lyse, wash

Good signal discrimination (+) Potential for an inconsistent amount of lysed vs

unlysed material across tubes (−)

Suitable for most antigens

Required for surface immunoglobin staining


Good separation of populations on SSC/CD45 for WBC differential (+)

Potential for increased debris (depending on lysing reagent) (−)

Potential for cell loss due to washing (−) Bulk lyse Lyse, stain, wash

The recommended method for paucicellular samples (+)

Improved consistency in lysing across staining tubes (+)

The following disadvantages have been noted with this method:

Monocyte activation (−) White cell clumping (−) Lymphocyte percentage discrepancies

(−) Increased cell loss on sitting due to

cellular clumping (−) pH changes and exposure to O2 can

effect performance (−)

Recommended when cell concentration must be adjusted prestaining

No Lyse/no wash No-lyse, no-wash

Increased RBC count may result in the need for a panleukocyte marker, such as CsD45 or nuclear dye (eg, SYTO16 or DAPI), to include or exclude RBCs based on the assay (−)

Required for RBC evaluation

H62

62


5.1.7 Fixatives 1934 1935 Fixation should be used primarily to stabilize the stained samples. Essentially, it renders the cells 1936 biologically inactive. In addition, it can preserve the structural integrity of the cells. Fixation is not difficult 1937 and may be seamlessly incorporated into the sample processing workflow. Table 13 highlights different 1938 commercially available reagents that simultaneously lyse and fix samples. The benefits of performing the 1939 fixation step include: 1940 1941 Increased processed sample stability 1942

- If sample acquisition is not going to take place within one hour of sample staining, fixation prevents 1943 the loss of staining due to internalization, prevents activation induced by staining, preserves 1944 staining, and preserves cellular morphology. 1945

1946 Safety 1947

- Safety is of universal concern when working with clinical samples. Most fixatives inactivate blood 1948 borne pathogens such as HIV and hepatitis B virus. 1949

1950 Intracellular analysis 1951

- When performing intracellular analysis, it is ideal to fix the sample to minimize the movement of 1952 the target epitopes from their native intracellular or intranuclear location and to optimize the true 1953 fluorescent expression. 1954

1955 Table 13. Commercially Available Fixatives 1956

1957

Type Mode of Action Disadvantages Aldehydes (Para)Formaldehyde/methanol-

free formaldehyde

Fixation by cross-linking and creating bonds between and within proteins, specifically lysine residues

Formaldehyde has a significant advantage in that it maintains the structural integrity of the target epitopes.

Formaldehyde has also shown, in posttranslational protein studies, to arrest all modifications to target amino acids while inhibiting any continued degradation of those targets in viable cells.87

Formaldehyde may cross-link to an extent that masks the target epitopes and renders them undetectable.

Alcohols Ethanol/methanol

Fixation by coagulating the proteins and other cellular components, simultaneously dissolving lipids and creating holes in the cytoplasmic membrane

The value of alcohols is that they may be used as a one-step fixative-permeabilization process for intracellular analysis; however, due to their intrinsic denaturing properties, there is often some loss of target epitopes.

H62

63


5.1.8 Gating and Gating Controls 1958 1959 The gating strategy has a major effect on the final specificity of the assay (see Figure 12). Gating should 1960 include a time gate to exclude errors introduced by fluidic instability. Gating should also include doublet 1961 discrimination. Next, a series of sequential gates should be created to specifically identify the population 1962 of interest while excluding cells expressing shared antigens. When a gating strategy is determined, it is 1963 useful to check published evidence for previously reported assays. The final documentation should provide 1964 a detailed description of how the gating was designed to ensure specificity, such as by excluding cell types 1965 with shared antigens.89 1966 1967 Data analysis templates are recommended to ensure that the same gating strategy is used by all operators. 1968 To minimize subjective decisions and maximize the reproducibility of the analysis, a rules-based decision-1969 making approach with predefined criteria for gate adjustment should be used when the gating template is 1970 developed. A comprehensive template ensures that all elements within a sample are included in the analysis 1971 and that no element has been overlooked. Templates should be locked before validation and the assay is 1972 placed into production. In regulated laboratories, read/write permission, which should be based on the user’s 1973 level, should be described in an SOP or other driving document, and a version control process should be 1974 established. 1975 1976 In addition to designing a solid gating strategy, the assay may require the use of internal or external gating 1977 controls (see Chapter 7 for additional information). One of the most important aspects of flow cytometry is 1978 the ability to distinguish marker-positive cells from marker-negative cells. In certain diseases, the ability to 1979 distinguish marker-dim cells from marker-negative cells is also crucial. The level of cellular 1980 autofluorescence, spectral overlap, population spreading, and nonspecific binding can be determined with 1981 internal control populations or the use of FMO tubes. For some assays, such as those for 1982 leukemia/lymphoma immunophenotyping, internal control populations within a sample that are negative 1983 for a particular marker may be used as gating controls.90,91 In many applications in which there is a positively 1984 stained population that is clearly separated from the negative, the use of internal controls works well. 1985 However, if this is not the case, or if the assay confers a change in a population of interest (ie, activation 1986 assays and flow crossmatch for transplant donors), an FMO control tube(s) may be required to correctly 1987 establish the positive:negative staining boundaries.92,93 When analyzing discrete populations, such as 1988 normal CD19+ B cells, an FMO is not necessary. However, when the loss of antigen expression is being 1989 evaluated and the need to resolve dim from negative cells is identified, FMO controls are essential. 1990 1991 5.2 Assay Evaluation 1992 1993 After the assay has been developed, the performance of the full panel should be evaluated. Undesirable 1994 interactions that might occur should be identified and corrected at this step in the process. Below are some 1995 effects that have been noted in the literature and may account for some issues that laboratories might 1996 encounter. 1997 1998 5.2.1 Steric Hindrance 1999 2000 Steric hindrance is defined as the slowing or stopping of a reaction (in this case, antibody/antigen binding) 2001 due to structure (in this case, arrangement or accessibility of the antibody binding site or epitope). To look 2002 for steric hinderance, samples stained with the full combination of antibodies should be compared with the 2003 single stained controls. Often changes can be seen, with a slight (≈ 10%) reduction in MFI seen in many 2004 combinations.94 More substantial variations should be additionally investigated and may not affect the 2005 ability of the assay to perform the testing required. 2006 2007 2008

H62

64


5.2.2 Differences in Binding Sites 2009 2010 There are different clones available for the same antigen. Different clones may map to different epitopes 2011 on the antigen and, for this reason, might result in different intensities or expression patterns for the same 2012 fluorophore. When a new antibody/fluorophore combination is being tested, it is prudent to consult the 2013 literature or consider testing different antibody clones.95 2014 2015 5.2.3 Fc Receptor Binding 2016 2017 FcR can bind to some fluorophores as well as to antibodies. R-phycoerythrin (PE) and cyanine fluorophores 2018 have been shown to bind FcR.96,97,98,99,100,101,102 In cases in which FcR binding may be an issue, a blocking 2019 step may help to reduce the effect. Additional information on blocking is included in Subchapter 5.1.5. 2020 2021 5.3 Assay Optimization 2022 2023 The extent of the assay optimization depends on the intended use. Assays intended for full validation must 2024 be fully optimized, and tolerance limits conditions, such as viability and incubation, must be established 2025 before validation. Additional information on full validations is included in Subchapter 6.4 and Appendix 2026 A. 2027 2028 5.3.1 Staining Incubation 2029 2030 The optimal time and temperature of incubation should be established. Many assays are performed for 15 2031 minutes at room temperature (18 to 25°C) and protected from light. The higher the temperature, the more 2032 nonspecific binding will occur.83 For some sample types, such as cryopreserved samples, incubation on ice 2033 or at 4°C may be required. For assays intended for full validation, the SOP prepared before validation must 2034 describe the allowable range for the incubation time and temperature (eg, 15 min ± xx min). See Subchapter 2035 6.4 and Appendix A for more information on full validations. 2036 2037 5.3.2 Assay Sensitivity 2038 2039 When the assay is expected to report low levels of a population, such as in MRD assessment or the 2040 evaluation of a cell-depleting therapy, assay sensitivity must be optimized and the LLOQ validated. 2041 Additional information is included in Subchapter 6.1.2.3. During optimization, the minimum number of 2042 events required in a particular gate should be determined. For assays intended for full validation, the SOP 2043 prepared before validation should describe the minimum number of events required and instructions 2044 regarding how the data should be reported if the criteria are not met. 2045 2046 Several publications have recommended that 20 to 50 events be present in the gate, and this suggestion 2047 makes sense in the context of Poisson distribution, in which the random distribution contribution to variance 2048 can be estimated relative to the number of events.103,104 Table 14 provides theoretical examples of total 2049 events to be acquired to achieve a specified precision for a population of interest present at different 2050 frequencies within the sample. The achieved precision needs to be demonstrated. 2051 2052 2053

H62

65


Table 14. Acquistion Requirements for Rare Event Analysis. The estimated number of events to be 2054 acquired based on the expected frequency of the population is shown below. (Modified from Allan and 2055 Keeney.103) 2056

Desired CV 1% 2.5% 5% 10% 20% Required Events

for the Population of

Interest 10 000 1 600 400 100 25 Population of

Interest Frequency, % Total Number Acquired Events

10 100 000 16 000 4 000 1 000 250

1 1 000 000 160 000 40 000 10 000 2 500

0.1 10 000 000 1 600 000 400 000 100 000 25 000

0.01 100 000 000 16 000 000 4 000 000 1 000 000 250 000

0.001 1 000 000 000 160 000 000 40 000 000 10 000 000 2 500 000

Abbreviation: CV, coefficient of variation. 2057 2058 5.3.3 Cell Counting 2059 2060 There are multiple methods for determining the number of cells or cell concentrations. These methods 2061 include: 2062 2063 The use of a hematology analyzer 2064 2065 The use of a counting hemocytometer 2066 2067 The acquisition of data on a cytometer (examples of methods used to determine cell counts include 2068

calibrating the flow rate or sample volume on the cytometer or adding beads into the sample at a known 2069 concentration and using cell-to-bead ratio to determine cell concentration). 2070

2071 5.3.4 Viability Assessment 2072 2073 In many cases, the effects of sample viability need to be assessed as part of assay optimization. The method 2074 for assessing viability needs to be characterized during assay optimization. For some clinical assays, a 2075 viability assessment is required. For assays intended for full validation, the SOP prepared before validation 2076 must describe the sample viability requirements and provide instructions regarding how the sample should 2077 be processed and reported if the criteria are not met. Blood stabilization reagents partially fix the samples; 2078 thus, it is not possible to assess viability. 2079 2080 5.3.5 Acquisition Template 2081 2082 The acquisition template should be associated with the assay-specific instrument settings and contain 2083 information about each labeled antibody in each detector. It should contain the appropriate threshold and 2084 stop gates. In addition, it should include sufficient multivariate plots or histograms to allow for a 2085 preliminary evaluation of assay acceptability, such as ensuring that there were no technical issues, that all 2086 the reagents are preforming as expected, and that a sufficient number of events were acquired. The template 2087 should always include ungated data for evaluation of possible fluidics issues. 2088 2089

H62

66


5.4 Assay Characterization 2090 2091 After the assay has been developed and optimized, it should be determined whether the assay is fit for the 2092 intended use. This determination includes evaluating stability and precision. If the assay is not acceptable 2093 for the intended use, additional assay optimization or development must be conducted (see Figure 13). QC 2094 samples for the assay should also be evaluated during the assay characterization phase. 2095 2096 5.4.1 Specimen and Sample Stability and Storage 2097 2098 If specimens will be shipped to a testing laboratory, specimen stability and the effect of storage and shipping 2099 conditions must be evaluated and validated. Before starting the method validation phase, it is important to 2100 know whether the assay meets the anticipated stability requirements. At this stage, before validation, it is 2101 sufficient to evaluate three specimens. The time points should be taken out to one or two points past the 2102 latest time points when the specimens are expected to arrive at the testing facility. 2103 2104 The literature describes in detail the general process for evaluating specimen stability for flow cytometry.76 2105 Assay-specific considerations are covered in CLSI documents H42,2 H43,3 and H52.4,76 Stability evaluation 2106 should also include a careful examination of analysis plots in addition to a review of the statistical data. 2107 The effect of any changes in the position of the population(s) and the relative fluorescence intensity of a 2108 given antigen should be assessed. The data analysis process and acceptance criteria for stability assessments 2109 are described in Subchapter 6.1.2.6. 2110 2111 Ambient temperature and storage at 4C should be evaluated. If this evaluation reveals that specimen 2112 temperature is critical, temperature tracking should be considered. The use of temperature tracking devices 2113 may add significantly to the cost of sample analysis through the increased cost of acquiring the devices and 2114 added weight to the packages. However, these devices can contribute valuable data to the extremes that the 2115 specimens experience during transit, and this knowledge may be of value in explaining unexpected 2116 observations or any outliers in the data sets. Options for temperature monitoring are described in Appendix 2117 C. 2118 2119 If the testing will be conducted on PBMC or BMMC, stability must be assessed at multiple stages, including 2120 postcollection stability before isolation, postisolation stability, and frozen sample storage.76 When cells are 2121 cryopreserved and stored before analysis, sample handling (freezing and thawing methods, and storage and 2122 transport temperature) as well as frozen sample stability (cell viability, phenotype, and function) should be 2123 evaluated for the duration of sample storage. Ideally, frozen sample stability should be evaluated 2124 prospectively for the period of the anticipated storage time (ie, length of a study). If this is not possible, 2125 stability studies should be set up to run in parallel with and throughout sample analysis. Evaluation of 2126 cryopreserved samples should include viability assessment. Various thawing processes and sample 2127 handling procedures should also be evaluated. Loss of cell viability occurs when cells are warmed, even 2128 temporarily, which can occur each time a rack of boxes containing vials is taken out of the drawer to add 2129 or remove a sample. Extensive loss of cell viability may affect the results of intended studies; if only a 2130 subpopulation is viable, it may not be representative of the whole. 2131 2132 2133

H62

67


5.4.2 Processed Sample Stability 2134 2135 Processed sample stability should also be evaluated. The time points should be selected based on the 2136 maximal time anticipated from sample staining to acquisition. Both ambient temperature and storage at 4C 2137 should be evaluated. Three samples are adequate for this prevalidation evaluation. 2138 2139 5.4.3 Evaluation of Quality Control Material 2140 2141 After it has been determined that the assay performs adequately for the intended use, QC material should 2142 be evaluated. There are a variety of commercially available QC materials for flow cytometry applications. 2143 Ideally the QC material should have properties similar to the matrix samples and a long shelf life, but this 2144 is not always the case.25,89 The QC material should be able to detect all the reportable results and/or antigens 2145 detected in the assay, but this is not always the case. The laboratory must find the best solution that is 2146 aligned with regulatory requirements. If QC material will be used in the testing phase, the performance 2147 should be characterized during assay validation. Additional information on assay validation is included in 2148 Appendix A, and details on performance during the examination and postexamination phase are described 2149 in Chapter 7. 2150 2151 5.4.4 Antibody Cocktail Stability 2152 2153 Using premade cocktails increases efficiency by reducing the time and potential pipetting errors associated 2154 with adding antibodies individually. Antibody stability should be evaluated to determine the performance 2155 of the reagents over time, as described in Subchapter 6.1.2.6. Tandem dyes can be unstable and prone to 2156 degradation due to protein interactions in the cocktail. It is important to use these reagents with care and 2157 ensure minimal light exposure and temperature fluctuations to increase stability. A reagent that is beyond 2158 its expiration date should not be used. Additional expiration date items to consider include: 2159 2160 Manufacturer-provided (premade) cocktails should not be used beyond the vendor’s expiration date 2161 2162 Vendor determination of an expiration date does not constitute validation of cocktail stability 2163 2164 The laboratory is responsible for determining an expiration date for any laboratory-made cocktails. 2165

The cocktail creation and time of expiration (reagent stability) should be documented and labeled on 2166 the final reagent creation, clearly demarked, and should not be used after the expiration date. In most 2167 instances, laboratory-made cocktails have a shorter shelf life than single reagents. 2168

2169 5.5 Documentation 2170 2171 A summary should be made of the assay development and optimization process. The individual decisions 2172 made while designing the panel should be clearly described. This information may be of value in 2173 troubleshooting future issues. The method should be finalized before validation and described in an SOP 2174 or other guidance document. It is also acceptable to include the method protocol in the validation plan. The 2175 method protocol should include: 2176 2177 Details on sample acquisition, storage, and processing 2178 Information on interfering substances or conditions that can influence the assay result 2179 A description of the specificity of each antigen and reportable results 2180

H62

68


Chapter 6: Analytical Method Validation 2181 2182

This chapter includes information on: Validation planning phase General validation plans Validation plans for quantitative and qualitative data/methods Validation implementation Validation reports Validation plans based on intended use and regulatory requirements

2183 To obtain the highest quality data, it is recommended that all assays undergo some analytical method 2184 validation irrespective of the intended use; therefore, an FFP approach to validation is presented throughout 2185 this guideline. This FFP approach has been widely applied in the drug discovery and development industry 2186 for more than 15 years and is described in numerous publications in peer-reviewed journals as well as FDA 2187 documents.1 It is a practical, iterative validation, tailored to the intended use of the data and the associated 2188 regulatory requirements, if any. FFP method validation is not synonymous with no validation or inadequate 2189 validation; thus, this guideline presents the minimal standards for FFP as well as full validation for a wide 2190 variety of intended uses. Additional information on full validation is included in Appendix A. 2191 2192 The validation strategy for each method should take several factors into consideration, including risk to the 2193 patient (if applicable), application, bioanalytical data category, industry standards, and regulatory 2194 requirements. For example, if a medical laboratory implements an assay approved for use as an IVD/CE, a 2195 method verification rather than a validation is required, as described by some regulatory agencies and/or 2196 accreditation organizations.105 In this particular case, the assay (ie, device) manufacturer has completed the 2197 full validation that has been accepted by the regulatory agencies. Thus, a laboratory’s objective is not to 2198 establish the assay’s performance parameters but to verify that the established performance parameters can 2199 be replicated in their laboratory. Verifications require that the laboratory follow all documented procedures 2200 from specimen collection to analysis, as described by the manufacturer. Other validation scenarios are 2201 described in Subchapter 6.4. 2202 2203 Validation is not a one-time activity. It is necessary for the initial validation to meet the requirements of the 2204 intended use of the data and the regulatory requirements associated with this use. As these aspects change, 2205 supplemental validation studies can be performed to confirm test suitability. If an assay is modified, 2206 depending on the nature of the modification, additional validation is required, as described in Subchapter 2207 6.4. 2208 2209 6.1 Validation Planning Phase (Say It!) 2210 2211 Analytical method validation can begin only after assay development/optimization has been fully 2212 completed (see Figure 13). Validation cannot be initiated until the final assay conditions are established. 2213 Validation should adequately cover the conditions for which the assay will be applied. This includes all 2214 specimen matrixes and intended use samples. 2215 2216 The first step of the validation is to prepare a validation plan, also referred to as a validation protocol. The 2217 validation plan serves as the driving document describing all the activities that should occur during the 2218 validation. In addition, the validation plan should specify the acceptance criteria for each of the validation 2219 parameters. The documentation practices and signatory requirements for the validation plan depend on the 2220 regulatory requirements of the laboratory, assay context of use or the intended use of the data, and 2221 institutional policies.106 2222

H62

69


The validation approach for quantitative methods and qualitative methods are very different; thus, they are 2223 presented in separate subchapters below. Subchapter 6.1.1 describes the general requirements for validation 2224 plans, while the specifics related to quantitative methods and qualitative methods are provided in 2225 Subchapters 6.1.2 and 6.1.3, respectively. 2226 2227 6.1.1 General Validation Plans 2228 2229 The validation plan should contain: 2230 2231 The purpose of the assay and relevant background information 2232

2233 A list of staff who are responsible for various aspects of the validation 2234

2235 A description of the scope of the validation 2236

- A summary of issues that are expected to affect the design of the validation experiments and which 2237 validation parameters will be discussed 2238

2239 - If specific validation parameters are not discussed, an explanation should be provided 2240

2241 A full description of the method or a reference to the SOP 2242 2243 A list of all critical reagents, including the manufacturer(s) and catalog number(s) 2244 2245 A list of all equipment, including the manufacturer(s), model(s), and serial number(s)/equipment 2246

identification 2247 2248 A list of all software, including the manufacturer(s) and version number(s) 2249 2250 A detailed description of the validation samples 2251

- Collection procedures, including anticoagulant and storage temperature 2252 - Disease-state status 2253 - Procurement source 2254

2255 A description of the QC material, if applicable 2256

2257 The number of sample replicates, runs, and operators to be used to cover each validation parameter 2258 2259 The statistical calculations and acceptance criteria to be applied 2260

2261 6.1.1.1 Acceptance Criteria 2262 2263 There is a considerable amount of debate and uncertainty surrounding the appropriate acceptance criteria. 2264 A recent FDA publication, Bioanalytical Method Validation Guidance for Industry, states that the “FFP 2265 validation approach in which the level of validation should be appropriate for the intended purpose of the 2266 study” should be applied.1 Moreover, in the same document, acceptance criteria for ligand binding assays 2267 precision is set up to 20% CV with up to 25% permissible at the LLOQ.1 Although the Bioanalytical Method 2268 Validation Guidance for Industry is primarily applicable to quantification assays, such as PK method and 2269 PD biomarker method with data being used for regulatory decision making, such as primary efficacy end 2270 point for drug approval, the target values serve as a reasonable reference for the acceptability criteria of 2271 flow cytometric methods.1 Typically, 25% CV is acceptable for flow cytometric assays, with 30% to 35% 2272 CV allowable for LLOQ.18,20 2273 2274

H62

70


The accuracy acceptability of % bias at 20% and 25% at LLOQ, as described in the Bioanalytical Method 2275 Validation Guidance for Industry,1 is applicable only to definitive and relative quantitative assays that 2276 include well-characterized reference standards, such as drugs for PK assays. For tests in which the 2277 assessment of accuracy may not be feasible, such as quasiquantitative and qualitative data, this acceptance 2278 criteria does not apply (See Subchapter 3.1.1 for more information on these data categories). 2279 2280 As described in CLSI document EP12,23 for assays that report qualitative results, the acceptance criterion 2281 is 100% concordance. If positive results are defined by a cut-point, the establishment of the cut-off should 2282 be data driven using statistical approaches. 2283 2284 6.1.1.2 Validation Samples 2285 2286 As with all aspects of analytical method validation, an FFP approach should be applied when the choice of 2287 validation samples is being considered. Validation must be conducted in the same matrix (ie, sample type 2288 and anticoagulant) that will be used for the assay. If an additional matrix is added later, additional validation 2289 needs to be conducted in that matrix. Additional information on validation samples is included in Appendix 2290 A. 2291 2292 If the intended use of the assay is as an IVD, disease state or intended use samples are mandatory for both 2293 the analytical method validation as well as the clinical validation. In contrast, for PD biomarkers, disease-2294 state samples are not appropriate for the initial analytical method validation when used in phase I clinical 2295 trials that involve healthy donors. Later, if the assay is applied to trials including patient populations, 2296 additional experiments should be conducted to verify comparable assay performance, as described in 2297 Subchapter 6.1.2.2. If the target population is expected to be different in healthy donors and disease-state 2298 (intended use) samples, including disease-state samples in the validation is essential. 2299 2300 6.1.2 Validation Plans for Quantitative Data/Methods 2301 2302 6.1.2.1 Accuracy/Trueness 2303 2304 Chapter 3 details the challenges associated with the validation of accuracy/trueness in flow cytometry. 2305 Depending on the intended use of the data and regulatory requirements, a level of accuracy assessment may 2306 nonetheless be required.107 Accepted alternative approaches to satisfying the accuracy validation 2307 requirement include: 2308 2309 Testing reference standards 2310 2311 Testing reference materials 2312

- This approach is recommended but difficult to implement because there are limited materials 2313 available, as described in Subchapter 3.1.2.2. 2314

2315 PT survey material 2316 2317 Comparison against current reference methodology 2318 2319 Interlaboratory comparison 2320 2321 Verification with specimens obtained from patients with a diagnosis confirmed by alternative methods 2322 2323 IVD/CE-marked clinical kits usually provide QC materials. When available, these QC materials should be 2324 included in the method validation to enhance the accuracy claims. These approaches are of value and can 2325 verify that the assay under validation provides results comparable to other laboratories; nonetheless, the 2326 true value of the measurand cannot be determined without calibrators. Because none of these approaches 2327

H62

71


are ideal, the validating laboratory needs to decide whether the assessment adds value to the method 2328 validation. 2329 2330 For proprietary assays (eg, novel biomarker methods), the approaches described above may not be 2331 appropriate because the interlaboratory comparison may not be possible and alternative methodologies may 2332 not exist. In these cases, if accuracy evaluation is requested by a regulatory agency, the laboratory can 2333 consider creating an in-house “surrogate” reference material. Laboratories can use a scaled-down version 2334 of processes used by metrological institutions to obtain a reasonable target value for the surrogate reference 2335 standard that can be generated from the mean value of repeat testing.108 This approach is feasible only with 2336 a preserved material with defined stability limits, such as cryopreserved cells or stabilized whole blood. A 2337 minimum of 20 values generated over four or more analytical runs or testing days, with multiple operators 2338 and instruments, is recommended. 2339 2340 The acceptance criteria depend on the required degree of accuracy for the intended use but must be clearly 2341 defined for each assay. When evaluating QC material provided with an IVD/CE-marked test, results should 2342 be within 20% of the target values. For assays with high imprecision, an acceptance criterion based on SD 2343 is more appropriate to use than percent difference. Because the Westgard rules for QC monitoring generally 2344 accept a two SD range, this is a reasonable target for accuracy as well.109 It is not uncommon to observe 2345 poor correlations when flow cytometry data are compared with other methodologies. In many cases, poor 2346 correlation may be due to the increased specificity and sensitivity of flow cytometric methods. In cases in 2347 which agreement is low, the validating laboratory should provide a probable explanation for the 2348 discrepancies. 2349 2350 6.1.2.2 Specificity and Selectivity 2351 2352 Specificity for flow cytometric methods is primarily accomplished during the assay development phase (see 2353 Chapter 5 for more information). Validation documentation should include a summary of the assay 2354 development approach with regard to specificity, such as the references that justify the choice of antigens 2355 and gating strategy. If required, selectivity can be evaluated in the initial validation or in a supplemental 2356 validation. A minimum of three samples from the disease state suspected to contain an interfering substance 2357 (or population) should be evaluated for precision and stability. 2358 2359 For immune monitoring, biomarker methods in which the assay may be applied to multiple disease states 2360 in addition to healthy individuals, the disease-state samples should be compared with the original validation. 2361 As with all flow cytometric methods, it is critical that the gating plots also be examined to ensure that the 2362 distribution of the populations is not affected. The acceptance criteria should be risk-based, balancing the 2363 level of observed interference with the effect on the data. Results from the initial validation should guide 2364 the decision regarding the level of influence of the interfering compound or population. 2365 2366 It is becoming increasingly common for patients to undergo treatment with immunotherapies that may 2367 interfere with the assay. For example, when a drug targets an antigen included in the flow cytometric panel, 2368 the effect of the compound on the reported results must be evaluated. In such cases, spiking experiments 2369 with the biotherapeutic agent is recommended. The number of levels of spiking and the dilution scheme 2370 should be established on a case-by-case basis. In general, at least three levels of the compound spiked into 2371 samples from three disease state or healthy donors should provide sufficient data. The concentration ranges 2372 should span the expected PK levels of the compound. Samples should be tested in replicate so that the effect 2373 on both the mean value as well as the precision are determined. 2374 2375 CLSI document EP1223 suggests that when validating an LDT (medical laboratories), a novel IVD, or CDx 2376 assay (manufacturers), there will be additional requirements for validating clinical specificity and 2377 establishing the positive and negative predictive value of the test. Additional information on LDT is 2378 included in this chapter and in Appendix A. 2379 2380

H62

72


6.1.2.3 Sensitivity 2381 2382 Typically, analytical sensitivity for quantitative methods is defined using both accuracy and precision to 2383 determine the LOB/LOD and LLOQ. In the case of flow cytometric methods in which accuracy cannot be 2384 established, sensitivity should be established using precision alone. For methods in which low levels of the 2385 cellular population of interest are not expected, it may not be necessary to evaluate analytical or functional 2386 sensitivity during validation beyond demonstrating the assays adequate performance at the lowest 2387 anticipated analyte level. Strategies for designing high sensitivity methods are described in Chapter 5. If 2388 sensitivity is not managed during validation, the validation plan should provide a clear justification. 2389 Practices for providing data that are below the validated sensitivity of the assay should follow the 2390 laboratory’s policies and are covered in Chapter 7. 2391 2392 Finding or generating appropriate samples for sensitivity validation can be challenging. For clinical assays 2393 in which the measurand (population of interest) will be absent from a nondiseased population, such as 2394 leukemia/lymphoma, healthy donor samples are recommended as blank samples. In other situations, a blank 2395 sample can be created through partial staining by omitting one or more key staining reagents that are used 2396 late in the gating hierarchy. When a staining reagent omission approach is used, possible contributions of 2397 the omitted staining antibodies to background should be adequately considered during the assay 2398 development phase. 2399 2400 Possible ways to generate samples for LOB/LOD include: 2401 2402 Partially staining a sample by omitting antibodies so that the population of interest or antigen of interest 2403

is not detected. For some assays, this may include the use of an FMO tube or an “FMx” tube in which 2404 more than one antibody/fluorophore is omitted. 2405 2406

Depleting the target cell population using immunomagnetic beads. The risk of applying this approach 2407 is that the process may not result in 100% depletion of the targeted cell population. Target cell depletion 2408 may be useful when the population of interest is a primary population, such as B cells, and cells are 2409 depleted with anti-CD19, for example, but not when the population(s) of interest are B-cell subsets. In 2410 the latter case, the subsets will most likely be uniformly (ie, not selectively) depleted along with the 2411 parent population; thus, it is not possible to modify the LLOQ of each subset. 2412

2413 Using a healthy sample that lacks the population of interest or antigen of interest, such as a healthy 2414

donor when evaluating leukemic cells. 2415 2416 Possible ways to generate samples for LLOQ include: 2417 2418 Partially staining a sample by omitting antibodies so that the population of interest or antigen of interest 2419

will not be detected. Then, admix with varying levels of a fully stained sample to create serial dilutions. 2420 2421 Partially depleting the population of interest. 2422 2423 Admixing the population of interest at varying levels into samples lacking that population. This 2424

approach can be applied in leukemia/lymphoma immunophenotyping assays that report 2425 quasiquantitative data such as MRD. 2426

2427 The use of cell lines, particularly admixed with matrix samples, can be of utility when evaluating sensitivity. 2428 However, cell lines are often poor mimics for the actual targets and frequently have unexpected interactions 2429 with normal matrix samples. The use of cell lines therefore should usually be supplemental to other 2430 approaches or reserved for cases in which other approaches are not feasible. 2431 2432

H62

73


6.1.2.4 Analytical Sensitivity (Limit of Blank/Limit of Detection for Population Frequency 2433 Measurements and Fluorescence Intensity Measurements) 2434 2435 A few different approaches can be used to establish the LOB/LOD for population frequency measurements. 2436 One method is to use values in the gated regions (number of events or the percent of the parent population) 2437 obtained with “blank samples” as described above. The LOB is calculated as the mean of the blank + 1.645 2438 SD and LOD as the mean of the blank + 3 SD or the LOB + 1.645 SD.110 Ideally, a minimum of 10 samples 2439 should be used for this calculation (see CLSI document EP1223 for more details).20,107,110,111 Data may be 2440 obtained from different donors and different runs used in the course of the validation experiments, provided 2441 that the appropriate blank is used in these experiments.18,20 2442 2443 For LDT (medical laboratories) and novel IVD or CDx assays (manufacturers), another approach for 2444 verifying the LOD is to evaluate replicates (N = 5) of five negative and five low-positive samples. The 2445 stained sample can be acquired multiple times (ie, in separate list mode files) to increase the statistical 2446 power. Using this approach, 125 data points can be generated (5 donors 5 replicates 5 acquisitions each). 2447 A best practice is to analyze the samples on two or more instruments over a minimum of three separate 2448 days to incorporate the effect of instrument and day-to-day variability. The LOB is calculated as the mean 2449 of the blank + 1.645 SD and LOD as the mean of the blank + 3 SD or the LOB + 1.645 SD.110 The acceptance 2450 criteria for confirming the LOB is that no more than 5% of the blank replicates exceed the low-positive 2451 target and that no more than 5% of the low-positive sample replicates fall below the target LOD. 2452 2453 To calculate the LOB/LOD for fluorescence intensity measurements, the MdFI values of the negative 2454 sample (blank) should be used. An alternative method, unique to flow cytometry, is to calculate an SW 2455 (refer to equation [11]). MdFI converted to alternative units such as MESF or ERF are also suitable.43 An 2456 alternative approach used in medical laboratories is to estimate the LOD by analyzing replicates (N = 5) of 2457 a single sample in which a low level of measurand is present along with replicates (N = 5) of a single negative 2458 FMO to estimate LOB.20 2459 2460 6.1.2.4.1 Functional Sensitivity (Lower Limit of Quantification) 2461 2462 For flow cytometric methods, the LLOQ can be established by analyzing samples with levels of the 2463 measurand approaching the LOB/LOD. A minimum of five levels for each of the three donors should be 2464 created as described earlier in this chapter and tested in triplicate. An alternative approach to verify the 2465 LLOQ used in medical laboratories is to test five or more clinical samples near the LLOQ in triplicate (at 2466 a minimum). 2467 2468 When the LLOQ is established, it is essential to evaluate the number of events in the gate in addition to the 2469 reportable result. The validating laboratory must determine the minimum number of gated events needed 2470 to achieve the required sensitivity level. This process should be clearly defined in the SOP. Several 2471 publications recommend that at least 20 to 50 events are present in the gate, and this suggestion makes 2472 sense in the context of Poisson statistics in which the random distribution contribution to variance can be 2473 estimated relative to the number of events.104 Additional information on event counting is included in 2474 Subchapter 5.3 and Table 14. The LLOQ should be the lowest value when: 2475 2476 The entire population of interest remains within the gate, and the population cluster remains intact. 2477 The results for each replicate are above the LOD. 2478 The minimum number of events are present for all replicates. 2479 There is evidence of a titration effect. 2480 The acceptable precision achieved (usually ≤ 35% CV or as defined in the validation plan). 2481 2482 2483

H62

74


6.1.2.5 Precision 2484 2485 6.1.2.5.1 Experimental Design 2486 2487 As discussed in Chapter 3, a factorial design strategy in which multiple factors are evaluated within a single 2488 experimental design rather than each factor being evaluated in independent experiments is recommended 2489 for flow cytometric methods.33 2490 2491 It is recommended that precision is determined based on a minimum of three matrix samples (see Figure 2492 17). If samples are rare or if it is difficult to obtain matrix samples, two matrix samples may be sufficient. 2493 The precision evaluation should be performed with matrix samples that match the intended use samples and 2494 are tested within the sample stability window (see Subchapter 6.1.1.2 for more information). Although not 2495 recommended for repeatability (intra-assay), for reproducibility (interassay), an acceptable alternative to 2496 matrix samples is to use surrogate-preserved QC material that has been shown to exhibit appropriate 2497 behavior relative to real samples (such as stabilized whole blood).18,20 Sample stability and the closeness of 2498 the QC material to the actual samples should be taken into account when precision experiments are 2499 designed. Multiple runs may need to be conducted on the same day to avoid sample stability contributions 2500 to the precision evaluation. 2501 2502 Samples for precision studies should span a significant portion of the measuring range. A typical 2503 recommendation is three levels: low, medium, and high. For flow cytometric methods in which multiple 2504 cell types are reported, it is not possible to obtain samples with three levels of all reportable results. In such 2505 cases, several different donors should be used for the validation. Data from the LLOQ evaluation can be 2506 used for the precision assessment. The samples should be selected to span the expected analytical range. 2507 Three precision approaches are shown in Figure 17. The appropriate approach for a given validation is 2508 dependent on the intended use of the data and the associated regulatory environment. The approaches shown 2509 in panels A and B meet the minimal requirements for an FFP, whereas; the approach shown in Panel C is 2510 required for a full validation. See Appendix A for additional information. 2511 2512

2513 2514

H62

75


2515

2516 Abbreviation: R, run. 2517 Figure 17. Factorial Design Strategies for Precision Assessment 2518 2519 6.1.2.5.2 Statistical Analysis 2520 2521 Based on the intended use of the data, different statistical methods can be applied, and the proposed method 2522 should be specified in the validation plan. Most flow cytometry laboratories apply a simple statistical 2523 method; however, more robust statistical approaches may be required.20,107 Examples of two approaches are 2524 presented below. For a factorial study, ideally, the data are analyzed by a mixed-effect analysis of variance 2525 (ANOVA) model that can determine the variance components contributed by different factors. An example 2526 of a mixed-effect ANOVA model is shown in Table 15. In this type of analysis, both repeatability and 2527 reproducibility error (total error, the sum of all variance components) can be estimated from the model. The 2528 repeatability SD of within-run error (SDerror) should correspond directly to the residual or the error term 2529 variance component, and the reproducibility precision SD of total error (SDwl) should correspond to the 2530 sum of all four variance components (instrument, operator, run, repeatability error). Typically, these 2531 measures are re-expressed as % CV or relative SD, which is calculated by multiplying the ratio of the SD 2532 over the mean of the replicates by 100. A more in-depth description of this procedure is provided in 2533 Appendix D and CLSI document EP05.30 2534

H62

76


Mixed-model analysis requires advanced statistical skills and software that are not routinely accessible to 2535 all laboratories. A simpler alternative approach, described in more detail below, is to ignore the data 2536 structure (the factorial design) and calculate repeatability and reproducibility by pooling the data from all 2537 the runs. This data-generated method will be close to the more rigorous calculation of variance components 2538 from the random effect ANOVA described above. With this simple statistical method, repeatability can be 2539 determined by analyzing the variability of the within-run replicates for each sample in which the mean, SD, 2540 and % CV of each sample for each run are calculated. If the factorial design strategy is used and individual 2541 samples are assayed in multiple runs, the repeatability for each sample should be calculated as the mean % 2542 CV of each run. Finally, the overall repeatability should be reported as grand mean % CV across all samples 2543 and the range (minimum and maximum % CV). This strategy for repeatability is outlined in Table 16. 2544 2545 With the factorial design approach, a simple statistical method to determine reproducibility is to calculate 2546 for each sample the mean, SD, and % CV for all replicates from all runs for a sample. Then the overall 2547 reproducibility should be reported as grand mean % CV across all samples and the range (minimum and 2548 maximum % CV). This strategy for reproducibility is outlined in Table 17. 2549 2550 Table 15. Mixed-Effect ANOVA Model 2551

Sample ID Mean

Repeatability Error, SDerror

Reproducibility Error, SDwl

Repeatability CV

Reproducibility CV

Sample 1 μ σ σ , σμ

100% σ ,

μ100%


100% σ ,

μ100%


100% σ ,

μ100%

Abbreviations: ANOVA, analysis of variance; CV, coefficient of variation; ID, identification; SD, standard deviation; SDerror, 2552 standard deviation of within-run error; SDwl, standard deviation of total error. 2553 2554 Table 16. Calculating Repeatability Using the Factorial Design Strategy for Precision Assessment 2555

Sample ID

Run Number

Replicate 1

Replicate 2

Replicate 3 Mean SD % CV

Mean % CV per Sample

Sample 1 1

2 3 4

Sample 2 1

2 3 4

Sample 3 1

2 3 4

Grand mean % CV, range Abbreviations: % CV, coefficient of variation expressed as a percentage; ID, identification; SD, standard deviation. 2556 2557 2558 2559 2560 2561

H62

77


Table 17. Calculating Reproducibility Using the Factorial Design Strategy for Precision Assessment 2562

Sample ID

Run 1 Replicate Results



Run 4 Replicate Results Mean SD

% CV

Sample 1

Sample 2

Sample 3

Grand mean % CV Abbreviations: % CV, coefficient of variation expressed as a percentage; ID, identification; SD, standard deviation. 2563 2564 6.1.2.5.3 Precision Acceptance Criteria 2565 2566 When the precision acceptance criteria of the reportable results and the intended use of the assay are 2567 established, the frequency of the cell population and the number of events acquired within the reporting 2568 gate should be taken into account. Overall a desirable target for assay precision is a % CV below 10% to 2569 20% for most abundant populations, with up to 25% typically being acceptable (30% to 35% near the 2570 LLOQ). Higher imprecision for less abundant/rare or difficult-to-measure cell populations may be 2571 acceptable, but in such cases, justification must be provided. This justification should include the basis for 2572 how the assay, despite the imprecision, is an FFP in adequately quantifying the measurand.18,20 In some 2573 instances, when imprecision exceeds the above-recommended levels, it may be useful to report data in a 2574 semiquantitative fashion (eg, low, medium, high). 2575 2576 6.1.2.6 Linearity 2577 2578 As described in Subchapter 3.2.2, true linearity validation is only applicable for data that is categorized as 2579 definitive quantitative and relative quantitative. True linearity cannot be demonstrated with 2580 quasiquantitative data; nonetheless, there may be value in creating samples with various levels of the 2581 population of interest and analyzing the data on a linear regression plot. As with samples for sensitivity 2582 assessments, generating samples for linearity assessments can be difficult and often uses the same 2583 approaches of partially stained and/or admixed samples. Refer to Subchapter 6.1.2.3 for additional 2584 information. Linearity is covered in CLSI document EP06,112 and a review of this guideline for additional 2585 details on data analysis is recommended. Specific types of linearity assessments are outlined below. 2586 2587 6.1.2.6.1 Linearity for Quasiquantitative Data 2588 2589 If desired, the data from the LLOQ experiments can be plotted (observed vs expected) to evaluate linearity. 2590 Because there are no target (or expected) values, owing to the lack of calibration or reference material 2591 standards, true linearity cannot be evaluated; therefore, no recommended acceptance criteria are provided 2592 in this guideline. 2593 2594 6.1.2.6.2 Linearity for Receptor Occupancy and Receptor Modulation Assays 2595 2596 For RO and receptor modulation assays, the intent should be to describe the relationship between a drug 2597 and its target. Because the therapeutic compound is available for use as a reference standard in this case, 2598 the effect of concentration on the measurement of its target can be assessed for linearity. Thus, for RO 2599 assays, it is recommended that the effect of drug concentration on the readout be evaluated. In general, RO 2600 assays make use of a variety of formats and approaches that are outside the scope of this guideline but are 2601 extensively covered in Volume 90B of Cytometry B: Clinical Cytometry, which was dedicated entirely to 2602 RO.27,66,67 Validating linearity for RO assays or one of its component measures involves using appropriate 2603

H62

78


patient or healthy donor samples (the intent to test matrix) that are preincubated ex vitro with at least five 2604 drug concentrations ranging from partial to full saturation of the targeted receptor. In either development 2605 or validation, a high drug concentration sample should be used to assess the sensitivity of the assay to a 2606 prozone or hook effect. Various statistical methods can be used to fit a regression line to these data to allow 2607 a calculation of the deviation from linearity.113 It is common for a certain percentage of allowable 2608 nonlinearity to be applied, depending on the degree of polynomial and distance of the best fit from the 2609 observed values. Generally, linearity should fall within precision variance for at least 80% of the samples 2610 tested across the linearity range. 2611 2612 6.1.2.6.3 Linearity for Relative Quantitative Data 2613 2614 When an assay includes reporting fluorescence values that are derived from commercially available 2615 calibration beads, the linearity of the bead should be demonstrated. They must meet the manufacturer’s 2616 specifications.26 2617 2618 6.1.2.7 Stability 2619 2620 6.1.2.7.1 Specimen Stability 2621 2622 It is recommended that the stability assessment include a minimum of five samples.76 Ideally, testing should 2623 include at least one time point beyond when specimens are expected to be received at the testing laboratory 2624 or one time point past the acceptability limits. Samples should be tested as single replicates. Stability should 2625 be evaluated for every result that is reported. In addition, stability should be established at the latest time 2626 point where, for each reportable result with < 20% change and/or a change within initial assay imprecision, 2627 between an individual time point and the baseline sample is achieved for a minimum of 80% of the samples. 2628 When possible, the baseline should be tested within two hours of collection. Tables 18A and 18B can be 2629 adopted to record each result. 2630 2631 Some stability of the individual reportable results may not be the same. Each laboratory must follow its 2632 institution’s guidelines for handling and reporting specimens tested outside of the stability limits. The 2633 recommendation in CLSI document H422 is to always test irreplaceable specimens, such as bone marrow, 2634 and report the data with a comment. When specimen stability is established, both the percent change from 2635 baseline and the variance from the baseline should be calculated for each time point, as shown in Tables 2636 18A and 18B. 2637 2638 Table 18A. Percent Change 2639

Sample ID

Baseline Value Time Point 1 Time Point 2 Time Point 3 Time Point 4 Time Point 5

Result %

Change Result %

Change Result %

Change Result %

Change Result %

Change Result Sample 1 Sample 2 Sample 3 Sample 4 Sample 5 Meana

Abbreviations: % change, percent change from baseline; ID, identification. 2640 a The mean percent change across all samples should be calculated. 2641 2642 2643

H62

79


Table 18B. % CV 2644 Sample

ID Baseline

Value Time Point 1 Time Point 2 Time Point 3 Time Point 4 Time Point 5 Result % CV Result % CV Result % CV Result % CV Result % CV Result

Sample 1 Sample 2 Sample 3 Sample 4 Sample 5 Meana

Abbreviations: % CV, coefficient of variation expressed as a percentage; ID, identification. 2645 a The mean % CV across all samples should be calculated. 2646 2647 6.1.2.7.2 Processed Sample Stability 2648 2649 It is recommended that the prepared sample stability assessment include a minimum of three samples.76 2650 Ideally, disease-state samples should be included. The time points and storage conditions of the processed 2651 samples should reflect the actual conditions likely to occur in the laboratory. If the baseline and two time 2652 points are to be tested, the samples should be processed three times so that each staining is only acquired 2653 once, and each storage condition is held constant until the acquisition time. Samples are tested as single 2654 replicates. 2655 2656 Stability should be evaluated for every result reported, and it should be established at the latest time point 2657 where, for each reportable result with < 20% change and/or a change within initial assay imprecision, 2658 between an individual time point and the baseline sample, is achieved for a minimum of 80% of the samples. 2659 Ideally, the baseline time point should be tested within 60 minutes of processing. Tables 18A and 18B can 2660 be adopted to record the results. 2661 2662 6.1.2.7.3 Antibody Cocktail Stability 2663 2664 A cocktail stability validation study can be accomplished by comparing the results from the original cocktail 2665 prepared at day 0 with the results of a freshly prepared cocktail using paired samples. Alternatively, the 2666 results from the original cocktail can be compared with the individually pipetted single antibodies using 2667 paired samples. A best practice is to include both nondiseased and diseased samples (if applicable) for the 2668 comparative study. Cocktails should be evaluated every week or biweekly for the duration of the cocktail 2669 stability study.107 Refer to the table in Appendix E for data evaluation. The acceptance criteria are as 2670 described above for specimen stability. 2671 2672 6.1.2.8 Assay Carryover 2673 2674 Instrument carryover is covered in Subchapter 4.2.5. Although this approach works well when the same 2675 assays are run repeatedly on different patient samples, in standard practice the preceding assay is often 2676 undefined and may be a completely different test that can present a higher or lower risk for carryover. 2677 Additionally, many flow cytometric assays differ substantially from single analyte assays in that they either 2678 monitor multiple measurands or report a single measurand that may be complex and composed of various 2679 possible staining patterns that must first be recognized by the analyst. For such situations, following 2680 traditional carryover approaches for each possible combination may be impractical. Even for a simple 2681 measurand, obtaining samples with low and high levels for each measurand may not be reasonably feasible; 2682 however, assessment of carryover can often be adequately assessed by running blank samples after normal 2683 or elevated concentration samples or a calibration standard. Carryover by this method is determined by 2684 event count in the blank sample and can be extrapolated to that which is anticipated for a sample at the 2685 expected functional upper limit of that test. This extrapolated ULOQ carryover count should be sufficiently 2686 low that when applied to representative LLOQ situations, they will minimally not exceed 20% of the LLOQ 2687 value. For example, if 20 events are collected in the measurand gate, and the sample is 80% of the expected 2688

H62

80


upper limit for the measurand that is expected in the intended test population, one can apply the presumption 2689 that up to an additional 20% (five) carryover events can be observed at the upper limit of normal. Thus, for 2690 samples with low measurand in the validation study, the addition of 25 events should not change the 2691 reported value by more than 20%. This method requires that the event counts are recorded across the testing. 2692 As with the approach shown in equation (7), triplicates of the test sample should be prepared and run 2693 following the laboratory’s established protocol. Carryover is a function of instrument noise, flow rate, and 2694 volume (conveniently calculable as flow rate time). Therefore, if the blank sample approach is used, it is 2695 critical that flow rates and volumes that run through the instrument in the blank sample are representative 2696 of the maximally allowable volumes that will run through the machine in the performance of the assay. 2697 2698 Additionally, results need to be expressed as event counts relative to time. For example, the measurand 2699 might be 2% within a gating scheme, but what matters is how many events (event count X) are minimally 2700 collected to achieve that value under the maximum volume run. Passing carryover by running a blank 2701 sample requires that the number of events falling within that gate does not exceed 20% of the event count 2702 X. 2703 2704 A slight variation on the concept described above is to monitor carryover from population gates that are 2705 unlikely to differ between assays within a given matrix, such as those that are the product of FSC/SSC and 2706 doublet discrimination gates. If the carryover events from this type of a gate are low enough that they are 2707 unlikely to affect determinations in LLOQ samples (eg, less than 20% of the LLOQ), additional 2708 demonstration that carryover is controlled becomes unnecessary because a subgate of a population cannot 2709 have more carryover events than its parent gate. Although this approach may need more stringent “washes” 2710 between samples than are necessary when the assay-specific approach is followed, it is much more 2711 amenable to validation at the instrument performance level. If this type of an approach is taken, the 2712 demonstration that carryover is controlled in analytic assay validation should include citing the instrument 2713 performance characterization testing following determination that the stained test samples do not differ 2714 substantially in carryover characteristics relative to those used for establishing and monitoring instrument 2715 performance (eg, FSC/SSC and doublet discrimination carryover from the intent to test samples is not so 2716 qualitatively or quantitatively different as to prevent extrapolation of carryover findings). 2717 2718 6.1.2.9 Reference Intervals 2719 2720 It is often useful or necessary to establish reference intervals for measurands in flow cytometric methods. 2721 This process is usually done after the analytic validation has been completed. For all flow cytometric 2722 methods requiring reference intervals for interpretation, each laboratory is required to establish or confirm 2723 reference intervals. CLSI document EP2835 is a comprehensive guideline for establishing reference 2724 intervals for new tests as well for verifying the applicability of previously established reference intervals 2725 for transferred tests. Because this guideline applies to flow cytometry testing, the reader is advised to consult 2726 it if a reference interval needs to be established for a flow cytometric method. 2727 2728 6.1.3 Validation Plans for Qualitative Data/Methods 2729 2730 In the clinical testing environment, flow cytometry is often used to assist in the diagnosis and classification 2731 of hematolymphoid neoplasms, such as leukemia, lymphoma, and plasma cell myeloma. 2732 Leukemia/lymphoma immunophenotyping assays rely on knowledge of the composition and 2733 immunophenotype of normal or reactive hematopoietic and lymphoid cells in different specimen types and 2734 the ability to recognize abnormal (neoplastic) populations that differ from normal populations. The presence 2735 or absence of an abnormal neoplastic population should be reported categorically (present or absent), along 2736 with a description of the immunophenotype of any abnormal populations. Although the relative percentage 2737 of neoplastic cells is often reported, it is not used for clinical decision making; therefore, 2738 leukemia/lymphoma immunophenotyping assays are considered “qualitative.” To validate qualitative 2739 assays, an approach that differs from the one used to validate quantitative assays should be used. 2740

H62

81


6.1.3.1 Accuracy/Trueness 2741 2742 For qualitative assays, such as leukemia/lymphoma immunophenotyping assays, accuracy validation 2743 involves evaluating whether the test successfully identifies, within statistical bounds, the presence or 2744 absence of an abnormal/neoplastic population.107,114 Accuracy represents the degree of concordance 2745 (agreement) of the results of the assay in comparison with another valid method. Historically, the reference 2746 standard for diagnosing leukemia and lymphoma has been morphology. The evaluation of morphology 2747 requires stained smears or tissue sections to be reviewed by a pathologist trained in the diagnosis of 2748 hematolymphoid neoplasms. When tissue sections are available, the morphologic assessment may also 2749 include immunohistochemical staining, which can assist in the recognition of differences from normal and 2750 the determination of the immunophenotype of neoplastic cells. 2751 2752 Accuracy in the validation of qualitative leukemia/lymphoma immunophenotyping assay should be 2753 assessed by splitting samples and comparing flow cytometric results with predicate validated flow 2754 cytometric assay performed in the same laboratory, a validated flow cytometric assay performed in another 2755 laboratory, or morphologic findings. When this comparison is performed, a variety of the expected types 2756 of neoplasm and non-neoplastic/reactive specimens, including all the specimen types that will be evaluated 2757 using this assay (eg, peripheral blood, bone marrow, tissue, or body fluid), should be used. Also, when 2758 validation specimens are selected, it is important to consider circumstances that can have an effect on 2759 specificity and sensitivity of the assay. The accuracy comparison should be performed on two parameters: 2760 presence or absence of a neoplastic population and expression of each of the markers used in the panel (ie, 2761 CD19+

= CD19+ on lymphoid population). For both parameters, accuracy should be established using a 2762 concordance table, as shown in Table 19. As described in CLSI document EP12,23 the acceptance criterion 2763 is 100% concordance for assays that report qualitative results. 2764 2765 This evaluation also determines the diagnostic/clinical accuracy of the assay. Although this is primarily a 2766 qualitative assessment, if comparing with another validated flow cytometric assay, it may be of interest to 2767 compare the percentage of cells in populations of interest and their staining intensity. These numerical 2768 determinations can help to highlight differences between the assays that might lead to diagnostic challenges 2769 with some diseases/specimen types. For example, a test with weaker intensity staining for a particular 2770 marker may be less able to detect differences from normal. 2771 2772 Qualitative data are first reviewed in a concordance table, as shown in Table 19. The data are then used to 2773 calculate concordance (or accuracy), as shown in equation (12), specificity as shown in equation (13), and 2774 sensitivity as shown in equation (14). 2775 2776 2777

H62

82


Table 19. Concordance Table 2778

Candidate Method

Comparative Method

Total Positive Negative Positive A (true positives) B (false negatives) A + B

Negative C (false positives) D (true negatives) C + D

Total A + C B + D A + B + C + D

2779

Concordance A D A B C D 100% 2780 2781 Where: 2782 A = true positives 2783 B = false negatives 2784 C = false positives 2785 D = true negatives 2786 2787 6.1.3.2 Specificity 2788 2789 Specificity in the validation of qualitative leukemia/lymphoma immunophenotyping assay (or in assays 2790 with binary classification) can be determined with data obtained from the previously described accuracy 2791 dataset and by using equation (13). 2792 2793

Specificity D D C 100% 2794 2795 Where: 2796 C = false positives 2797 D = true negatives 2798 2799 When specimens for the validation studies are selected, it is important to include circumstances that can 2800 interfere with the detection of disease, such as prior immune therapy (ie, anti-CD20 monoclonal antibody 2801 therapy), that might alter the immunophenotype of reactive and neoplastic cells and even eliminate key 2802 markers essential for gating. Consideration should also be given to anticipated differences between diseases, 2803 such as the ability to detect the weak staining for CD20 and surface immunoglobulin characteristic of 2804 chronic lymphocytic leukemia, as well as specimen types, such as body fluids, which show a frequent lack 2805 of specific staining for surface immunoglobulin.115 2806 2807 6.1.3.3 Sensitivity 2808 2809 Sensitivity in a qualitative assay (or in assays with binary classification) can be determined with data 2810 obtained from the previously described accuracy dataset and by using equation (14): 2811 2812

Sensitivity A A B 100% 2813 2814 Where: 2815 A = true positives 2816 B = false negatives 2817 2818 Sensitivity determined this way provides the diagnostic/clinical sensitivity of the assay and does not refer 2819 to the ability to detect low levels of involvement by the disease that is the assay’s LOD, as discussed in 2820 Subchapter 6.1.2.3. 2821 2822

(12)

(13)

(14)

H62

83


Although leukemia and lymphoma immunophenotyping is a qualitative (binary) assay, there may also be a 2823 need to verify/identify the LOD for a given neoplastic population. This LOD should be set during assay 2824 development and optimization based on the sensitivity desired by the laboratory. Generally, the LOD is 2825 related to the number of events acquired for each of the populations reviewed. For example, if one requires 2826 a 0.1% detection level for B cells, at least 50 000 leukocytes should be acquired, and > 50 CD19+ 2827 lymphocytes should be present to verify this recovery. For leukemia/lymphoma immunophenotyping assays 2828 that are not designed to measure MRD, recommended optimal LOD values for different cell lineages are as 2829 shown below. Refer to Table 14 for guidance on the number of events that must be acquired to achieve this 2830 level of sensitivity. 2831 2832

Myelomonocytic 0.5% 2833 T cells 1% 2834 B cells 0.1% 2835 Plasma cells 0.1% 2836

2837 As part of the sensitivity assessment, it is necessary to verify that the desired LOD for each cell lineage and 2838 neoplastic cells can be recovered in the assay. To do this, the abnormal cells (T cells, B cells, 2839 myelomonocytic cells, or plasma cells) can be spiked into a normal sample at dilutions that approach the 2840 desired LOD. The appropriate number of leukocytes should be acquired to obtain such a LOD. The presence 2841 of the cell lineage of interest should be recovered at the target LOD. Because this is a qualitative assay, it 2842 is not necessary to determine the LLOQ. When specimens for the validation studies are selected, it is 2843 important to include circumstances that can lead to false negative results, such as the ability to detect weak 2844 intensity staining of CD10 for germinal center B-cell lymphoma. 2845 2846 6.1.3.4 Precision 2847 2848 To account for significant factors that may cause variability in the assay, including but not limited to testing 2849 multiple samples, runs, reagent lots, and operators, precision testing must be conducted. For leukemia and 2850 lymphoma samples, it is not recommended to run samples on multiple days because of the limited stability 2851 of the sample, but it is recommended to conduct multiple runs on the same day or within the demonstrated 2852 stability time frame (eg, 24 or 48 hours). If applicable, testing that accounts for other sources of variability, 2853 such as multiple instruments and multiple testing sites, should be conducted. 2854 2855 Table 20 shows a reproducibility assessment of five samples and 20 runs (5 4 conditions/variables) for 2856 qualitative leukemia/lymphoma immunophenotyping assays. In this example, two nonmalignant cases and 2857 three malignant cases with different immunophenotypes are assayed by two different operators and in two 2858 different analytical runs on the same day. The final diagnosis based on the immunophenotype for each run 2859 is equivalent. Statistically, all results agreed for the 20 separate runs (N = 20). Concordance = (20 / 20) × 100 2860 = 100% established that reproducibility is concordant. Another example of qualitative reproducibility is 2861 included in Appendix F. Acceptance criteria should be justified using objective evidence and statistical 2862 threshold. In this case, concordance with replicate results is used to confirm reproducibility. 2863 2864

H62

84


Table 20. Qualitative Reproducibility 2865 Run 1 Run 2

Sample ID

Operator 1

Operator 2

Operator 1

Operator 2 Concordance

1 Abnormal CLL Abnormal CLL Abnormal CLL Abnormal CLL Yes 2 Normal Normal Normal Normal Yes 3 Abnormal

ALL(B) Abnormal ALL(B)

Abnormal ALL(B)

Abnormal ALL(B)

Yes

4 Abnormal AML Abnormal AML Abnormal AML Abnormal AML Yes 5 Normal Normal Normal Normal Yes

Abbreviations: ALL(B), acute B-lymphoblastic leukemia; AML, acute myeloid leukemia; CLL , chronic lymphoblastic leukemia 2866 with lambda light-chains; ID, identification. 2867 2868 6.1.3.5 Linearity 2869 2870 Because linearity is not applicable in the validation of qualitative methods, it is not covered in this guideline. 2871 2872 6.1.3.6 Stability 2873 2874 When sample stability is established for qualitative assays, a minimum of five samples assayed at different 2875 time points postcollection should be included. If possible, both diseased and nondiseased samples should 2876 be evaluated. Acceptability criteria should be determined by categorical concordance (see Table 21). 2877 2878 Table 21. Concordance Table for Stability of Qualitative Data 2879

Sample ID Baseline

Time Point 1

Time Point 2

Time Point 3

Time Point 4 Concordance

1 Normal Normal Normal Normal Normal Yes 2 Abnormal

CLL Abnormal CLL

Abnormal CLL

Abnormal CLL

Abnormal CLL

Yes

3 Myeloma c Myeloma c Myeloma c Myeloma c Myeloma c Yes 4 Normal Normal Normal Normal Normal Yes 5 AML AML AML AML AML Yes 6 ALL(T) ALL(T) ALL(T) ALL(T) ALL(T) Yes

Abbreviations: ALL(T), acute T-lymphoblastic leukemia; AML, acute myeloid leukemia; cK, cytokeratin; CLL , chronic 2880 lymphoblastic leukemia with lambda light-chains; ID, identification. 2881 2882 6.1.3.7 Cocktail Stability 2883 2884 Cocktail stability should be validated for qualitative assays. Acceptability criteria depend on the assay and 2885 may be either categorical concordance or change in MdFI. Refer to Subchapter 6.1.2.6 and Appendix E for 2886 more information. 2887 2888 6.1.3.8 Assay Carryover (Instrument) 2889 2890 Refer to Subchapter 6.1.2.7 for more information about assay carryover. 2891 2892 6.1.3.9 Reference Intervals 2893 2894 Refer to Subchapter 6.1.2.8 for more information about reference intervals. 2895 2896

H62

85


6.2 Validation Implementation (Do It!) 2897 2898 When the validation is implemented, the experiments described in the validation plan should be conducted. 2899 Careful records should be maintained by all laboratories during all types of validation. The reagent lot 2900 numbers, instrument acquisition details, etc., must be recorded. This information is critical when the final 2901 SOP is created and when any failed runs or erroneous data need to be troubleshot. 2902 2903 6.3 Validation Reports (Prove It!) 2904 2905 The validation report presents a complete summary of the validation data and a conclusion regarding the 2906 suitability of the method for its intended use. Typically, a validation binder that contains copies of the final 2907 gated data and appropriate reagent and specimen documentation should be prepared. The documentation 2908 practices and signatory requirements for the validation binder and validation report depend on the 2909 regulatory requirements of the laboratory, the intended use of the data, and institutional policies. Typically, 2910 the validation report should follow a similar structure to the validation plan and contain the following 2911 information: 2912 2913 Any deviations from the method procedure or validation plan 2914 2915 A list of all reagents, including the manufacturer(s), catalog(s), and lot number(s) 2916 2917 A list of all equipment, including the manufacturer(s), model(s), and serial number(s) 2918 2919 A list of all software, including the manufacturer(s) and version number(s) 2920 2921 A description of the validation samples and QC materials 2922 2923 A clear summary (eg, text, figures, and data tables) of the outcome of the evaluation of each validation 2924

parameter 2925 – Individual results should be presented in an appendix 2926

2927 – A description outlier statistic applied, if applicable. When outliers are identified, a justification 2928

must be provided, and the original data must be presented 2929 2930

– A description of data that did not meet the acceptance criteria and the course of action taken 2931 2932 A list of staff who conducted the various aspects of the validation 2933 2934

H62

86


6.4 Validation Plans Based on Intended Use and Regulatory Requirements 2935 2936 Because the intended use of flow cytometric methods is broad (ie, basic research to CDx), a single 2937 validation strategy is not appropriate. A validation strategy appropriate for the intended use should be 2938 applied. Table 22 lists many different applications of flow cytometric methods and the appropriate 2939 validation strategy. Appendix A contains details on how to complete each recommended strategy. 2940 2941 Table 22. Validation Scenarios Based on Intended Use 2942

Regulatory Setting Intended Use of Data Assay Type Recommended

Validation Strategy Nonregulated Basic research Novel assay FFP validation type 1

Nonregulated Drug discovery Novel assay FFP validation type 1

Nonregulated Exploratory end points in clinical trials

Novel assay FFP validation type 1

Nonregulated (GCLP recommended)

Secondary end points in clinical trials

Novel assay FFP validation type 2

Clinical laboratory (CAP, CLIA, or ISO15189)

Patient care and/or treatment IVD Verificationa

Clinical laboratory (CAP, CLIA, or ISO 15189)

Patient care and/or treatment Qualitative LDT assay

CLIA/IMDRF qualitative validation


Patient care and/or treatment Quantitative LDT assay

CLIA/IMDRF quantitative validation


Patient care and/or treatment Laboratory- initiated assay revision

Laboratory initiated assay revision

GLP, GCLPb Primary end point in clinical development

Novel assay Full validation type 1

Manufacturing (GMP)

Regulatory submission for new diagnostic test

Novel assay Full validation type 2

Manufacturing (GMP)

CDx Novel assay Full validation type 2

Abbreviations: CAP, College of American Pathologists; CDx, companion diagnostic; CLIA, Clinical Laboratory Improvement 2943 Amendments of 1988; GCLP, good clinical laboratory practice; GLP, good laboratory practice; GMP, good manufacturing practice; 2944 IMDRF, International Medical Device Regulators Forum; ISO, International Organization for Standardization; IVD, in vitro 2945 diagnostic; LDT, laboratory-developed test. 2946 a Verifications are performed when setting up an IVD assay, which the manufacturers have already validated. Verifications require 2947 that the laboratory follow all documented procedures that accompany the validated assay. The same documentation outline should 2948 be followed for verification. 2949 b The regulatory requirements for primary end point data are not clear. Most laboratories will follow GLP. 2950 2951 2952

H62

87


Chapter 7: Examination and Postexamination Phase 2953 2954

This chapter includes information on: Instrumentation setup Reagent verification Examination phase Postexamination phase File storage and retention Training records and competency Accreditation EQA/PT

2955 Before testing samples (examination phase) and releasing results (postexamination phase), the laboratory 2956 must have QMS in place, a qualified instrument, and an optimized assay that has been validated 2957 appropriately for the intended use. These topics are covered in detail in Chapters 2 to 6. The QMS specific 2958 to the examination and postexamination phases include monitoring the environment, the instrument, the 2959 reagents, and the assay. This chapter focuses primarily on aspects of QA and QC procedures that are unique 2960 to assays performed by flow cytometry. Table 23 highlights these different recommendations. Operations 2961 common to all laboratories, such as environmental monitoring, is not discussed. 2962 2963

H62

88


Table 23. QC Overview for Flow Cytometry 2964

Considerations Recommended QC H62

Chapter In

stru

men

t

Beads – voltage Daily QC should be performed, and Levey-Jennings plots should be checked daily.

Chapter 7

Beads – alignment Daily QC should be performed, and Levey-Jennings plots should be checked daily.

Chapter 7

Beads – linearity QC should be performed 2 times per year, unless the type of instrument does not require linearity check.

Chapter 7

Beads – sensitivity QC should be performed 2 times per year. Chapter 7

Compensation setup

A monthly assay control should be run (ie, each assay should be set up, and performance of all steps should be verified based on criteria of acceptability as determined by the laboratory).

Chapter 4

Instrument-to- instrument

A monthly assay control should be run on all flow cytometers used for this assay.

Chapter 4

Pre

exam

inat

ion

Specimen preparation

A monthly assay QC should be set up to verify assay performance (all steps of the test); the assay QC needs to meet the criteria of acceptability as determined by the laboratory/vendor.

Chapter 5

Reagent QC QC should be performed on all incoming reagents and an assay QC should be set up monthly to verify assay performance; the comparison needs to meet criteria of acceptability as determined by the laboratory/vendor.

Chapter 5

Antibody QC New antibodies should be titered and validated; in-use antibodies should undergo lot-to-lot comparison; this comparison needs to meet criteria of acceptability as determined by the laboratory/vendor.

Chapter 5

Lysing reagent QC

QC should be performed on new lysing reagents and meet standards of acceptability as defined by the laboratory/vendor.

Chapter 5

Val

idat

ion

Background (LOB)

Initial validation should be conducted. Chapter 6

Assay sensitivity (LOD and LLOQ)

Initial validation should be conducted. Chapter 6

Accuracy/precision Initial validation should be conducted. Chapter 6

Reproducibility Initial validation should be conducted. Chapter 6

Reportable ranges Initial verification should be conducted. Chapter 6

Exa

min

atio

n

Confirmation of QC results

Review by supervisor/manager should occur monthly. Chapter 6

Compensation check

Each panel tube run should be assessed for compensation issues and compared with expected patterns.

Chapter 7

External controls Required before each patient sample for specific assays (eg, CD4, fetal hemoglobin); recommended monthly assay verification on a sample obtained from a healthy donor (for leukemia/lymphoma)

Chapter 7

Internal controls Each patient run should provide internal controls, and verification of performance should be documented.

Chapter 7

Training and competency

Each employee’s training and competency assessment needs to be documented.

Chapters 2 and 7

Abbreviations: LLOQ, lower limit of quantification; LOB, limit of blank; LOD, limit of detection; QC, quality control. 2965

H62

89


7.1 Instrumentation 2966 2967 The processes for instrument IQ, OQ, and PQ are described in detail in Chapter 4. Each laboratory must 2968 also establish procedures for daily set up/calibration, maintenance, and monitoring, including longitudinal 2969 performance. These processes must be defined in SOPs that describe the steps to be followed for frequency, 2970 tolerance limits, troubleshooting, review, and documentation. 2971 2972 7.1.1 Alignment 2973 2974 Fluorescent particles should be used for the daily monitoring of CV for the fluorescent channels. Typically, 2975 the CV of the histogram for the beads in each detector is monitored. The laboratory should follow 2976 manufacturer specifications for acceptability. A laboratory may establish its own specifications provided 2977 that they are not less stringent than those recommended by the manufacturer and that they are defined in an 2978 SOP. 2979 2980 7.1.2 Calibration 2981 2982 The process and frequency of instrument calibration should be instrument-specific and should follow the 2983 manufacturer’s recommendations. Most analyzers include software for daily setup and monitoring. The 2984 optical alignment, laser power, and photomultiplier tube (PMT) voltages should be monitored. Levey-2985 Jennings charts are the preferred method for reviewing performance characteristics, detecting instrument 2986 drift, and laser decay. 2987 2988 7.1.3 Linearity 2989 2990 Fluorescent particles should be used to measure each detector’s linear range (see Chapter 4 for more 2991 information). Instrument linearity should be verified on instrument installation and following service or 2992 repair. The frequency and acceptance criteria for linearity evaluation will be determined, in part, by the 2993 regulatory requirements to which the laboratory adheres. 2994 2995 7.1.4 Sensitivity 2996 2997 On instrument installation or following significant service, the laboratory should establish the instrument 2998 resolution, sensitivity, background noise, and carryover. The frequency of monitoring these parameters 2999 (annually, semiannually, etc.) should follow the regulatory requirements and the instrument manufacturer’s 3000 recommendations. Verification of these parameters is also required after service or repair. Some instruments 3001 provide these data as part of daily QC reports. 3002 3003 7.1.5 Compensation 3004 3005 Compensation is discussed at length in Subchapter 4.4. The frequency of compensation and timing (ie, pre- 3006 or postacquisition) depends on the type of instrument and software as well as the processes used for daily 3007 set up. The specific compensation procedure used depends on the individual fluorophore and assay 3008 requirements. Procedures for compensation frequency and verification should be described in an SOP. 3009 3010 7.2 Reagents 3011 3012 7.2.1 Specimen preparation reagents 3013 3014 Specimen preparation reagents are considered to be the reagents used during specimen preparation, such as 3015 buffers, lysing solutions, and fixatives. All specimen preparation reagents should be assessed for optimum 3016 performance during assay development and optimization, as described in Chapter 5. Once these 3017 performance criteria are established, every lot must be verified that it is within the acceptable tolerance. 3018

H62

90


7.2.2 Monoclonal Antibodies 3019 3020 Antibodies are initially titrated during assay development and optimization, as described in Chapter 5. New 3021 lots of antibodies must be verified against a current lot that is being used before being deployed. Each 3022 laboratory must establish acceptance criteria for performance including expected staining patterns and 3023 MdFI. The acceptance criteria depends on the assay type. 3024 3025 7.2.3 Antibody Cocktails 3026 3027 There are many reasons why it is advantageous to precombine or “cocktail” antibodies. The stability of 3028 “cocktailed” antibodies must be determined empirically for each cocktail. This is particularly important 3029 when detecting antigens that may have low-density expression. Individual fluorophores have different 3030 stability; therefore, it is important to establish the stability for each antibody/fluorophore in the cocktail. 3031 Each antibody in the cocktail must have the same staining activity as if the antibody is used as a single 3032 reagent. A study should be performed to determine the length of time (in days) that the cocktail continues 3033 to stain in the same manner as when it was first prepared. See Appendix E for an example table.107 Each 3034 time a new cocktail is made with current antibody lots, it should be compared with a cocktail made with 3035 the prior lot. 3036 3037 7.2.4 Quality Control Samples 3038 3039 The use and frequency of running controls, as well as the acceptance criteria, depend on the regulatory 3040 environment, the intended use of the data, and the type of assay. Some assays require external controls, 3041 while others may rely on internal controls. In medical and other regulated laboratories, each analytical run 3042 should include QC samples.22 The laboratory must establish the mean target values and SD for each lot of 3043 commercial, or in-house developed, QC material. If reagents are supplied with QC ranges, the laboratory 3044 must still establish its ranges that must be within the manufacturer’s specified ranges. A new lot is run 3045 alongside the current lot to verify continued acceptable performance. Refer to CLSI document H422 for 3046 more information about establishing QC ranges. QC results should be checked and verified by experienced 3047 laboratorians in the period specified by laboratory procedures. In daily practice, any control-result that is 3048 out-of-range needs to be investigated. Sample results are valid only when the batch control results are within 3049 the defined range. The QC results should also be evaluated for longitudinal performance in a Levey-3050 Jennings graph. Both external and internal controls can be used to verify the performance of reagents and 3051 staining procedures, but they “control” different aspects of the flow assays. 3052 3053 External controls are an integral part of every laboratory assay because they are used to “control” the 3054

overall assay performance, including instrument settings, antibody performance, and specimen 3055 preparation. Beads are typically used to verify instrument performance. Commercially available QC 3056 material or patient samples (eg, bone marrow, peripheral blood, tissue) may be used as external controls 3057 to verify antibody and reagent performance at least monthly. An approximate range of acceptable 3058 percentages and/or MdFI can be used to determine if the performance of the antibody meets the 3059 acceptability criteria as determined by the laboratory. 3060

3061 Internal controls serve a different role because they are used to verify the assay performance within 3062

the actual patient sample. For example, residual normal cells present within the patient’s sample can be 3063 used and should be a part of the interpretation of the patient’s results. For example, antibodies against 3064 the CD10 antigen can be verified as working and being present by examining that CD10 staining is 3065 present on mature neutrophils. The use of the internal control allows the individual examining the dot 3066 plots to identify possible abnormalities within the patient’s sample. A targeted design of “control plots” 3067 for each panel tube ensures that the correct antibodies or cocktails are added and verifies the 3068 performance of each antibody. The expected antibody performance on internal controls can then be 3069 documented for each patient. 3070

3071

H62

91


Leukemia/lymphoma immunophenotyping assays typically do not have commercially-available external 3072 controls for all antibodies tested, and the use of a healthy donor with each patient sample is not practical 3073 for many laboratories. Nonetheless, antibody performance needs to be established and verified on a defined 3074 regular basis before patient testing. Internal controls (cells in the test specimen that are negative or positive 3075 for the test antigen) may be used to verify the staining process for most antibodies in the panel.22 Populations 3076 within the specimen that are not expected to express a given antigen can serve as a negative control and 3077 used to establish the positive:negative staining boundary and the degree of autofluorescence. Internal 3078 negative control cells can be the same cell type as the target cells or a different cell type. For example, 3079 B cells can serve as a negative control when studying CD3 (same cell type, ie, lymphocyte); neutrophils 3080 (distinct from lymphocytes) can also be a negative control when studying CD3. 3081 3082 7.3 Examination Phase 3083 3084 Before starting a specific method procedure (sample staining and acquisition), the following steps should 3085 be completed: 3086 3087 1. Instrument setup 3088 2. Reagent verification 3089 3. Specimen receipt and verification 3090 4. Processing and acquisition 3091 3092 These key steps are highlighted in Figure 18. 3093

H62

92


3094 Abbreviations: PMT, photomultiplier tube; SOP, standard operating procedure. 3095 Figure 18. Examination Phase Process Map 3096 3097 7.3.1 Instrument Setup 3098 3099 When setting up the instrument, the following steps should be followed: 3100 3101 1. Document and perform required maintenance per schedule. 3102 3103 2. Perform instrument start-up procedure per SOP. 3104 3105 3. Run instrument-specific alignment beads; compare with expected values for data plots and all 3106

fluorescent channels. Proceed if in range; perform troubleshooting or call for service if specifications 3107 are not met. 3108

3109 4. Run instrument-specific beads/cells using the assay-specific protocol/experiment. Adjust PMT voltages 3110

as needed to ensure that each fluorescent peak appears in the assigned target channel using automated 3111 software or manual adjustment as appropriate for the particular instrument. 3112

3113 5. Review the Levey-Jennings plots to ensure PMT values are within the established range. 3114 3115 6. Perform and verify compensation, when applicable. 3116 3117 7.3.2 Reagent Verification 3118 3119 To verify the reagent, the following steps should be followed: 3120 3121 1. Handle all reagents per SOP. 3122 3123 2. Gather the reagents for the assay per SOP. 3124 3125 3. Verify that the reagent lot is not expired, has been parallel tested, and meets acceptance criteria 3126

(acceptable for use) to include cocktails. 3127 3128

H62

93


4. Prepare the in-house reagents per SOP and document dates of creation, use, and expiration. 3129 3130 7.3.3 Quality Control Verification 3131 3132 If using an assay QC, it may be tested before testing or along with the patient samples, depending on the 3133 laboratory practices. If using an assay QC, results should not be released until it has been verified that the 3134 QC met the acceptance criteria. 3135 3136 7.3.4 Sample Receipt and Verification 3137 3138 On receipt of the sample, the following steps should be followed: 3139 3140 1. Verify the test order and that the required specimen type, age, temperature, and anticoagulant meet 3141

expected sample requirements. 3142 3143 2. If any information is missing, the samples may be tested, but the data cannot be reported until after 3144

completing clarification. 3145 3146 3. Review all the samples for general sample integrity, including hemolysis and clots. 3147 3148 4. Label the sample per laboratory procedure. Label the daughter tubes per laboratory procedure, if 3149

appropriate. 3150 3151 5. Test the sample viability per laboratory procedure, if applicable. 3152 3153 7.3.5 Sample Processing and Acquisition 3154 3155 To process the sample after acquisition, the following steps should be followed: 3156 3157 1. Verify the sample labels against the worksheet. 3158 3159 2. Process the samples, (stain, lyse, etc.) per assay SOP. Include the required QC material designated per 3160

the assay or as required per SOP. 3161 3162 3. Verify that the appropriate tubes are in the respective templates. 3163 3164 4. Verify that the correct acquisition template, instrument settings, and compensation matrix are applied. 3165 3166 5. Acquire the assay tubes using the preestablished (and verified) acquisition template. 3167 3168 7.4 Postexamination Phase 3169 3170 There is a wide variety of flow cytometric assays used for a number of purposes. It is essential to understand 3171 that it is the intended use of the assay that drives the need for assay-specific quality procedures. Once the 3172 assay-specific criteria of acceptability are defined, any failures that are observed need to be investigated, 3173 and sample results cannot be issued until all requirements have passed. The examples shown in Figure 19 3174 provide recommendations based on the type of data reported from a method. Many tests, such as PNH, 3175 report both quantitative and qualitative data. Figure 19 describes the steps for run review and acceptance, 3176 which should be performed during the postexamination phase. 3177 3178

H62

94


3179 Abbreviations: L-J, Levey-Jennings; QC, quality control. 3180 Figure 19. Postexamination Phase Process Map 3181 3182 3183

H62

95


7.4.1 Run Review and Acceptance 3184 3185 During the run review and acceptance process, the following steps should be followed: 3186 3187 1. Review the data for acceptable staining (all reagents added). 3188 3189 2. Review the data for expected staining patterns. 3190 3191 3. Review the data to ensure a minimum number of events has been acquired. 3192 3193 4. Transfer the acquisition data to a preestablished analysis layout/template. 3194 3195 5. Analyze the sample and QC data per SOP. 3196 3197 6. Compare the QC results with expected range. If the QC results are out of range, troubleshoot the assay. 3198

Do not report results until resolved. 3199 3200 7. Document the QC values and create the Levey-Jennings charts, if required. 3201 3202 8. Review the Levey-Jennings charts for trends/shifts per SOP and take corrective action as needed per 3203

the criteria. 3204 3205 9. Some assays have sample-specific quality criteria, such as lymphosum for lymphocyte subsets, 3206

minimum cell count, cell gate purity, and internal controls. Follow the specific SOP and document the 3207 results. 3208

3209 10. Follow the laboratory SOP for the second review or the pathologist review. 3210 3211 7.4.2 Reporting Relative and Quasiquantitative Data 3212 3213 Most flow cytometric methods reporting numeric values are considered quasiquantitative, while some 3214 assays also provide relative quantitative values, as discussed in Subchapter 3.1.1. Quasiquantitative data 3215 are generated from lymphocyte subset assays, CD34 enumeration, fetal hemoglobin cell enumeration, and 3216 platelet enumeration assays. Relative quantitative results are generated from assays in which the level of 3217 antigen expression is reported, such as CD64.116 Laboratory reports for relative quantitative and 3218 quasiquantitative methods should include: 3219 3220 Numeric reportable results (relative percentage, cell counts, MFI, MdFI, MESF, ERF, ABC, etc.) for 3221

each cellular population, antigen, or antigen combination analyzed 3222 3223

The reference range for relevant reportable results, if applicable 3224 3225 For leukemia/lymphoma immunophenotyping assays, a percentage of the abnormal population either 3226

of the target population (eg, percent clonal B cells of the lymphoid cells in a lymph node) or of the 3227 total cells within a sample (eg, percent blasts of total peripheral blood or bone marrow cells) 3228

3229 When results are less than the LLOQ, results should not be reported as definitive or diagnostic. It is 3230

acceptable to report that a suspicious population has been identified but that the values are below the 3231 quantitative limits and thus should not be used for diagnosis. Institutional practices should be 3232 followed.107 3233

3234

H62

96


7.4.3 Reporting Qualitative Data 3235 3236 The results of qualitative assays are descriptive. Qualitative data are widely used in the analysis of leukemia 3237 and lymphoma cases and provides diagnostic information as well as information that guides patient care 3238 and treatment decisions. Qualitative data are also used to evaluate new therapeutic compounds and in-3239 compound screening. Laboratory reports for qualitative methods should include: 3240 3241 A summarized assessment of the findings in text format (normal vs abnormal populations identified) 3242 3243 A description of the phenotype of the abnormal population, when present 3244 3245 The level of antigen expression on an abnormal population in reference to its normal counterpart, 3246

especially in T- and B-cell lymphoproliferative disorders, eg: 3247 - Negative: not significantly different from the control population 3248

3249 - Positive: significantly greater than the control population 3250

3251 - Partially expressed: a subset of the population of interest is positive 3252

3253 - Normal-positive: uniform cell population with an MdFI that is essentially equal to that of a normal 3254

leukocyte subpopulation 3255 3256

- Dim-positive: uniform cell population with an MdFI that is abnormally less than that seen on an 3257 antigen positive normal population 3258

3259 - Bright-positive: uniform cell population with an MdFI that is abnormally greater than that seen on 3260

an antigen-positive normal population 3261 3262

- Heterogeneous–variably intense staining: not a tight uniform cluster of positive 3263 immunofluorescence staining 3264

3265 A list of all antibodies used for the analysis, which should include the source (manufacturer), 3266

fluorophore, and antibody clone name 3267 3268 Interpretational comments or recommendations for additional testing, including cytogenetic or 3269

molecular studies for certain findings. 3270 3271 7.5 File Storage and Retention 3272 3273 All work records, worksheets, log sheets, and laboratory copies of reports must be retained in the laboratory 3274 or in a designated location for a defined period as identified by the appropriate regulatory agencies.89 All 3275 reports or data that contain patient identifier information must be stored and disposed of in a manner that 3276 does not compromise patient confidentiality and is compliant with regulations. Raw data files (FCS format) 3277 should be backed up and stored. Some regulatory and accreditation requirements may also require the data 3278 to be saved in way that shows how specific gating or population identification procedures were undertaken. 3279 The data may be stored on either a local or remote server or on a permanent media storage device per 3280 institutional practices and regulatory and accreditation requirements. The integrity of back-up files must be 3281 confirmed before removing files from the instrument’s hard drive or moving files to a temporary storage 3282 location. The duration of file storage may vary according to the specific regulatory and accreditation 3283 organizations. Typically, a retention period of more than two years is required. All electronic record storage 3284 and transfer procedures should satisfy both the laboratory’s and regulatory and accreditation organization’s 3285 information technology security and privacy requirements. 3286 3287

H62

97


Table 24 summarizes the recommendations for storage retention of various data types. These are general 3288 recommendations that may not apply in all cases, depending on the institution regulatory agency and the 3289 accreditation organization’s requirements. The types of data and documents that should be retained are 3290 listed below. 3291 3292 Project-specific records should include: 3293 3294 Accession logs 3295 Worksheets 3296 QC reports 3297 Reported results (electronic) 3298 Raw data files (FCS) 3299 Project-specific SOP 3300 Project-specific certificates of analysis for reagents 3301 Project-specific preparation forms 3302 3303 Laboratory records should include: 3304 3305 Workload statistics 3306 Instrument maintenance and service records 3307 External QC records/reports 3308 Training records 3309 Competency records 3310 General SOPs 3311 Certificates of analysis for general reagents 3312 Reagent preparation forms per laboratory policy 3313 3314 Table 24. Recommended Storage Requirements 3315

Data Type Retention Perioda

Accession logs 2 yearsWorksheets 2 yearsIncident/accident reports Retain permanently in the employee file QC reports 5 years Workload statistics 2 yearsInstrument maintenance and service records 2 years External QC records/reports 8 years Reported results (electronic) 2 years–indefinitely, depending on results Raw data files (FCS)b 10 years Training records During employment plus 7 years Technical competency records During employment plus 7 years SOPs Discontinuation date plus 2 years

Abbreviations: FCS, flow cytometry standard; QC, quality control; SOP, standard operating procedure. 3316 a Retention periods vary by institution, regulatory and governing agencies, and national accreditation requirements. This is a general 3317 guideline of retention periods. 3318 b Flow cytometry analysis can be stored electronically but must not be altered once the original report has been issued. 3319 3320

H62

98


7.6 Training Records and Competency 3321 3322 Training records and competency requirements are discussed in detail in Subchapter 2.3 under the QSE of 3323 Personnel Management. The employer must be able to demonstrate that any concerns regarding the training 3324 of the individuals performing a given task are fully trained. Initial training and competency documentation 3325 should be stored per the institution’s policies. The process of training and competency assessment has been 3326 covered extensively in the literature.17,22 It is a best practice to assess staff competency on a regular basis, 3327 at least annually, and require staff to keep records of their continuing professional development. An 3328 example of a competency model is one required by Clinical and Laboratory Improvement Amendments of 3329 1988 that uses the following six procedures for assessing individual competency.117 3330 3331 1. Directly observing patient testing 3332 3333 2. Monitoring the recording and reporting of results 3334 3335 3. Reviewing worksheets for samples and preventative maintenance records 3336 3337 4. Directly observing instrument maintenance and checks 3338 3339 5. Assessing test performance through testing previously analyzed specimens, internal blind testing, or 3340

external PT samples 3341 3342 6. Assessing problem-solving skills 3343 3344 The training and competency documents should reference the above assessments for each component of 3345 the competency that is being evaluated for the staff and include preexamination, examination, and 3346 postexamination phases of testing. It may consist of direct observation of the employee performing all facets 3347 of the testing to include processing, staining, running samples, and analysis. Instrument maintenance and 3348 daily startup, including running beads to monitor alignment and system performance, is an essential part of 3349 a comprehensive competency assessment. It is necessary that competency include evaluating QC regarding 3350 the acceptability of results. Competency analysis can consist of gating, back gating, evaluating 3351 compensation, and using previously analyzed FCS files for analysis that include all specimen types and 3352 disease categories (more detailed information is available in CLSI documents H42 and H43).2,3 QC and PT 3353 sample material are helpful tools in assessing competency. For medical laboratories, test result reporting 3354 should be included in competency assessment. 3355 3356 7.7 Accreditation 3357 3358 Accreditation is a process that a regulated laboratory must undergo to demonstrate compliance with 3359 standards developed by an official agency. The international standard for clinical laboratory accreditation 3360 is ISO 15189.13 All medical laboratories should be accredited to this international standard whenever 3361 possible. However, it is accepted that some countries may have implemented accreditation that is not 3362 necessarily compliant with ISO 15189,13 and as a minimum, the medical laboratory must be accredited to 3363 the current national standards. All reports that are issued must show the accreditation status of the laboratory 3364 issuing the results and show its official accreditation number. However, not all laboratories, such as research 3365 laboratories, require accreditation. 3366 3367 3368

H62

99


7.8 External Quality Assessment/Proficiency Testing 3369 3370 Training, education, and QC (both internal QC and EQA/PT) are necessary to ensure reliable data. Medical 3371 laboratories must participate in an EQA/PT program that, whenever possible, is accredited per ISO 3372 17043.118 PT is testing unknown samples a laboratory receives from an approved EQA program. 3373 Laboratories receive EQA samples two to six times annually and test these samples using their standard 3374 clinical approach (whenever practically possible). The process of EQA/PT has been extensively covered in 3375 detail.22 It is recommended that evidence of EQA/PT participation should be retained in accordance with 3376 the criteria outlined in Table 24. In the absence of EQA/PT programs, sample exchange with laboratories 3377 undertaking the same testing is recommended. This can be done at frequent intervals to ascertain that 3378 comparable data are being obtained. If discrepancies occur, the probable cause of the variance should be 3379 identified by exchanging and evaluating techniques and processes. PT should be available for every test run 3380 in the laboratory. If external PT is not available, the laboratory should create a program and monitor the 3381 results at a minimum of twice a year. All data obtained from the laboratory-created program should be 3382 documented. 3383 3384 3385

H62

100


Chapter 8: Conclusion 3386 3387 This guideline provides up-to-date best practices for all aspects of flow cytometry testing, including 3388 instrument optimization; assay development and characterization; method validation; and data review and 3389 reporting. The target audience for this guideline is anyone developing, validating, or performing flow 3390 cytometric methods (eg, research laboratories, clinical laboratories, and drug discovery and development 3391 laboratories), as well as regulatory agencies reviewing data generated by flow cytometric methods. 3392 3393 When developed properly using a qualified instrument, flow cytometric methods are highly specific, 3394 sensitive, and precise. Because all data should be reliable, an FFP approach to method validation should be 3395 followed, meaning that the validation should be aligned with the intended use of the data and the regulatory 3396 requirements associated with this use. If the intended use or regulatory requirements change, additional 3397 validation experiments may be required. 3398 3399 Chapter 9: Supplemental Information 3400 3401

This chapter includes: References Additional Resources Appendixes The Quality Management System Approach Related CLSI Reference Materials

3402

H62

101


References 3403

1 US Department of Health and Human Services, Food and Drug Administration. Bioanalytical Method Validation Guidance for Industry. Silver Spring, MD: Office of Communications, Division of Drug Information Center for Drug Evaluation and Research; May 2018.

2 CLSI. Enumeration of Immunologically Defined Cell Populations by Flow Cytometry; Approved Guideline—Second Edition. CLSI

document H42-A2. Wayne, PA: Clinical Laboratory and Standards Institute; 2007. 3 CLSI. Clinical Flow Cytometric Analysis of Neoplastic Hematolymphoid Cells; Approved Guideline—Second Edition. CLSI

document H43-A2. Wayne, PA: Clinical Laboratory and Standards Institute; 2007. 4 CLSI. Red Blood Cell Diagnostic Testing Using Flow Cytometry; Approved Guideline—Second Edition. CLSI document H52-A2.

Wayne, PA: Clinical and Laboratory Standards Institute; 2014. 5 CLSI. Laboratory Instruments and Data Management Systems: Design of Software User Interfaces and End-User Software Systems

Validation, Operation, and Monitoring; Approved Guideline—Second Edition. CLSI document AUTO13-A2. Wayne, PA: Clinical and Laboratory Standards Institute; 2003.

6 Miller JM, Astles R, Baszler T, et al.; Biosafety Blue Ribbon Panel, Centers for Disease Control and Prevention (CDC). Guidelines

for Safe Work Practices in human and animal medical diagnostic laboratories. MMWR Suppl. 2012;61(1):1-102. 7 CLSI. Protection of Laboratory Workers From Occupationally Acquired Infections; Approved Guideline—Fourth Edition. CLSI

document M29-A4. Wayne, PA: Clinical and Laboratory Standards Institute; 2014. 8 ISO. In vitro diagnostic medical devices – Measurement of quantities in biological samples – Metrological traceability of values

assigned to calibrators and control materials. ISO 17511. Geneva, Switzerland: International Organization for Standardization; 2003. 9 Bureau International des Poids et Mesures (BIPM). International Vocabulary of Metrology – Basic and General Concepts and

Associated Terms (VIM, 3rd edition, JCGM 200:2012). http://www.bipm.org/en/publications/guides/vim.html. Accessed 9 September 2019.

10 CLSI. Newborn Screening for Cystic Fibrosis; Approved Guideline. CLSI document NBS05-A. Wayne, PA: Clinical and Laboratory

Standards Institute; 2011. 11 US FDA. Context of Use.

https://www.fda.gov/Drugs/DevelopmentApprovalProcess/DrugDevelopmentToolsQualificationProgram/Biomarker QualificationProgram/ucm535395.htm. Accessed 9 September 2019.

12 Lee JW, Weiner RS, Sailstad JM, et al. Method validation and measurement of biomarkers in nonclinical and clinical samples in drug

development: a conference report. Pharm Res. 2005;22(4):499-511. 13 ISO. Medical laboratories – Requirements for quality and competence. ISO 15189. Geneva, Switzerland: International Organization

for Standardization; 2014. 14 ISO. Reference materials – Contents of certificates, labels, and accompanying documentation. ISO Guide 31. Geneva, Switzerland:

International Organization for Standardization; 2015. 15 CLSI. A Quality Management System Model for Laboratory Services. 5th ed. CLSI guideline QMS01. Wayne, PA: Clinical and

Laboratory Standards Institute; 2019. 16 CLSI. Training and Competence Assessment. 4th ed. CLSI guideline QMS03. Wayne, PA: Clinical and Laboratory Standards

Institute; 2016. 17 Greig B, Oldaker TA, Warzynski M, Wood B. 2006 Bethesda International Consensus recommendations on the immunophenotypic

analysis of hematolymphoid neoplasia by flow cytometry: recommendations for training and education to perform clinical flow cytometry. Cytometry B Clin Cytom. 2007;72(suppl 1):S23-S33.

18 O’Hara DM, Xu Y, Liang Z, Reddy MP, Wu DY, Litwin V. Recommendations for the validation of flow cytometric testing during

drug development: II assays. J Immunol Methods. 2011;363(2):120-134. 19 Kammel M, Kummrow A, Neukammer J. Reference measurement procedures for the accurate determination of cell concentrations:

present status and future developments. J Lab Med. 2012;36(1):25-35. 20 Wood B, Jevremovic D, Béné MC, Yan M, Jacobs P, Litwin V; ICSH/ICCS Working Group. Validation of cell-based fluorescence

assays: practice guidelines from the ICSH and ICCS – part V – assay performance criteria. Cytometry B Clin Cyytom. 2013;84(5):315-323.

H62

102


21 Juskewitch JE, Abraham RS, League SC, et al. Monocyte HLA-DR expression and neutrophil CD64 expression as biomarkers of

infection in critically ill neonates and infants. Pediatr Res. 2015;78(6):683-690. 22 Barnett D, Louzao R, Gambell P, De J, Oldaker T, Hanson CA; ICSH/ICCS Working Group. Validation of cell-based fluorescence

assays: practice guidelines from the ICSH and ICCS – part IV – post-analytic considerations. Cytometry B Clin Cytom. 2013;84(5):309-314.

23 CLSI. User Protocol for Evaluation of Qualitative Test Performance; Approved Guideline—Second Edition. CLSI document EP12-

A2. Wayne, PA: Clinical and Laboratory Standards Institute; 2008. 24 81 Federal Register 46054-46055. National Institute of Standards and Technology. Flow Cytometry Quantitation Consortium. Office

of the Federal Register; 2016. 25 Czechowska, K, Lannigan J, Wang L, et al. Cyt-Geist: current and future challenges in cytometry: reports of the CYTO 2018

conference workshops. Cytometry. 2019;95(6): 598-644. 26 Wang L, Gaigalas AK, Marti G, Abbasi F, Hoffman RA. Toward quantitative fluorescence measurements with multicolor flow

cytometry. Cytometry A. 2008;73(4):279-288.

27 Green CL, Stewart JJ, Högerkorp CM, et al. Recommendations for the development and validation of flow cytometry-based receptor

occupancy assays. Cytometry B Clin Cytom. 2016;90(2):141-149. 28 CLSI. Interference Testing in Clinical Chemistry. 3rd ed. CLSI guideline EP07. Wayne, PA: Clinical and Laboratory Standards

Institute; 2018. 29 CLSI. Evaluation of Commutability of Processed Samples; Approved Guideline—Third Edition. CLSI document EP14-A3. Wayne,

PA: Clinical and Laboratory Standards Institute; 2014. 30 CLSI. Evaluation of Precision of Quantitative Measurement Procedures; Approved Guideline—Third Edition. CLSI document EP05-

A3. Wayne, PA: Clinical and Laboratory Standards Institute; 2014. 31 Loken MR, Voigt AP, Eidenschink Brodersen L, et al. Consistent quantitative gene product expression: #2. Antigen intensities on

bone marrow cells are invariant between individuals. Cytometry A. 2016;89(11):987-996. 32 Cunliffe J, Derbyshire N, Keeler S, Coldwell R. An approach to the validation of flow cytometry methods. Pharm Res.

2009;26(12):2551-2557. 33 von Holst C, Müller A, Björklund E, Anklam E. In-house validation of a simplified method for the determination of PCBs in food

and feedingstuffs. Eur Food Res Technol. 2001;213(2):154-160.

34 ISO. Reference materials -- Selected terms and definitions. ISO Guide 30. Geneva, Switzerland: International Organization for

Standardization; 2015. 35 CLSI. Defining, Establishing, and Verifying Reference Intervals in the Clinical Laboratory; Approved Guideline—Third Edition.

CLSI document EP28-A3c. Wayne, PA: Clinical and Laboratory Standards Institute; 2008.

36 CLSI. Laboratory Quality Control Based on Risk Management; Approved Guideline. CLSI document EP23-A™. Wayne, PA: Clinical and Laboratory Standards Institute; 2011.

37 US Food and Drug Administration, US Department of Health and Human Services. Part 820—Quality System Regulation (Codified

at 21 CFR §820). Office of the Federal Register; published annually. 38 ISO. Medical devices – Application of risk management to medical devices. ISO 14971. Geneva, Switzerland: International

Organization for Standardization; 2007. 39 ISO. Medical devices – Quality management systems – Requirements for regulatory purposes. ISO 13485. Geneva, Switzerland:

International Organization for Standardization; 2016. 40 Regulation (EU) 2017/746 of the European Parliament and of the Council of 5 April 2017 on in vitro diagnostic medical devices and

repealing Directive 98/79/EC and Commission Decision 2010/227/EU. https://ec.europa.eu/growth/sectors/medical-devices/regulatory-framework_en. Accessed 9 September 2019. 41 Green CL, Brown L, Stewart JJ, Xu Y, Litwin V, McCloskey TW. Recommendations for the validation of flow cytometric testing

during drug development: I instrumentation. J Immunol Methods. 2011;363(2):104-119.

H62

103


42 Wang L, DeRose P, Gaigalas AK. Assignment of the number of equivalent reference fluorophores to dyed microspheres. J Res Natl

Inst Stand Technol. 2016;121:264-281. 43 Wang L, Hoffman RA. Standardization, calibration, and control in flow cytometry. Curr Protoc Cytom. 2017;79:1.3.1-1.3.27. 44 National Institute of Standards and Technology. SRM 1934: Fluorescent dyes for quantitative flow cytometry (visible spectral range).

https://www-s.nist.gov/srmors/view_cert.cfm?srm=1934. Accessed 9 September 2019. 45 Bagwell CB, Baker D, Whetstone S, et al. A simple and rapid method for determining the linearity of a flow cytometer amplification

system. Cytometry. 1989;10(6):689-694. 46 Wang L, Hoffman, RA. Flow cytometer performance characterization, standardization, and control. In: Robinson JP, Cossarizza A,

eds. Single Cell Analysis: Contemporary Research and Clinical Applications. Singapore: Springer Nature; 2017:171-200. 47 Perfetto SP, Chattopadhyay PK, Wood J, et al. Q and B values are critical measurements required for inter-instrument standardization

and development of multicolor flow cytometry staining panels. Cytometry A. 2014;85(12):1037-1048. 48 Wood, JC. Fundamental flow cytometer properties governing sensitivity and resolution. Cytometry. 1998;33(2):260-266. 49 Hoffman RA, Wood JC. Characterization of flow cytometer instrument sensitivity. Curr Protoc Cytom. 2007;40:1.20.1-1.20.18. 50 Lacombe F, Bernal E, Bloxham D, et al. Harmonemia: a universal strategy for flow cytometry immunophenotyping: a European

LeukemiaNet WP10 study. Leukemia. 2016;30:1769-1772. 51 Borowitz MJ, Shuster J, Carroll AJ, et al. Prognostic significance of fluorescence intensity of surface marker expression in childhood

B-precursor acute lymphoblastic leukemia. A Pediatric Oncology Group Study. Blood. 1997;89(11):3960-3966. 52 Solly F, Rigollet L, Baseggio L, et al. Comparable flow cytometry data can be obtained with two types of instruments, Canto II, and

Navios. A GEIL study. Cytometry A. 2013; 83(12):1066-1072. 53 Preijers FW, Huys E, Favre C, Moshaver B. Establishment of harmonization in immunophenotyping: a comparative study of a

standardized one-tube lymphocyte screening panel. Cytometry B Clin Cytom. 2014;86(6):418-425. 54 Mathis S, Chapuis N, Borgeot J, et al. Comparison of cross-platform flow cytometry minimal residual disease evaluation in multiple

myeloma using a common antibody combination and analysis strategy. Cytometry B Clin Cytom. 2015;88(2):101-109. 55 Nováková M, Glier H, Brdičková N, et al. How to make usage of the standardized EuroFlow 8-color protocols possible for instruments

of different manufacturers. https://doi.org/10.1016/j.jim.2017.11.007. J Immunol Methods. 2017 Nov 20. pii: S0022-1759(17)30139-4. doi: 10.1016/j.jim.2017.11.007 [Epub ahead of print].

56 Schwartz A, Marti GE, Poon R, Gratama JW, Fernández-Repollet E. Standardizing flow cytometry: a classification system of

fluorescence standards used for flow cytometry. Cytometry. 1998;33(2):106-114. 57 Spidlen J, Moore W, Parks D, et al. Data file standard for flow cytometry, version FCS 3.1. Cytometry A, 2010:77(1);97-100. 58 Kalina T, Flores-Montero J, van der Velden VH, et al. EuroFlow standardization of flow cytometer instrument settings and

immunophenotyping protocols. Leukemia. 2012;26(9):1986-2010. 59 Kalina T, Flores-Montero J, Lecrevisse Q, et al. Quality assessment program for EuroFlow protocols: summary results of four-year

(2010−2013) quality assurance rounds. Cytometry A. 2015; 87(2):145-156. 60 Bainbridge J, Rountree W, Louzao R, et al. Laboratory accuracy improvement in the UK NEQAS leucocyte immunophenotyping

immune monitoring program: an eleven-year review via longitudinal mixed effects modeling. Cytometry B Clin Cytom. 2018;94(2):250-256.

61 Keeney M, Wood BL, Hedley BD, et al. A QA program for MRD testing demonstrates that systematic education can reduce

discordance among experienced interpreters. Cytometry B Clin Cytom. 2018;94(2):239-249. 62 Cossarizza A, Chang HD, Radbruch A, et al. Guidelines for the use of flow cytometry and cell sorting in immunological studies. Eur

J Immunol. 2017;47(10):1584-1797. 63 Swerdlow SH, Campo E, Harris NL, et al. WHO Classification of Tumours of Haematopoietic and Lymphoid Tissues 4th ed. Lyon:

International Agency for Research on Cancer; 2008. 64 Arber DA, Orazi A, Hasserjian R, et al. The 2016 revision to the World Health Organization classification of myeloid neoplasms and

acute leukemia. Blood. 2016;127(20):2391-2405.

H62

104


65 Swerdlow SH, Campo E, Pileri SA, et al. The 2016 revision of the World Health Organization classification of lymphoid neoplasms.

Blood. 2016;127(20):2375-2390. 66 Stewart JJ, Green CL, Jones N, et al. Role of receptor occupancy assays by flow cytometry in drug development. Cytometry B Clin

Cytom. 2016;90(2):110-116. 67 Litwin V, Green C, Stewart JJ. Receptor occupancy by flow cytometry. Cytometry B Clin Cytom. 2016;90(2):108-109. 68 Maecker HT, Frey T, Nomura LE, Trotter J. Selecting fluorochrome conjugates for maximum sensitivity. Cytometry A.

2004;62(2):169-173. 69 Rawstron AC, Kreuzer KA, Soosapilla A, et al. Reproducible diagnosis of chronic lymphocytic leukemia by flow cytometry: an

European Research Initiative on CLL (ERIC) & European Society for Clinical Cell Analysis (ESCCA) Harmonisation project. Cytometry B Clin Cytom. 2018;94(1):121-128.

70 Lanza F, Healy L, Sutherland DR. Structural and functional features of the CD34 antigen: an update. J Biol Regul Homeost Agents.

2001;15(1):1-13. 71 Otto M, Chen X, Martin WJ, et al. Selection of stem cells by using antibodies that target different CD34 epitopes yields different

patterns of T-cell differentiation. Stem Cells. 2007;25(2):537-542. 72 Sutherland DR, Anderson L, Keeney M, Nayar R, Chin-Yee I. The ISHAGE guidelines for CD34+ cell determination by flow

cytometry. International Society of Hematotherapy and Graft Engineering. J Hematother. 1996;5(3):213-226. 73 Wood BL, Levin GR. Interactions between mouse IgG2 antibodies are common and mediated by plasma C1q. Cytometry B Clin

Cytom. 2006;70(5):321-328. 74 Mahnke YD, Roederer M. Optimizing a multicolor immunophenotyping assay. Clin Lab Med. 2007;27(3):469-485, v. 75 Deka C, Lehnert BE, Lehnert NM, Jones GM, Sklar LA, Steinkamp JA. Analysis of fluorescence lifetime and quenching of FITC-

conjugated antibodies on cells by phase-sensitive flow cytometry. Cytometry. 1996;25(3):271-279. 76 Brown L, Green CL, Jones N, et al. Recommendations for the evaluation of specimen stability for flow cytometric testing during drug

development. J Immunol Methods. 2015;418:1-8. 77 Tsieh S. Flow Cytometry, Immunohistochemistry, and Molecular Genetics for Hematologic Neoplasms. 2nd ed. Philadelphia, PA:

Lippincott Williams & Wilkins; 2012. 78 Greig B, Stetler-Stevenson M, Lea J. Stabilization media increases recovery in paucicellular cerebrospinal fluid specimens submitted

for flow cytometry testing. Cytometry B Clin Cytom. 2014;86(2):135-138. 79 Mosiman VL, Patterson BK, Canterero L, Goolsby CL. Reducing cellular autofluorescence in flow cytometry: an in situ method.

Cytometry. 1997;30(3):151-156. 80 Gratama JW, D’hautcourt JL, Mandy F, et al. Flow cytometric quantitation of immunofluorescence intensity: problems and

perspectives. European Working Group on Clinical Cell Analysis. Cytometry. 1998;33(2):166-178. 81 Cedeño-Arias M, Sánchez-Ramirez J, Blanco-Santana R, Rengifo-Calzado E. Validation of a flow cytometry based binding assay for

evaluation of monoclonal antibody recognizing EGF receptor. Sci Pharm. 2011;79(3):569-581. 82 Tario JD, Wallace PK. Reagents and cell staining for immunophenotyping by flow cytometry. In: McManus LM, Mitchell RN, eds.

Pathobiology of Human Disease. Elsevier; 2014:3678-3701. 83 Hulspas R, O’Gorman MR, Wood BL, Gratama JW, Sutherland DR. Considerations for the control of background fluorescence in

clinical flow cytometry. Cytometry B Clin Cytom. 2009;76(6):355-364. 84 Davis BH, Wood B, Oldaker T, Barnett D. Validation of cell-based fluorescence assays: practice guidelines from the ICSH and ICCS

- part I - rationale and aims. Cytometry B Clin Cytom. 2013;84(5):282-285. 85 Zhang JH, Chung TD, Oldenburg KR. A simple statistical parameter for use in evaluation and validation of high throughput screening

assays. J Biomol Screen. 1999;4(2):67-73. 86 Andersen MN, Al-Karradi SN, Kragstrup TW, Hokland M. Elimination of erroneous results in flow cytometry caused by antibody

binding to Fc receptors on human monocytes and macrophages. Cytometry A. 2016;89(11):1001-1009.

H62

105


87 Tanqri S, Vall H, Kaplan D, et al. Validation of cell-based fluorescence assays: practice guidelines from the ICSH and ICCS - part

III - analytical issues. Cytometry B Clin Cytom. 2013;84(5):291-308. 88 O’Donahue M, Johnson L. Lysing methods and reagents for flow cytometry immunophenotyping. International Clinical Cytometry

Society Quality and Standards Committee; 2016. www.cytometry.org/web/modules/module1.pdf. Accessed 9 September 2019. 89 van der Strate B, Longdin R, Geerlings M, et al. Best practices in performing flow cytometry in a regulated environment: feedback

from experience within the European Bioanalysis Forum. Bioanalysis. 2017;9(16):1253-1264. 90 Davis BH, Holden JT, Bene MC, et al. 2006 Bethesda International Consensus recommendations on the flow cytometric

immunophenotypic analysis of hematolymphoid neoplasia: medical indications. Cytometry B Clin Cytom. 2007;72(suppl 1):S5-S13. 91 Wood BL, Arroz M, Barnett D, et al. 2006 Bethesda International Consensus recommendations on the immunophenotypic analysis

of hematolymphoid neoplasia by flow cytometry: optimal reagents and reporting for the flow cytometric diagnosis of hematopoietic neoplasia. Cytometry B Clin Cytom. 2007;72(suppl 1):S14-S22.

92 Herzenberg LA, Tung J, Moore WA, Herzenberg LA, Parks DR. Interpreting flow cytometry data: a guide for the perplexed. Nat

Immunol. 2006;7(7):681-685. 93 Lindemann M, Lenz V, Nyadu B, et al. Effect of ABO incompatibility on T-cell flow cytometry cross-match results prior to living

donor kidney transplantation. Cytometry B Clin Cytom. 2018;94(4):623-630. 94 Böttcher S, van der Velden VHJ, Villamor N, et al. Lot-to-lot stability of antibody reagents for flow cytometry. J Immunol Methods.

2017 Mar 30 [Epub ahead of print]. 95 Njemini R, Onyema OO, Renmans W, Bautmans I, De Waele M, Mets T. Shortcomings in the application of multicolour flow

cytometry in lymphocyte subsets enumeration. Scand J Immunol. 2014;79(2):75-89. 96 Takizawa F, Kinet JP, Adamczewski M. Binding of phycoerythrin and its conjugates to murine low affinity receptors for

immunoglobulin G. J Immunol Methods. 1993;162(2):269-272. 97 Hulspas R, Dombkowski D, Preffer F, Douglas D, Kildew-Shah B, Gilbert J. Flow cytometry and the stability of phycoerythrin-

tandem dye conjugates. Cytometry A. 2009;75(11):966-972. 98 Jahrsdorfer B, Blackwell SE, Weiner GJ. Phosphorothyoate oligodeoxynucleotides block nonspecific binding of Cy5 conjugates to

monocytes. J Immunol Methods. 2005;297(1-2):259-263. 99 van Vugt MJ, van den Herik-Oudijk IE, van de Winkle JG. Binding of PE-CY5 conjugates to the human high-affinity receptor for

IgG (CD64). Blood. 1996;88(6):2358-2361. 100 Stewart CC, Stewart SJ. Multiparameter data acquisition and analysis of leukocytes by flow cytometry. Methods Cell Biol.

2001;64:289-312. 101 Stewart CC, Stewart SJ. Cell preparation for the identification of leukocytes. Methods Cell Biol. 2001;63:217-251. 102 Beavis AJ, Pennline KJ. Allo-7: a new fluorescent tandem dye for use in flow cytometry. Cytometry. 1996;24(4):390-395. 103 Allan AL, Keeney M. Circulating tumor cell analysis: technical and statistical considerations for application to the clinic. J Oncol.

2010;2010:426218. 104 Rawstron AC, Orfao A, Beksac M, et al. Report of the European Myeloma Network on multiparametric flow cytometry in multiple

myeloma and related disorders. Haematologica. 2008;93(3):431-438. 105 Centers for Medicare & Medicaid Services, Department of Health and Human Services. Laboratory Requirements: Establishment of

verification of performance specifications (Codified at 42 CFR, §493.1253(b)). Office of the Federal Register; 2003. 106 US Department of Health and Human Services, Food and Drug Administration. Biomarker Qualification: Evidentiary Framework

Guidance for Industry and FDA Staff. Office of the Federal Register; 2018. 107 Oldaker T, Whitby L, Saber M, Holden J, Wallace PK, Litwin V. ICCS/ESCCA consensus guidelines to detect GPI-deficient cells

in paroxysmal nocturnal hemoglobinuria (PNH) and related disorders part 4: assay validation and quality assurance. Cytometry B Clin Cytom. 2018;94(1):67-81.

108 National Institute of Standards and Technology. Definitions of Terms and Modes Used at NIST for Value-Assignment of Reference

Materials for Chemical Measurements. Publication 260-136. https://www.nist.gov/publications/definitions-terms-and-modes-used-nist-value-assignment-reference-materials-chemical. Accessed 9 September 2019.

H62

106


109 Carroll TA, Pinnick HA, Carroll WE. Probability and the Westgard rules. Ann Clin Lab Sci. 2003;33(1):113-114. 110 Armbruster DA, Pry T. Limit of blank, limit of detection and limit of quantitation. Clin Biochem Rev. 2008;29(suppl 1)S49-S52. 111 Shrivastava A, Gupta VB. Methods for the determination of limit of detection and limit of quantitation of the analytical methods.

Chron Young Sci. 2011;2(1):21-25. 112 CLSI. Evaluation of the Linearity of Quantitative Measurement Procedures: A Statistical Approach; Approved Guideline. CLSI

document EP06-A. Wayne, PA: Clinical and Laboratory Standards Institute; 2003. 113 Holford N. Holford NHG and Sheiner LB “Understanding the Dose-Effect Relationship-Clinical Application of Pharmacokinetic-

Pharmacodynamic Models,” Clin Pharmacokin 6:429-453 (1981): the backstory. AAPS J. 2011;13(4):662-664. 114 Horton H, Thomas EP, Stucky JA, et al. Optimization and validation of an 8-color intracellular cytokine staining (ICS) assay to

quantify antigen-specific T cells induced by vaccination. J Immunol Methods. 2007:323(1):39-54. 115 Horna P, Olteanu H, Kroft SH, Harrington AM. Flow cytometric analysis of surface light chain expression patterns in B-cell

lymphomas using monoclonal and polyclonal antibodies. Am J Clin Pathol. 2011;136(6):954-959. 116 Davis BH. Quantitative neutrophil CD64 expression: promising diagnostic indicator of infection or systemic acute inflammatory

response. Clin Immunol Newslett. 1996;16(9):121-130. 117 CMS. What Do I Need to Do to Assess Personnel Competency? Baltimore, MD: Center for Medicare and Medicaid Services; 2012.

https://www.cms.gov/Regulations-and-Guidance/Legislation/CLIA/Downloads/CLIA_CompBrochure_508.pdf. Accessed 9 September 2019.

118 ISO. Conformity assessment – General requirements for proficiency testing. ISO 17043. Geneva, Switzerland: International

Organization for Standardization; 2010.

H62


Additional Resources 3404 3405 CAP. CAP Accreditation Checklists. Northfield, IL: College of American Pathologists; published annually. 3406 3407 CLSI. The Key to Quality™. CLSI product K2Q. Wayne, PA: Clinical and Laboratory Standards Institute; 3408 2013. 3409 3410 The Joint Commission. Laboratory Accreditation Standards. Oakbrook Terrace, IL: Joint Commission 3411 Resources; published annually. 3412 3413

H62

“DR

AF

T D

OC

UM

EN

T. T

his

draf

t CL

SI d

ocum

ent i

s no

t to

be re

prod

uced

or c

ircu

late

d fo

r an

y pu

rpos

e ot

her

than

rev

iew

and

com

men

t. It

is n

ot to

be

cons

ider

ed e

ithe

r fi

nal o

r pu

blis

hed

and

may

not

be

quot

ed o

r re

fere

nced

. 18

Sep

tem

ber

201

9.”

Ap

pen

dix

A. D

etai

led

Val

idat

ion

Men

us

3414

3415

T

able

A1.

FF

P V

alid

atio

n T

ype

1 34

16

Par

amet

er

Com

men

ts

Sam

ple

s R

epli

cate

s A

nal

ytic

al R

un

s A

ccu

racy

/tru

enes

s R

efer

to S

ubch

apte

rs 3

.2.1

an

d 6.

1.2.

1 fo

r m

ore

info

rmat

ion.

Not

req

uire

d N

ot r

equi

red

Not

req

uire

d

Sp

ecif

icit

y

Doc

umen

tati

on s

houl

d in

clud

e a

sum

mar

y of

the

assa

y de

velo

pmen

t wit

h re

gard

to s

peci

fici

ty, s

uch

as th

e re

fere

nces

ju

stif

ying

the

choi

ce o

f an

tige

ns a

nd th

e ga

ting

st

rate

gy.

R

efer

to S

ubch

apte

rs 5

.1

and

6.1.

2.2

for

mor

e in

form

atio

n.

Add

itio

nal e

valu

atio

n is

no

t req

uire

d du

ring

va

lida

tion

.

Add

itio

nal e

valu

atio

n is

no

t req

uire

d du

ring

va

lida

tion

.

Add

itio

nal e

valu

atio

n is

no

t req

uire

d du

ring

va

lida

tion

.

Sel

ecti

vity

Req

uire

d on

ly if

test

ing

in

the

pres

ence

of

a bi

olog

ic

that

inte

rfer

es w

ith

the

assa

y.

R

efer

to S

ubch

apte

r 6.

1.2.

2 fo

r m

ore

info

rmat

ion.

3 3

1 or

mor

e

Sen

siti

vity

LO

D/L

OB

Req

uire

d on

ly if

test

ing

low

leve

ls o

f m

easu

rand

.

Ref

er to

Sub

chap

ter

6.1.

2.3.

1 fo

r m

ore

info

rmat

ion.

10 d

ata

poin

ts

1 or

mor

e 1

or m

ore

L

LO

Q

R

equi

red

only

if te

stin

g lo

w le

vels

of

mea

sura

nd.

R

efer

to S

ubch

apte

r 6.

1.2.

3.2

for

mor

e in

form

atio

n.

3 (5

leve

ls e

ach)

3

1 or

mor

e

H

62

109

“DR

AF

T D

OC

UM

EN

T. T

his

draf

t CL

SI d

ocum

ent i

s no

t to

be re

prod

uced

or c

ircu

late

d fo

r an

y pu

rpos

e ot

her

than

rev

iew

and

com

men

t. It

is n

ot to

be

cons

ider

ed e

ithe

r fi

nal o

r pu

blis

hed

and

may

not

be

quot

ed o

r re

fere

nced

. 18

Sep

tem

ber

201

9.”

34

17

Tab

le A

1. (

Con

tin

ued

) 34

18

Par

amet

er

Com

men

ts

Sam

ple

s R

epli

cate

s A

nal

ytic

al R

un

s P

reci

sion

Rep

eata

bil

ity

(

Intr

a-as

say)

Req

uire

d

Ref

er to

Fig

ure

17 a

nd

Sub

chap

ter

6.1.

2.4

for

mor

e in

form

atio

n.

4

H

ealt

hy d

onor

sam

ples

m

ay b

e us

ed.

D

isea

se-s

tate

sam

ples

ar

e op

tion

al.

3

Eac

h sa

mpl

e sh

ould

be

test

ed in

1 o

r m

ore

runs

.

If m

ore

than

1 o

pera

tor

and

mor

e th

an 1

in

stru

men

t:

1

run

per

sam

ple

O

pera

tor

1 sh

ould

test

ha

lf o

f th

e sa

mpl

es o

n in

stru

men

t 1.

O

pera

tor

2 sh

ould

test

ha

lf o

f th

e sa

mpl

es o

n in

stru

men

t 2.

R

epro

du

cib

ilit

y

(In

tera

ssay

)

Req

uire

d

Ref

er to

Fig

ure

17 a

nd

Sub

chap

ter

6.1.

2.4

for

mor

e in

form

atio

n.

2

or m

ore

Q

C o

r sa

mpl

es m

ay b

e us

ed.

3

Eac

h sa

mpl

e sh

ould

be

test

ed in

2 o

r m

ore

runs

.

If m

ore

than

1 o

pera

tor

and

mor

e th

an 1

in

stru

men

t:

O

pera

tor

1 sh

ould

test

2

QC

on

inst

rum

ent 1

an

d pe

rfor

m 1

–2 r

uns.

Ope

rato

r 2

shou

ld te

st

2 Q

C o

n in

stru

men

t 2

and

perf

orm

1–2

runs

.

In

tero

per

ator

Opt

iona

l

Req

uire

d on

ly if

mor

e th

an 1

ope

rato

r w

ill b

e co

nduc

ting

test

ing.

Ref

er to

Fig

ure

17 a

nd

Sub

chap

ter

6.1.

2.4

for

mor

e in

form

atio

n.

D

ata

gene

rate

d fr

om

intr

a- a

nd in

tera

ssay

ex

peri

men

ts m

ay b

e us

ed f

or th

e ca

lcul

atio

ns.

A

ddit

iona

l exp

erim

ents

ar

e no

t req

uire

d.

D

ata

gene

rate

d fr

om

intr

a- a

nd in

tera

ssay

ex

peri

men

ts m

ay b

e us

ed f

or th

e ca

lcul

atio

ns.

A

ddit

iona

l exp

erim

ents

ar

e no

t req

uire

d.

D

ata

gene

rate

d fr

om in

tra-

an

d in

tera

ssay

exp

erim

ents

m

ay b

e us

ed f

or th

e ca

lcul

atio

ns.

A

ddit

iona

l exp

erim

ents

ar

e no

t req

uire

d.

H62

11

0

“DR

AF

T D

OC

UM

EN

T. T

his

draf

t CL

SI d

ocum

ent i

s no

t to

be re

prod

uced

or c

ircu

late

d fo

r an

y pu

rpos

e ot

her

than

rev

iew

and

com

men

t. It

is n

ot to

be

cons

ider

ed e

ithe

r fi

nal o

r pu

blis

hed

and

may

not

be

quot

ed o

r re

fere

nced

. 18

Sep

tem

ber

201

9.”

Tab

le A

1. (

Con

tin

ued

) 34

19

Par

amet

er

Com

men

ts

Sam

ple

s R

epli

cate

s A

nal

ytic

al R

un

s

In

teri

nst

rum

ent

O

ptio

nal

R

equi

red

only

if m

ore

than

1 in

stru

men

t wil

l be

use

d du

ring

test

ing.

Ref

er to

Fig

ure

17 a

nd

Sub

chap

ter

6.1.

2.4

for

mor

e in

form

atio

n.

D

ata

gene

rate

d fr

om

intr

a- a

nd in

tera

ssay

ex

peri

men

ts m

ay b

e us

ed f

or th

e ca

lcul

atio

ns.

A

ddit

iona

l exp

erim

ents

ar

e no

t req

uire

d.

D

ata

gene

rate

d fr

om

intr

a- a

nd in

tera

ssay

ex

peri

men

ts m

ay b

e us

ed f

or th

e ca

lcul

atio

ns.

A

ddit

iona

l exp

erim

ents

ar

e no

t req

uire

d.

D

ata

gene

rate

d fr

om

intr

a- a

nd in

tera

ssay

ex

peri

men

ts m

ay b

e us

ed f

or th

e ca

lcul

atio

ns.

A

ddit

iona

l exp

erim

ents

ar

e no

t req

uire

d.

Lin

eari

ty

R

efer

to S

ubch

apte

rs

3.2.

2 an

d 6.

1.2.

5 fo

r m

ore

info

rmat

ion.

Not

req

uire

d N

ot r

equi

red

Not

req

uire

d

Sta

bil

ity

S

pec

imen

Req

uire

d on

ly if

sa

mpl

es w

ill n

ot b

e te

sted

wit

hin

2 ho

urs

of c

olle

ctio

n.

R

efer

to S

ubch

apte

r 6.

1.2.

6.1

for

mor

e in

form

atio

n.

3 1

1 ru

n pe

r ti

me

poin

t

P

roce

ssed

sam

ple

Req

uire

d on

ly if

sa

mpl

es w

ill n

ot b

e te

sted

wit

hin

1 ho

ur o

f st

aini

ng/f

ixin

g.

R

efer

to S

ubch

apte

r 6.

1.2.

6.2

for

mor

e in

form

atio

n.

3 1

1 ru

n pe

r ti

me

poin

t

Car

ryov

er

S

houl

d be

con

duct

ed

duri

ng in

stru

men

t qu

alif

icat

ion.

Ref

er to

Sub

chap

ter

4.2.

5 fo

r m

ore

info

rmat

ion.

Add

itio

nal e

valu

atio

n is

no

t req

uire

d du

ring

va

lida

tion

.

Add

itio

nal e

valu

atio

n is

no

t req

uire

d du

ring

va

lida

tion

.

Add

itio

nal e

valu

atio

n is

no

t req

uire

d du

ring

va

lida

tion

.

3420

H

62

111

“DR

AF

T D

OC

UM

EN

T. T

his

draf

t CL

SI d

ocum

ent i

s no

t to

be re

prod

uced

or c

ircu

late

d fo

r an

y pu

rpos

e ot

her

than

rev

iew

and

com

men

t. It

is n

ot to

be

cons

ider

ed e

ithe

r fi

nal o

r pu

blis

hed

and

may

not

be

quot

ed o

r re

fere

nced

. 18

Sep

tem

ber

201

9.”

Tab

le A

1. (

Con

tin

ued

) 34

21

Par

amet

er

Com

men

ts

Sam

ple

s R

epli

cate

s A

nal

ytic

al R

un

s

Ref

eren

ce in

terv

als

Ref

er to

Sub

chap

ters

3.2

.8

and

6.1.

2.8

for

mor

e in

form

atio

n.

Not

req

uire

d N

ot r

equi

red

Not

req

uire

d

Doc

um

enta

tion

Val

idat

ion

pla

n

The

val

idat

ion

docu

men

tati

on m

ay b

e in

clud

ed in

the

assa

y de

velo

pmen

t pla

n; a

se

para

te v

alid

atio

n pl

an is

no

t man

dato

ry.

R

efer

to S

ubch

apte

r 6.

1 fo

r m

ore

info

rmat

ion.

V

alid

atio

n r

epor

t

The

val

idat

ion

docu

men

tati

on m

ay b

e in

clud

ed in

the

assa

y de

velo

pmen

t rep

ort;

a

sepa

rate

val

idat

ion

repo

rt is

no

t man

dato

ry.

R

efer

to S

ubch

apte

r 6.

3 fo

r m

ore

info

rmat

ion.

Q

A r

evie

w

N

ot r

equi

red

A

bbre

viat

ions

: FF

P, f

it-f

or-p

urpo

se; L

LO

Q, l

ower

lim

it of

qua

ntif

icat

ion;

LO

B, l

imit

of

blan

k; L

OD

, lim

it of

det

ectio

n; Q

A, q

uali

ty a

ssur

ance

. 34

22

3423

H62

11

2

“DR

AF

T D

OC

UM

EN

T. T

his

draf

t CL

SI d

ocum

ent i

s no

t to

be re

prod

uced

or c

ircu

late

d fo

r an

y pu

rpos

e ot

her

than

rev

iew

and

com

men

t. It

is n

ot to

be

cons

ider

ed e

ithe

r fi

nal o

r pu

blis

hed

and

may

not

be

quot

ed o

r re

fere

nced

. 18

Sep

tem

ber

201

9.”

Tab

le A

2. F

FP

Val

idat

ion

Typ

e 2

Par

amet

er

Com

men

ts

Sam

ple

s R

epli

cate

s A

nal

ytic

al R

un

s A

ccu

racy

/tru

enes

s R

efer

to S

ubch

apte

rs 3

.2.1

an

d 6.

1.2.

1 N

ot r

equi

red

Not

req

uire

d N

ot r

equi

red

Sp

ecif

icit

y

Doc

umen

tati

on s

houl

d in

clud

e a

sum

mar

y of

the

assa

y de

velo

pmen

t wit

h re

gard

to s

peci

fici

ty, s

uch

as th

e re

fere

nces

just

ifyi

ng

the

choi

ce o

f an

tige

ns a

nd

the

gati

ng s

trat

egy.

Ref

er to

Sub

chap

ters

5.1

an

d 6.

1.2.

2 fo

r m

ore

info

rmat

ion.

Add

itio

nal e

valu

atio

n is

no

t req

uire

d du

ring

va

lida

tion

.

Add

itio

nal e

valu

atio

n is

no

t req

uire

d du

ring

va

lida

tion

.

Add

itio

nal e

valu

atio

n is

no

t req

uire

d du

ring

va

lida

tion

.

Sel

ecti

vity

Req

uire

d on

ly if

test

ing

in

the

pres

ence

of

a bi

olog

ic

that

inte

rfer

es w

ith

the

assa

y.

R

efer

to S

ubch

apte

r 6.

1.2.

2 fo

r m

ore

info

rmat

ion.

3 3

1 or

mor

e

Sen

siti

vity

LO

D/L

OB

Req

uire

d on

ly if

test

ing

low

le

vels

of

mea

sura

nd.

R

efer

to S

ubch

apte

r 6.

1.2.

3.1

for

mor

e in

form

atio

n.

10 d

ata

poin

ts

1 or

mor

e 1

or m

ore

L

LO

Q

R

equi

red

only

if te

stin

g lo

w

leve

ls o

f m

easu

rand

.

Ref

er to

Sub

chap

ter

6.1.

2.3.

2 fo

r m

ore

info

rmat

ion.

3 (5

leve

ls e

ach)

3

1 or

mor

e

3424

H

62

113

“DR

AF

T D

OC

UM

EN

T. T

his

draf

t CL

SI d

ocum

ent i

s no

t to

be re

prod

uced

or c

ircu

late

d fo

r an

y pu

rpos

e ot

her

than

rev

iew

and

com

men

t. It

is n

ot to

be

cons

ider

ed e

ithe

r fi

nal o

r pu

blis

hed

and

may

not

be

quot

ed o

r re

fere

nced

. 18

Sep

tem

ber

201

9.”

Tab

le A

2. (

Con

tin

ued

) 34

25

Par

amet

er

Com

men

ts

Sam

ple

s R

epli

cate

s A

nal

ytic

al R

un

s P

reci

sion

Rep

eata

bil

ity

(

Intr

a-as

say)

Req

uire

d

Ref

er to

Fig

ure

17 a

nd

Sub

chap

ter

6.1.

2.4

for

mor

e in

form

atio

n.

6

H

ealt

hy d

onor

sam

ples

m

ay b

e us

ed.

D

isea

se-s

tate

sam

ples

ar

e op

tion

al.

3

Eac

h sa

mpl

e sh

ould

be

test

ed in

1 o

r m

ore

runs

.

1 ru

n pe

r sa

mpl

e

Ope

rato

r 1

shou

ld te

st

half

of

the

sam

ples

on

inst

rum

ent 1

.

Ope

rato

r 2

shou

ld te

st

half

of

the

sam

ples

on

inst

rum

ent 2

. or

2 ru

ns p

er s

ampl

e

Ope

rato

r 1

shou

ld te

st

all o

f th

e sa

mpl

es o

n in

stru

men

t 1.

O

pera

tor

2 sh

ould

test

al

l of

the

sam

ples

on

inst

rum

ent 2

.

Rep

rod

uci

bil

ity

(

Inte

rass

ay)

R

equi

red

R

efer

to F

igur

e 17

and

S

ubch

apte

r 6.

1.2.

4 fo

r m

ore

info

rmat

ion.

2

or m

ore

Q

C o

r sa

mpl

es m

ay b

e us

ed.

3

4

Ope

rato

r 1

shou

ld te

st

2 Q

C o

n in

stru

men

t 1 a

nd

perf

orm

2 r

uns.

Ope

rato

r 2

shou

ld te

st

2 Q

C o

n in

stru

men

t 2 a

nd

perf

orm

2 ru

ns.

I

nte

rop

erat

or

R

equi

red

R

efer

to F

igur

e 17

and

S

ubch

apte

r 6.

1.2.

4 fo

r m

ore

info

rmat

ion.

D

ata

gene

rate

d fr

om

intr

a- a

nd in

tera

ssay

ex

peri

men

ts m

ay b

e us

ed f

or th

e ca

lcul

atio

ns.

A

ddit

iona

l exp

erim

ents

ar

e no

t req

uire

d.

D

ata

gene

rate

d fr

om

intr

a- a

nd in

tera

ssay

ex

peri

men

ts m

ay b

e us

ed f

or th

e ca

lcul

atio

ns.

A

ddit

iona

l exp

erim

ents

ar

e no

t req

uire

d.

D

ata

gene

rate

d fr

om

intr

a- a

nd in

tera

ssay

ex

peri

men

ts m

ay b

e

used

for

the

calc

ulat

ions

.

Add

itio

nal e

xper

imen

ts

are

not r

equi

red.

3426

H62

11

4

“DR

AF

T D

OC

UM

EN

T. T

his

draf

t CL

SI d

ocum

ent i

s no

t to

be re

prod

uced

or c

ircu

late

d fo

r an

y pu

rpos

e ot

her

than

rev

iew

and

com

men

t. It

is n

ot to

be

cons

ider

ed e

ithe

r fi

nal o

r pu

blis

hed

and

may

not

be

quot

ed o

r re

fere

nced

. 18

Sep

tem

ber

201

9.”

Tab

le A

2. (

Con

tin

ued

) 34

27

Par

amet

er

Com

men

ts

Sam

ple

s R

epli

cate

s A

nal

ytic

al R

un

s

In

teri

nst

rum

ent

R

equi

red

R

efer

to F

igur

e 17

and

S

ubch

apte

r 6.

1.2.

4 fo

r m

ore

info

rmat

ion.

D

ata

gene

rate

d fr

om

intr

a- a

nd in

tera

ssay

ex

peri

men

ts m

ay b

e us

ed f

or th

e ca

lcul

atio

ns.

A

ddit

iona

l exp

erim

ents

ar

e no

t req

uire

d.

D

ata

gene

rate

d fr

om

intr

a- a

nd in

tera

ssay

ex

peri

men

ts m

ay b

e us

ed f

or th

e ca

lcul

atio

ns.

A

ddit

iona

l exp

erim

ents

ar

e no

t req

uire

d.

D

ata

gene

rate

d fr

om

intr

a- a

nd in

tera

ssay

ex

peri

men

ts m

ay b

e us

ed

for

the

calc

ulat

ions

.

Add

itio

nal e

xper

imen

ts

are

not r

equi

red.

Lin

eari

ty

Ref

er to

Sub

chap

ters

3.

2.2

and

6.1.

2.5

for

mor

e in

form

atio

n.

Not

req

uire

d N

ot r

equi

red

Not

req

uire

d

Sta

bil

ity

S

pec

imen

Req

uire

d on

ly if

sa

mpl

es w

ill n

ot b

e te

sted

wit

hin

2 ho

urs

of

coll

ecti

on.

R

efer

to S

ubch

apte

r 6.

1.2.

6.1

for

mor

e in

form

atio

n.

6 1

1 ru

n pe

r ti

me

poin

t

P

roce

ssed

sam

ple

Req

uire

d on

ly if

sa

mpl

es w

ill n

ot b

e te

sted

wit

hin

1 ho

ur o

f st

aini

ng/f

ixin

g.

R

efer

to S

ubch

apte

r 6.

1.2.

6.2

for

mor

e in

form

atio

n.

3 1

1 ru

n pe

r ti

me

poin

t

Car

ryov

er

S

houl

d be

con

duct

ed

duri

ng in

stru

men

t qu

alif

icat

ion.

Ref

er to

Sub

chap

ter

4.2.

5 fo

r m

ore

info

rmat

ion.

Add

itio

nal e

valu

atio

n is

no

t req

uire

d du

ring

va

lida

tion

.

Add

itio

nal e

valu

atio

n is

no

t req

uire

d du

ring

va

lida

tion

.

Add

itio

nal e

valu

atio

n is

no

t req

uire

d du

ring

va

lida

tion

.

Ref

eren

ce in

terv

als

Ref

er to

Sub

chap

ters

3.

2.8

and

6.1.

2.8

for

mor

e in

form

atio

n.

Not

req

uire

d N

ot r

equi

red

Not

req

uire

d

3428

H

62

115

“DR

AF

T D

OC

UM

EN

T. T

his

draf

t CL

SI d

ocum

ent i

s no

t to

be re

prod

uced

or c

ircu

late

d fo

r an

y pu

rpos

e ot

her

than

rev

iew

and

com

men

t. It

is n

ot to

be

cons

ider

ed e

ithe

r fi

nal o

r pu

blis

hed

and

may

not

be

quot

ed o

r re

fere

nced

. 18

Sep

tem

ber

201

9.”

Tab

le A

2. (

Con

tin

ued

) 34

29

Par

amet

er

Com

men

ts

Sam

ple

s R

epli

cate

s A

nal

ytic

al R

un

s D

ocu

men

tati

on

V

alid

atio

n p

lan

Req

uire

d

Ref

er to

Sub

chap

ter

6.1

for

mor

e in

form

atio

n.

Val

idat

ion

rep

ort

R

equi

red

R

efer

to S

ubch

apte

r 6.

3 fo

r m

ore

info

rmat

ion.

Q

A r

evie

w

Req

uire

d A

bbre

viat

ions

: FF

P, f

it-f

or-p

urpo

se; L

LO

Q, l

ower

lim

it of

qua

ntif

icat

ion;

LO

B, l

imit

of

blan

k; L

OD

, lim

it of

det

ectio

n; Q

A, q

uali

ty a

ssur

ance

. 34

30

34

31

3432

H62

11

6

“DR

AF

T D

OC

UM

EN

T. T

his

draf

t CL

SI d

ocum

ent i

s no

t to

be re

prod

uced

or c

ircu

late

d fo

r an

y pu

rpos

e ot

her

than

rev

iew

and

com

men

t. It

is n

ot to

be

cons

ider

ed e

ithe

r fi

nal o

r pu

blis

hed

and

may

not

be

quot

ed o

r re

fere

nced

. 18

Sep

tem

ber

201

9.”

Tab

le A

3. V

erif

icat

ion

of

an I

VD

-Ap

pro

ved

/CD

-Mar

ked

Ass

ay

Par

amet

er

Com

men

ts

Sam

ple

s R

epli

cate

s A

nal

ytic

al R

un

s A

ccu

racy

/tru

enes

s

Inte

rlab

orat

ory

com

pari

son

wit

h a

labo

rato

ry r

unni

ng

the

sam

e IV

D s

houl

d be

co

nduc

ted.

Ref

er to

Sub

chap

ters

3.2

.1

and

6.1.

2.1

for

mor

e in

form

atio

n.

≥

10

S

ampl

es th

at c

over

th

e ex

pect

ed

repo

rtab

le r

ange

sh

ould

be

incl

uded

.

1 1

run

per

sam

ple

Sp

ecif

icit

y R

efer

to th

e pa

ckag

e in

sert

. A

ddit

iona

l eva

luat

ion

is

not r

equi

red

duri

ng

vali

dati

on.

Add

itio

nal e

valu

atio

n is

no

t req

uire

d du

ring

va

lida

tion

.

Add

itio

nal e

valu

atio

n is

no

t req

uire

d du

ring

va

lida

tion

.S

elec

tivi

ty

T

he r

ecom

men

dati

ons

in

the

pack

age

inse

rt s

houl

d be

fo

llow

ed.

A

ddit

iona

l eva

luat

ion

is

only

req

uire

d if

test

ing

in

the

pres

ence

of

a bi

olog

ic

that

inte

rfer

es w

ith

the

assa

y.

R

efer

to S

ubch

apte

r 6.

1.2.

2 fo

r m

ore

info

rmat

ion.

3 3

1 or

mor

e

Sen

siti

vity

LO

D

Ref

er to

the

pack

age

inse

rt.

Add

itio

nal e

valu

atio

n is

no

t req

uire

d du

ring

va

lida

tion

.

Add

itio

nal e

valu

atio

n is

no

t req

uire

d du

ring

va

lida

tion

.

Add

itio

nal e

valu

atio

n is

no

t req

uire

d du

ring

va

lida

tion

.

LL

OQ

R

efer

to th

e pa

ckag

e in

sert

. A

ddit

iona

l eva

luat

ion

is

not r

equi

red

duri

ng

vali

dati

on.

Add

itio

nal e

valu

atio

n is

no

t req

uire

d du

ring

va

lida

tion

.

Add

itio

nal e

valu

atio

n is

no

t req

uire

d du

ring

va

lida

tion

.

34

33

H

62

117

“DR

AF

T D

OC

UM

EN

T. T

his

draf

t CL

SI d

ocum

ent i

s no

t to

be re

prod

uced

or c

ircu

late

d fo

r an

y pu

rpos

e ot

her

than

rev

iew

and

com

men

t. It

is n

ot to

be

cons

ider

ed e

ithe

r fi

nal o

r pu

blis

hed

and

may

not

be

quot

ed o

r re

fere

nced

. 18

Sep

tem

ber

201

9.”

Tab

le A

3. (

Con

tin

ued

) 34

34

Par

amet

er

Com

men

ts

Sam

ple

s R

epli

cate

s A

nal

ytic

al R

un

s P

reci

sion

Rep

eata

bil

ity

(

Intr

a-as

say)

Req

uire

d

Ref

er to

Fig

ure

17 a

nd

Sub

chap

ter

6.1.

2.4

for

mor

e in

form

atio

n.

6

H

ealt

hy d

onor

sa

mpl

es a

nd/o

r us

ed

dise

ase-

stat

e sa

mpl

es

D

epen

ds o

n th

e m

etho

d an

d th

e in

tend

ed u

se.

3

Eac

h sa

mpl

e sh

ould

be

test

ed in

1 o

r m

ore

runs

.

If m

ore

than

1 o

pera

tor

and

mor

e th

an 1

inst

rum

ent:

1 ru

n pe

r sa

mpl

e

Ope

rato

r 1

shou

ld te

st

half

of

the

sam

ples

on

inst

rum

ent 1

.

Ope

rato

r 2

shou

ld te

st

half

of

the

sam

ples

on

inst

rum

ent 2

.

Rep

rod

uci

bil

ity

(

Inte

rass

ay)

R

equi

red

R

efer

to F

igur

e 17

and

S

ubch

apte

r 6.

1.2.

4 fo

r m

ore

info

rmat

ion.

Ref

er to

S

ubch

apte

r 6.

1.3.

4 fo

r m

ore

info

rmat

ion.

3

4

Eac

h sa

mpl

e sh

ould

be

test

ed in

4 r

uns.

If m

ore

than

1 o

pera

tor

and

mor

e th

an 1

inst

rum

ent:

Ope

rato

r 1

shou

ld te

st

2 Q

C o

n in

stru

men

t 1

and

perf

orm

2 r

uns.

Ope

rato

r 2

shou

ld te

st

2 Q

C o

n in

stru

men

t 2

and

perf

orm

2 r

uns.

3435

H62

11

8

“DR

AF

T D

OC

UM

EN

T. T

his

draf

t CL

SI d

ocum

ent i

s no

t to

be re

prod

uced

or c

ircu

late

d fo

r an

y pu

rpos

e ot

her

than

rev

iew

and

com

men

t. It

is n

ot to

be

cons

ider

ed e

ithe

r fi

nal o

r pu

blis

hed

and

may

not

be

quot

ed o

r re

fere

nced

. 18

Sep

tem

ber

201

9.”

Tab

le A

3. (

Con

tin

ued

) 34

36

Par

amet

er

Com

men

ts

Sam

ple

s R

epli

cate

s A

nal

ytic

al R

un

s

In

tero

per

ator

R

efer

to F

igur

e 17

and

S

ubch

apte

r 6.

1.2.

4 fo

r m

ore

info

rmat

ion.

D

ata

gene

rate

d fr

om

intr

a- a

nd in

tera

ssay

ex

peri

men

ts m

ay b

e us

ed f

or th

e ca

lcul

atio

ns.

A

ddit

iona

l exp

erim

ents

ar

e no

t req

uire

d.

D

ata

gene

rate

d fr

om

intr

a- a

nd in

tera

ssay

ex

peri

men

ts m

ay b

e us

ed

for

the

calc

ulat

ions

.

Add

itio

nal e

xper

imen

ts

are

not r

equi

red.

D

ata

gene

rate

d fr

om

intr

a- a

nd in

tera

ssay

ex

peri

men

ts m

ay b

e us

ed

for

the

calc

ulat

ions

.

Add

itio

nal e

xper

imen

ts

are

not r

equi

red.

In

teri

nst

rum

ent

Ref

er to

Fig

ure

17 a

nd

Sub

chap

ter

6.1.

2.4

for

mor

e in

form

atio

n.

D

ata

gene

rate

d fr

om

intr

a- a

nd in

tera

ssay

ex

peri

men

ts m

ay b

e us

ed f

or th

e ca

lcul

atio

ns.

A

ddit

iona

l exp

erim

ents

ar

e no

t req

uire

d.

D

ata

gene

rate

d fr

om

intr

a- a

nd in

tera

ssay

ex

peri

men

ts m

ay b

e us

ed

for

the

calc

ulat

ions

.

Add

itio

nal e

xper

imen

ts

are

not r

equi

red.

D

ata

gene

rate

d fr

om

intr

a- a

nd in

tera

ssay

ex

peri

men

ts m

ay b

e us

ed

for

the

calc

ulat

ions

.

Add

itio

nal e

xper

imen

ts

are

not r

equi

red.

L

inea

rity

R

efer

to th

e pa

ckag

e in

sert

. A

ddit

iona

l eva

luat

ion

is

not r

equi

red

duri

ng

vali

dati

on.

Add

itio

nal e

valu

atio

n is

no

t req

uire

d du

ring

va

lida

tion

.

Add

itio

nal e

valu

atio

n is

no

t req

uire

d du

ring

va

lida

tion

.

Sta

bil

ity

Sp

ecim

en

R

efer

to th

e pa

ckag

e in

sert

for

the

targ

et

valu

e.

C

onfi

rmat

ion

of

man

ufac

ture

r’s

clai

ms

is r

equi

red.

3–6

1 1

run

per

tim

e po

int

3437

H

62

119

“DR

AF

T D

OC

UM

EN

T. T

his

draf

t CL

SI d

ocum

ent i

s no

t to

be re

prod

uced

or c

ircu

late

d fo

r an

y pu

rpos

e ot

her

than

rev

iew

and

com

men

t. It

is n

ot to

be

cons

ider

ed e

ithe

r fi

nal o

r pu

blis

hed

and

may

not

be

quot

ed o

r re

fere

nced

. 18

Sep

tem

ber

201

9.”

Tab

le A

3. (

Con

tin

ued

) 34

38

Par

amet

er

Com

men

ts

Sam

ple

s R

epli

cate

s A

nal

ytic

al R

un

s

Pro

cess

ed s

amp

le

R

efer

to th

e pa

ckag

e in

sert

fo

r th

e ta

rget

val

ue.

C

onfi

rmat

ion

of th

e m

anuf

actu

rer’

s cl

aim

s is

re

quir

ed.

3 1

1 ru

n pe

r ti

me

poin

t

Car

ryov

er

S

houl

d be

con

duct

ed d

urin

g in

stru

men

t qua

lifi

cati

on.

R

efer

to S

ubch

apte

r 4.

2.5

for

mor

e in

form

atio

n.

Add

itio

nal e

valu

atio

n is

not

req

uire

d du

ring

va

lida

tion

.

Add

itio

nal e

valu

atio

n is

no

t req

uire

d du

ring

va

lida

tion

.

Add

itio

nal e

valu

atio

n is

no

t req

uire

d du

ring

va

lida

tion

.

Ref

eren

ce in

terv

als

Con

firm

atio

n of

the

man

ufac

ture

r’s

rang

es s

houl

d be

con

duct

ed.

20

1 1

run

per

sam

ple

Doc

um

enta

tion

Val

idat

ion

pla

n

R

equi

red

R

efer

to S

ubch

apte

r 6.

1 fo

r m

ore

info

rmat

ion.

V

alid

atio

n r

epor

t

Req

uire

d

Ref

er to

Sub

chap

ter

6.3

for

mor

e in

form

atio

n.

QA

rev

iew

R

equi

red

Abb

revi

atio

ns: C

D, c

lust

er o

f di

ffer

enti

atio

n; I

VD

, in

vitr

o di

agno

stic

; LL

OQ

, low

er li

mit

of q

uant

ific

atio

n; L

OD

, lim

it of

det

ectio

n; Q

A, q

ualit

y as

sura

nce.

34

39

3440

H62

12

0

“DR

AF

T D

OC

UM

EN

T. T

his

draf

t CL

SI d

ocum

ent i

s no

t to

be re

prod

uced

or c

ircu

late

d fo

r an

y pu

rpos

e ot

her

than

rev

iew

and

com

men

t. It

is n

ot to

be

cons

ider

ed e

ithe

r fi

nal o

r pu

blis

hed

and

may

not

be

quot

ed o

r re

fere

nced

. 18

Sep

tem

ber

201

9.”

Tab

le A

4. C

LIA

/IM

DR

F Q

ual

itat

ive

Val

idat

ion

P

aram

eter

C

omm

ents

S

amp

les

Rep

lica

tes

An

alyt

ical

Ru

ns

Acc

ura

cy/t

ruen

ess

S

houl

d be

com

pare

d w

ith

clin

ical

dia

gnos

is.

R

efer

to S

ubch

apte

rs 3

.2.1

an

d 6.

1.3.

1 fo

r m

ore

info

rmat

ion.

≥

10

N

egat

ive

and

posi

tive

sam

ples

as

wel

l as

low

- po

siti

ve/d

im

sam

ples

sho

uld

be in

clud

ed.

1 1

run

per

sam

ple

Sp

ecif

icit

y

Doc

umen

tati

on s

houl

d in

clud

e a

sum

mar

y of

the

assa

y de

velo

pmen

t wit

h re

gard

to s

peci

fici

ty, s

uch

as th

e re

fere

nces

just

ifyi

ng

the

choi

ce o

f an

tige

ns a

nd

the

gati

ng s

trat

egy.

Ref

er to

Sub

chap

ters

5.1

an

d 6.

1.3.

2 fo

r m

ore

info

rmat

ion.

The

acc

urac

y da

tase

t can

be

used

. 1

or m

ore

1 ru

n pe

r sa

mpl

e

Sel

ecti

vity

Req

uire

d on

ly if

test

ing

in

the

pres

ence

of

a bi

olog

ic

that

inte

rfer

es w

ith

the

assa

y.

R

efer

to S

ubch

apte

r 6.

1.2.

2 fo

r m

ore

info

rmat

ion.

3 3

1 or

mor

e

An

alyt

ical

Sen

siti

vity

LO

D

N

/A f

or q

uali

tati

ve a

ssay

s

Ref

er to

Sub

chap

ter

6.1.

3.3

and

clin

ical

sen

siti

vity

be

low

for

mor

e in

form

atio

n.

N/A

N

/A

N/A

L

LO

Q

N

/A f

or q

uali

tati

ve a

ssay

s

Ref

er to

Sub

chap

ter

6.1.

3.3

and

clin

ical

sen

siti

vity

be

low

for

mor

e in

form

atio

n.

N/A

N

/A

N/A

H

62

121

“DR

AF

T D

OC

UM

EN

T. T

his

draf

t CL

SI d

ocum

ent i

s no

t to

be re

prod

uced

or c

ircu

late

d fo

r an

y pu

rpos

e ot

her

than

rev

iew

and

com

men

t. It

is n

ot to

be

cons

ider

ed e

ithe

r fi

nal o

r pu

blis

hed

and

may

not

be

quot

ed o

r re

fere

nced

. 18

Sep

tem

ber

201

9.”

Tab

le A

4. (

Con

tin

ued

) 34

41

Par

amet

er

Com

men

ts

Sam

ple

s R

epli

cate

s A

nal

ytic

al R

un

s P

reci

sion

Rep

eata

bil

ity

(

Intr

a-as

say)

R

efer

to S

ubch

apte

r 6.

1.3.

4 fo

r m

ore

info

rmat

ion.

6

D

isea

se-s

tate

sa

mpl

es a

re

requ

ired

.

Dep

endi

ng o

n in

tend

ed u

se, i

t m

ay b

e ne

cess

ary

to a

lso

eval

uate

he

alth

y do

nor

sam

ples

.

3

1 ru

n pe

r sa

mpl

e

Ope

rato

r 1

shou

ld te

st

half

of

the

sam

ples

on

inst

rum

ent 1

.

Ope

rato

r 2

shou

ld te

st

half

of

the

sam

ples

on

inst

rum

ent 2

. or

2 ru

ns p

er s

ampl

e

Ope

rato

r 1

shou

ld te

st

half

of

the

sam

ples

on

inst

rum

ent 1

.

Ope

rato

r 2

shou

ld te

st

half

of

the

sam

ples

on

inst

rum

ent 2

.

Rep

rod

uci

bil

ity

(

Inte

rass

ay)

Ref

er to

Sub

chap

ter

6.1.

3.4

for

mor

e in

form

atio

n.

2–

3 di

seas

e-st

ate

sam

ples

If a

ppli

cabl

e, Q

C

sam

ples

sho

uld

also

be

eval

uate

d.Q

C s

ampl

es c

an

be u

sed

as w

ell t

o es

tabl

ish

init

ial

QC

ran

ges.

3

4 ru

ns p

er s

ampl

e

Ope

rato

r 1

shou

ld te

st

2 Q

C o

n in

stru

men

t 1

and

perf

orm

2 r

uns.

Ope

rato

r 2

shou

ld te

st

2 Q

C o

n in

stru

men

t 2

and

perf

orm

2 r

uns.

3442

H62

12

2

“DR

AF

T D

OC

UM

EN

T. T

his

draf

t CL

SI d

ocum

ent i

s no

t to

be re

prod

uced

or c

ircu

late

d fo

r an

y pu

rpos

e ot

her

than

rev

iew

and

com

men

t. It

is n

ot to

be

cons

ider

ed e

ithe

r fi

nal o

r pu

blis

hed

and

may

not

be

quot

ed o

r re

fere

nced

. 18

Sep

tem

ber

201

9.”

Tab

le A

4. (

Con

tin

ued

) 34

43

Par

amet

er

Com

men

ts

Sam

ple

s R

epli

cate

s A

nal

ytic

al R

un

s

In

tero

per

ator

R

efer

to S

ubch

apte

r 6.

1.3.

4 fo

r m

ore

info

rmat

ion.

D

ata

gene

rate

d fr

om

intr

a- a

nd in

tera

ssay

ex

peri

men

ts m

ay b

e us

ed f

or th

e ca

lcul

atio

ns.

A

ddit

iona

l exp

erim

ents

ar

e no

t req

uire

d.

D

ata

gene

rate

d fr

om

intr

a- a

nd in

tera

ssay

ex

peri

men

ts m

ay b

e us

ed f

or th

e ca

lcul

atio

ns.

A

ddit

iona

l exp

erim

ents

ar

e no

t req

uire

d.

D

ata

gene

rate

d fr

om

intr

a- a

nd in

tera

ssay

ex

peri

men

ts m

ay b

e

used

for

the

calc

ulat

ions

.

Add

itio

nal e

xper

imen

ts a

re

not r

equi

red.

I

nte

rin

stru

men

t R

efer

to S

ubch

apte

r 6.

1.3.

4 fo

r m

ore

info

rmat

ion.

D

ata

gene

rate

d fr

om

intr

a- a

nd in

tera

ssay

ex

peri

men

ts m

ay b

e us

ed f

or th

e ca

lcul

atio

ns.

A

ddit

iona

l exp

erim

ents

ar

e no

t req

uire

d.

D

ata

gene

rate

d fr

om

intr

a- a

nd in

tera

ssay

ex

peri

men

ts m

ay b

e us

ed f

or th

e ca

lcul

atio

ns.

A

ddit

iona

l exp

erim

ents

ar

e no

t req

uire

d.

D

ata

gene

rate

d fr

om

intr

a- a

nd in

tera

ssay

ex

peri

men

ts m

ay b

e

used

for

the

calc

ulat

ions

.

Add

itio

nal e

xper

imen

ts

are

not r

equi

red.

Lin

eari

ty

N

/A f

or q

uali

tati

ve

assa

ys

R

efer

to S

ubch

apte

r 6.

1.3.

5 fo

r m

ore

info

rmat

ion.

N/A

N

/A

N/A

Mea

sure

men

t/

rep

orta

ble

ran

ge

N

/A f

or q

uali

tati

ve

assa

ys

R

efer

to S

ubch

apte

r 6.

1.3.

5 fo

r m

ore

info

rmat

ion.

N/A

N

/A

N/A

Sta

bil

ity

S

pec

imen

The

tim

e po

int s

houl

d ex

tend

1 p

oint

bey

ond

whe

re s

tabi

lity

is

acce

ptab

le.

R

efer

to S

ubch

apte

r 6.

1.3.

6 fo

r m

ore

info

rmat

ion.

≥ 3

1 1

run

per

tim

e po

int

3444

H

62

123

“DR

AF

T D

OC

UM

EN

T. T

his

draf

t CL

SI d

ocum

ent i

s no

t to

be re

prod

uced

or c

ircu

late

d fo

r an

y pu

rpos

e ot

her

than

rev

iew

and

com

men

t. It

is n

ot to

be

cons

ider

ed e

ithe

r fi

nal o

r pu

blis

hed

and

may

not

be

quot

ed o

r re

fere

nced

. 18

Sep

tem

ber

201

9.”

Tab

le A

4. (

Con

tin

ued

) 34

45

Par

amet

er

Com

men

tsS

amp

les

Rep

lica

tes

An

alyt

ical

Ru

ns

P

roce

ssed

sam

ple

The

tim

e po

int s

houl

d ex

tend

1

poin

t bey

ond

whe

re

stab

ilit

y is

acc

epta

ble.

Ref

er to

Sub

chap

ter

6.1.

3.6

for

mor

e in

form

atio

n.

3 1

1 ru

n pe

r ti

me

poin

t

C

ock

tail

R

efer

to S

ubch

apte

r 6.

1.3.

7 an

d A

ppen

dix

E f

or m

ore

info

rmat

ion.

3 1

1 ru

n pe

r ti

me

poin

t

Car

ryov

er

T

he p

osit

ive

and

nega

tive

sa

mpl

es s

houl

d be

ev

alua

ted

sequ

enti

ally

.

Ref

er to

Sub

chap

ter

6.1.

2.7

for

mor

e in

form

atio

n.

2 or

mor

e (p

osit

ive

and

nega

tive

) 3

(alt

erna

ting

pos

itiv

e an

d ne

gati

ve)

1

Ref

eren

ce in

terv

als

R

efer

to S

ubch

apte

r 6.

1.2.

8 fo

r m

ore

info

rmat

ion.

D

epen

ding

on

the

inte

nded

use

:

A f

ull r

efer

ence

in

terv

al s

houl

d be

es

tabl

ishe

d.

A

lim

ited

eva

luat

ion

of n

orm

al a

nd/o

r di

seas

e-st

ate

dono

rs

shou

ld b

e te

sted

.

Val

ues

can

be

com

pare

d w

ith

publ

ishe

d ra

nges

(a

ccep

tabi

lity

is

wit

hin

95%

CI)

.

1 1

run

per

sam

ple

3446

H62

12

4

“DR

AF

T D

OC

UM

EN

T. T

his

draf

t CL

SI d

ocum

ent i

s no

t to

be re

prod

uced

or c

ircu

late

d fo

r an

y pu

rpos

e ot

her

than

rev

iew

and

com

men

t. It

is n

ot to

be

cons

ider

ed e

ithe

r fi

nal o

r pu

blis

hed

and

may

not

be

quot

ed o

r re

fere

nced

. 18

Sep

tem

ber

201

9.”

Tab

le A

4. (

Con

tin

ued

) 34

47

Par

amet

er

Com

men

tsS

amp

les

Rep

lica

tes

An

alyt

ical

Ru

ns

Cli

nic

al V

alid

atio

n

Cli

nic

al s

pec

ific

ity

R

equi

red

R

efer

to S

ubch

apte

r 6.

1.3.

2 an

d C

LS

I do

cum

ent E

P12

1 for

mor

e in

form

atio

n.

The

acc

urac

y da

tase

t can

be

use

d to

cal

cula

te

clin

ical

spe

cifi

city

.

The

acc

urac

y da

tase

t can

be

use

d to

cal

cula

te

clin

ical

spe

cifi

city

.

The

acc

urac

y da

tase

t can

be

use

d to

cal

cula

te

clin

ical

spe

cifi

city

.

C

lin

ical

sen

siti

vity

Req

uire

d

Ref

er to

Sub

chap

ter

6.1.

3.3

and

CL

SI

docu

men

t EP

121 f

or m

ore

info

rmat

ion.

The

acc

urac

y da

tase

t can

be

use

d to

cal

cula

te

clin

ical

sen

siti

vity

.

The

acc

urac

y da

tase

t can

be

use

d to

cal

cula

te

clin

ical

sen

siti

vity

.

The

acc

urac

y da

tase

t can

be

use

d to

cal

cula

te

clin

ical

sen

siti

vity

.

Doc

um

enta

tion

Val

idat

ion

pla

n

R

equi

red

R

efer

to S

ubch

apte

r 6.

1 fo

r m

ore

info

rmat

ion.

V

alid

atio

n r

epor

t

Req

uire

d

Ref

er to

Sub

chap

ter

6.3

for

mor

e in

form

atio

n.

Q

A r

evie

w

R

equi

red

A

bbre

viat

ions

: C

I, c

onfi

denc

e in

terv

al;

CL

IA, C

lini

cal

Lab

orat

ory

Impr

ovem

ents

Am

endm

ents

of

1988

; IM

DR

F,

Inte

rnat

iona

l M

edic

al D

evic

e R

egul

ator

s F

orum

; L

LO

Q,

low

er

3448

li

mit

of q

uant

ific

atio

n; L

OD

, lim

it o

f de

tect

ion;

N/A

, not

app

lica

ble;

QA

, qua

lity

assu

ranc

e.

3449

34

50

H

62

125

“DR

AF

T D

OC

UM

EN

T. T

his

draf

t CL

SI d

ocum

ent i

s no

t to

be re

prod

uced

or c

ircu

late

d fo

r an

y pu

rpos

e ot

her

than

rev

iew

and

com

men

t. It

is n

ot to

be

cons

ider

ed e

ithe

r fi

nal o

r pu

blis

hed

and

may

not

be

quot

ed o

r re

fere

nced

. 18

Sep

tem

ber

201

9.”

Tab

le A

5. C

LIA

/IM

DR

F Q

uan

tita

tive

Val

idat

ion

Par

amet

er

Com

men

ts

Sam

ple

s R

epli

cate

s A

nal

ytic

al R

un

s A

ccu

racy

/tru

enes

s

Sho

uld

be c

ompa

red

wit

h cl

inic

al d

iagn

osis

.

Ref

er to

Sub

chap

ters

3.2

.1

and

6.1.

2.1

for

mor

e in

form

atio

n.

≥

10

S

houl

d in

clud

e sa

mpl

es th

at c

over

th

e ex

pect

ed

repo

rtab

le r

ange

.

1 1

run

per

sam

ple

Sp

ecif

icit

y

Doc

umen

tati

on s

houl

d in

clud

e a

sum

mar

y of

the

assa

y de

velo

pmen

t wit

h re

gard

to s

peci

fici

ty, s

uch

as

the

refe

renc

es ju

stif

ying

the

choi

ce o

f an

tige

ns a

nd th

e ga

ting

str

ateg

y.

R

efer

to S

ubch

apte

rs 5

.1 a

nd

6.1.

2.2

for

mor

e in

form

atio

n.

3 ≥

1 or

mor

e 1

run

per

sam

ple

Sel

ecti

vity

Req

uire

d on

ly if

test

ing

in th

e pr

esen

ce o

f a

biol

ogic

that

in

terf

eres

wit

h th

e as

say.

Ref

er to

Sub

chap

ter

6.1.

2.2

for

mor

e in

form

atio

n.

3 3

1 or

mor

e ru

ns p

er s

ampl

e

Sen

siti

vity

LO

D

R

equi

red

R

efer

to S

ubch

apte

r 6.

1.2.

3.1

for

mor

e in

form

atio

n.

10 (

5 ne

gati

ve a

nd

5 lo

w-p

osit

ive

sam

ples

) 5

(eac

h re

plic

ate

can

be a

cqui

red

5 ti

mes

to

incr

ease

the

stat

isti

cal

pow

er)

A

ll th

e sa

mpl

es s

houl

d no

t be

test

ed o

n th

e sa

me

day.

Tes

ting

sho

uld

be s

prea

d ou

t ove

r a

min

imum

of

3

days

.

34

51

H62

12

6

“DR

AF

T D

OC

UM

EN

T. T

his

draf

t CL

SI d

ocum

ent i

s no

t to

be re

prod

uced

or c

ircu

late

d fo

r an

y pu

rpos

e ot

her

than

rev

iew

and

com

men

t. It

is n

ot to

be

cons

ider

ed e

ithe

r fi

nal o

r pu

blis

hed

and

may

not

be

quot

ed o

r re

fere

nced

. 18

Sep

tem

ber

201

9.”

Tab

le A

5. (

Con

tin

ued

) 34

52

Par

amet

er

Com

men

ts

Sam

ple

s R

epli

cate

s A

nal

ytic

al R

un

s

LL

OQ

Req

uire

d

Ref

er to

Sub

chap

ter

6.1.

2.3.

2 fo

r m

ore

info

rmat

ion.

≥ 5

sam

ples

nea

r th

e L

LO

Q in

tr

ipli

cate

3 1

or m

ore

Pre

cisi

on

R

epea

tab

ilit

y

(In

tra-

assa

y)

R

equi

red

R

efer

to F

igur

e 17

and

S

ubch

apte

r 6.

1.2.

4 fo

r m

ore

info

rmat

ion.

6

D

isea

se-s

tate

sa

mpl

es a

re

requ

ired

.

Dep

endi

ng o

n th

e in

tend

ed u

se, i

t m

ay b

e ne

cess

ary

to a

lso

eval

uate

he

alth

y do

nor

sam

ples

.

3

Eac

h sa

mpl

e is

test

ed in

1

or m

ore

runs

:

Ope

rato

r 1

shou

ld te

st

half

of

the

sam

ples

on

inst

rum

ent 1

.

Ope

rato

r 2

shou

ld te

st

half

of

the

sam

ples

on

inst

rum

ent 2

. or

2 ru

ns p

er s

ampl

e:

O

pera

tor

1 sh

ould

test

ha

lf o

f th

e sa

mpl

es o

n in

stru

men

t 1.

O

pera

tor

2 sh

ould

test

ha

lf o

f th

e sa

mpl

es o

n in

stru

men

t 2.

3453

H

62

127

“DR

AF

T D

OC

UM

EN

T. T

his

draf

t CL

SI d

ocum

ent i

s no

t to

be re

prod

uced

or c

ircu

late

d fo

r an

y pu

rpos

e ot

her

than

rev

iew

and

com

men

t. It

is n

ot to

be

cons

ider

ed e

ithe

r fi

nal o

r pu

blis

hed

and

may

not

be

quot

ed o

r re

fere

nced

. 18

Sep

tem

ber

201

9.”

Tab

le A

5. (

Con

tin

ued

) 34

54

Par

amet

er

Com

men

ts

Sam

ple

s R

epli

cate

s A

nal

ytic

al R

un

s

Rep

rod

uci

bil

ity

(

Inte

rass

ay)

R

equi

red

R

efer

to F

igur

e 17

an

d S

ubch

apte

r 6.

1.2.

4 fo

r m

ore

info

rmat

ion.

2–

3 di

seas

e-st

ate

sam

ples

sh

ould

be

eval

uate

d.

If

app

lica

ble,

QC

sam

ples

sh

ould

als

o be

eva

luat

ed.

Q

C s

ampl

es c

an b

e us

ed a

s w

ell t

o es

tabl

ish

init

ial Q

C

rang

es.

3

4 ru

ns p

er s

ampl

e

Ope

rato

r 1

shou

ld te

st

2 Q

C o

n in

stru

men

t 1

and

perf

orm

2 r

uns.

Ope

rato

r 2

shou

ld te

st

2 Q

C o

n in

stru

men

t 2

and

perf

orm

2 r

uns.

I

nte

rop

erat

or

R

equi

red

R

efer

to F

igur

e 17

an

d S

ubch

apte

r 6.

1.2.

4 fo

r m

ore

info

rmat

ion.

D

ata

gene

rate

d fr

om in

tra-

an

d in

tera

ssay

exp

erim

ents

m

ay b

e us

ed f

or th

e ca

lcul

atio

ns.

A

ddit

iona

l exp

erim

ents

are

no

t req

uire

d.

D

ata

gene

rate

d fr

om in

tra-

an

d in

tera

ssay

exp

erim

ents

m

ay b

e us

ed f

or th

e ca

lcul

atio

ns.

A

ddit

iona

l exp

erim

ents

ar

e no

t req

uire

d.

D

ata

gene

rate

d fr

om in

tra-

an

d in

tera

ssay

exp

erim

ents

m

ay b

e us

ed f

or th

e ca

lcul

atio

ns.

A

ddit

iona

l exp

erim

ents

ar

e no

t req

uire

d.

In

teri

nst

rum

ent

R

equi

red

R

efer

to F

igur

e 17

an

d S

ubch

apte

r 6.

1.2.

4 fo

r m

ore

info

rmat

ion.

D

ata

gene

rate

d fr

om in

tra-

an

d in

tera

ssay

exp

erim

ents

m

ay b

e us

ed f

or th

e ca

lcul

atio

ns.

A

ddit

iona

l exp

erim

ents

are

no

t req

uire

d.

D

ata

gene

rate

d fr

om in

tra-

an

d in

tera

ssay

exp

erim

ents

m

ay b

e us

ed f

or th

e ca

lcul

atio

ns.

A

ddit

iona

l exp

erim

ents

ar

e no

t req

uire

d.

D

ata

gene

rate

d fr

om in

tra-

an

d in

tera

ssay

exp

erim

ents

m

ay b

e us

ed f

or th

e ca

lcul

atio

ns.

A

ddit

iona

l exp

erim

ents

ar

e no

t req

uire

d.

3455

H62

12

8

“DR

AF

T D

OC

UM

EN

T. T

his

draf

t CL

SI d

ocum

ent i

s no

t to

be re

prod

uced

or c

ircu

late

d fo

r an

y pu

rpos

e ot

her

than

rev

iew

and

com

men

t. It

is n

ot to

be

cons

ider

ed e

ithe

r fi

nal o

r pu

blis

hed

and

may

not

be

quot

ed o

r re

fere

nced

. 18

Sep

tem

ber

201

9.”

Tab

le A

5. (

Con

tin

ued

) 34

56

Par

amet

er

Com

men

ts

Sam

ple

s R

epli

cate

s A

nal

ytic

al R

un

s L

inea

rity

R

efer

to S

ubch

apte

rs

3.2.

2 an

d 6.

1.2.

5 fo

r m

ore

info

rmat

ion.

D

ata

gene

rate

d fr

om

LL

OQ

exp

erim

ents

may

be

use

d fo

r th

e ca

lcul

atio

ns/g

raph

s.

A

ddit

iona

l exp

erim

ents

ar

e no

t req

uire

d.

D

ata

gene

rate

d fr

om

LL

OQ

exp

erim

ents

may

be

use

d fo

r th

e ca

lcul

atio

ns/g

raph

s.

A

ddit

iona

l exp

erim

ents

ar

e no

t req

uire

d.

D

ata

gene

rate

d fr

om

LL

OQ

exp

erim

ents

may

be

use

d fo

r th

e ca

lcul

atio

ns/g

raph

s.

A

ddit

iona

l exp

erim

ents

ar

e no

t req

uire

d.M

easu

rem

ent/

re

por

tab

le r

ange

The

upp

er a

nd lo

wer

li

mit

inpu

t cel

l nu

mbe

r or

vol

ume

shou

ld b

e es

tabl

ishe

d du

ring

as

say

deve

lopm

ent.

D

ata

from

LL

OQ

an

d pr

ecis

ion

eval

uati

on s

houl

d be

us

ed to

est

abli

sh th

e up

per

and

low

er

repo

rtin

g li

mits

.

Add

itio

nal e

xper

imen

ts a

re

not r

equi

red.

A

ddit

iona

l exp

erim

ents

are

no

t req

uire

d.

Add

itio

nal e

xper

imen

ts a

re

not r

equi

red.

Sta

bil

ity

S

pec

imen

The

tim

e po

int

shou

ld e

xten

d 1

poin

t bey

ond

whe

re

stab

ilit

y is

ac

cept

able

.

Ref

er to

Sub

chap

ter

6.1.

2.6

for

mor

e in

form

atio

n.

≥ 3

1 1

run

per

tim

e po

int

P

roce

ssed

sam

ple

R

efer

to S

ubch

apte

r 6.

1.2.

6 fo

r m

ore

info

rmat

ion.

3 1

1 ru

n pe

r ti

me

poin

t

3457

H

62

129

“DR

AF

T D

OC

UM

EN

T. T

his

draf

t CL

SI d

ocum

ent i

s no

t to

be re

prod

uced

or c

ircu

late

d fo

r an

y pu

rpos

e ot

her

than

rev

iew

and

com

men

t. It

is n

ot to

be

cons

ider

ed e

ithe

r fi

nal o

r pu

blis

hed

and

may

not

be

quot

ed o

r re

fere

nced

. 18

Sep

tem

ber

201

9.”

Tab

le A

5. (

Con

tin

ued

) 34

58

Par

amet

er

Com

men

ts

Sam

ple

s R

epli

cate

s A

nal

ytic

al R

un

s C

arry

over

Low

and

hig

h sa

mpl

es

shou

ld b

e ev

alua

ted

sequ

enti

ally

.

Ref

er to

Sub

chap

ter

6.1.

2.7

for

mor

e in

form

atio

n.

2 or

mor

e (l

ow a

nd h

igh)

3

(alt

erna

ting

low

and

hi

gh)

1

Ref

eren

ce in

terv

als

Ref

er to

Sub

chap

ter

6.1.

2.8

for

mor

e in

form

atio

n.

Dep

endi

ng o

n th

e in

tend

ed u

se:

A

ful

l ref

eren

ce

inte

rval

sho

uld

be

esta

blis

hed.

A li

mit

ed e

valu

atio

n of

no

rmal

and

/or

dise

ase-

st

ate

dono

rs s

houl

d be

te

sted

.

Val

ues

can

be

com

pare

d w

ith

publ

ishe

d ra

nges

(a

ccep

tabi

lity

is w

ithi

n 95

% C

I).

1 1

run

per

sam

ple

Cli

nic

al V

alid

atio

n

Cli

nic

al s

pec

ific

ity

R

equi

red

R

efer

to S

ubch

apte

r 6.

1.3.

2 an

d C

LS

I do

cum

ent E

P12

1 fo

r m

ore

info

rmat

ion.

The

acc

urac

y da

tase

t can

be

use

d to

cal

cula

te

clin

ical

spe

cifi

city

.

The

acc

urac

y da

tase

t can

be

use

d to

cal

cula

te

clin

ical

spe

cifi

city

.

The

acc

urac

y da

tase

t can

be

use

d to

cal

cula

te c

lini

cal

spec

ific

ity.

C

lin

ical

sen

siti

vity

Req

uire

d

Ref

er to

Sub

chap

ter

6.1.

3.3

and

CL

SI

docu

men

t EP

121

for

mor

e in

form

atio

n.

The

acc

urac

y da

tase

t can

be

use

d to

cal

cula

te

clin

ical

sen

siti

vity

.

The

acc

urac

y da

tase

t can

be

use

d to

cal

cula

te

clin

ical

sen

siti

vity

.

The

acc

urac

y da

tase

t can

be

use

d to

cal

cula

te c

lini

cal

sens

itiv

ity.

3459

H62

13

0

“DR

AF

T D

OC

UM

EN

T. T

his

draf

t CL

SI d

ocum

ent i

s no

t to

be re

prod

uced

or c

ircu

late

d fo

r an

y pu

rpos

e ot

her

than

rev

iew

and

com

men

t. It

is n

ot to

be

cons

ider

ed e

ithe

r fi

nal o

r pu

blis

hed

and

may

not

be

quot

ed o

r re

fere

nced

. 18

Sep

tem

ber

201

9.”

Tab

le A

5. (

Con

tin

ued

) 34

60

Par

amet

er

Com

men

ts

Sam

ple

s R

epli

cate

s A

nal

ytic

al R

un

s

Val

idat

ion

pla

n

R

equi

red

R

efer

to S

ubch

apte

r 6.

1 fo

r m

ore

info

rmat

ion.

V

alid

atio

n r

epor

t

Req

uire

d

Ref

er to

Sub

chap

ter

6.3

for

mor

e in

form

atio

n.

Q

A r

evie

w

Req

uire

d

Abb

revi

atio

ns:

CI,

con

fide

nce

inte

rval

; C

LIA

, Cli

nica

l L

abor

ator

y Im

prov

emen

ts A

men

dmen

ts o

f 19

88;

IMD

RF

, In

tern

atio

nal

Med

ical

Dev

ice

Reg

ulat

ors

For

um;

LL

OQ

, lo

wer

34

61

lim

it of

qua

ntif

icat

ion;

LO

D, l

imit

of

dete

ctio

n; Q

A, q

ualit

y as

sura

nce.

34

62

34

63

3464

H

62

131

“DR

AF

T D

OC

UM

EN

T. T

his

draf

t CL

SI d

ocum

ent i

s no

t to

be re

prod

uced

or c

ircu

late

d fo

r an

y pu

rpos

e ot

her

than

rev

iew

and

com

men

t. It

is n

ot to

be

cons

ider

ed e

ithe

r fi

nal o

r pu

blis

hed

and

may

not

be

quot

ed o

r re

fere

nced

. 18

Sep

tem

ber

201

9.”

Tab

le A

6. L

abor

ator

y-In

itia

ted

Ass

ay R

evis

ion

34

65

Ext

ent

of

Ch

ange

C

ateg

ory

of C

han

ge

Rev

isio

n

Rea

son

for

Ch

ange

R

equ

ired

QA

Pro

cess

M

inor

O

pera

tion

al

Sou

rce

of b

uffe

r or

oth

er s

impl

e re

agen

ts s

houl

d be

cha

nged

It

em n

o lo

nger

ava

ilab

le, o

r a

less

exp

ensi

ve a

lter

nati

ve is

av

aila

ble.

R

eage

nt q

uali

fica

tion

N =

5

Min

or

Ope

rati

onal

1

or m

ore

mar

kers

sho

uld

be o

mit

ted

from

a v

alid

ated

pan

el.

Sim

plif

icat

ion

of a

ssay

whe

n a

mar

ker

is n

ot a

ddin

g to

the

clin

ical

sen

siti

vity

/spe

cifi

city

of

assa

y

Non

e

Mod

erat

e O

pera

tion

al

An

anti

body

of

the

sam

e sp

ecif

icit

y,

but r

epre

sent

ing

a di

ffer

ent c

lone

co

njug

ated

to th

e sa

me

fluo

roch

rom

e,

shou

ld b

e us

ed a

s a

subs

titu

te.

Item

no

long

er a

vail

able

, or

a le

ss e

xpen

sive

alt

erna

tive

is

avai

labl

e.

V

erif

icat

ion

that

the

popu

latio

n id

enti

fied

and

th

e st

reng

th o

f th

e fl

uore

scen

t sig

nal a

re th

e sa

me

is r

equi

red

N

= 1

0M

oder

ate

Ope

rati

onal

/cli

nica

l A

n an

tibo

dy o

f a

diff

eren

t spe

cifi

city

sh

ould

be

subs

titu

ted

for

1 w

ith

the

sam

e fl

uoro

chro

me

wit

hout

req

uiri

ng

any

addi

tion

al c

hang

es in

the

anti

body

pan

el (

othe

r th

an le

avin

g ou

t an

anti

body

, eg,

cha

ngin

g an

an

tibo

dy o

n a

fluo

roch

rom

e w

ith

no

com

pens

atio

n is

sues

in th

e pa

nel)

.

Bet

ter

iden

tifi

cati

on o

f di

seas

e su

bcat

egor

ies,

eg,

sub

stit

utin

g C

D20

0 fo

r FM

C-7

in a

B-c

ell

pane

l

V

erif

icat

ion

that

ther

e is

no

dif

fere

nce

in th

e se

nsit

ivit

y an

d sp

ecif

icit

y fo

r di

seas

e is

req

uire

d

N =

20

(5 is

nor

mal

and

15

is a

bnor

mal

)

Mod

erat

e O

pera

tion

al/c

lini

cal

An

anti

body

of

the

sam

e sp

ecif

icity

, co

njug

ated

to a

dif

fere

nt

fluo

roch

rom

e w

itho

ut r

equi

ring

any

ad

diti

onal

cha

nges

in th

e an

tibo

dy

pane

l (ot

her

than

leav

ing

out a

n an

tibo

dy),

sho

uld

be u

sed

as a

su

bsti

tute

.

Bet

ter

iden

tifi

cati

on o

f di

seas

e

Ver

ific

atio

n th

at th

ere

is

no d

iffe

renc

e in

the

sens

itiv

ity

and

spec

ific

ity

for

dise

ase

is r

equi

red.

N =

20

(5 is

nor

mal

and

15

is a

bnor

mal

)

3466

H62

13

2

“DR

AF

T D

OC

UM

EN

T. T

his

draf

t CL

SI d

ocum

ent i

s no

t to

be re

prod

uced

or c

ircu

late

d fo

r an

y pu

rpos

e ot

her

than

rev

iew

and

com

men

t. It

is n

ot to

be

cons

ider

ed e

ithe

r fi

nal o

r pu

blis

hed

and

may

not

be

quot

ed o

r re

fere

nced

. 18

Sep

tem

ber

201

9.”

Tab

le A

6. (

Con

tin

ued

) 34

67

Ext

ent

of

Ch

ange

C

ateg

ory

of C

han

ge

Rev

isio

nR

easo

n f

or C

han

geR

equ

ired

QA

Pro

cess

Mod

erat

e O

pera

tion

al/c

lini

cal

An

anti

body

of

the

sam

e or

dif

fere

nt

spec

ific

ity,

con

juga

ted

to a

dif

fere

nt

fluo

roch

rom

e bu

t nec

essi

tati

ng

addi

tion

al c

hang

es in

the

anti

body

pa

nel (

beca

use

of p

oten

tial

co

mpe

nsat

ion

issu

es),

sho

uld

be

used

as

a su

bsti

tute

.

Bet

ter

iden

tifi

cati

on o

f di

seas

e

CL

IA/I

MD

R v

alid

atio

n

is r

equi

red.

N =

20

for

sens

itiv

ity/

spec

ific

ity

and

expe

rim

ents

for

pre

cisi

on,

repr

oduc

ibil

ity, L

OB

, LO

D,

and,

if q

uasi

quan

tita

tive

, LL

OQ

an

d li

near

ity

Mod

erat

e O

pera

tion

al/c

lini

cal

1 or

mor

e an

tibo

dies

of

nove

l sp

ecif

icit

y an

d fl

uoro

chro

me

shou

ld

be a

dded

to a

n ex

isti

ng p

anel

(e

g, g

oing

fro

m 8

to 1

0 co

lor)

.

Bet

ter

iden

tifi

cati

on o

f di

seas

e

CL

IA/I

MD

R v

alid

atio

n

is r

equi

red.

N =

20

for

sens

itiv

ity/

spec

ific

ity

and

expe

rim

ents

for

pre

cisi

on,

repr

oduc

ibil

ity, L

OB

, LO

D,

and,

if q

uasi

quan

tita

tive

, LL

OQ

an

d li

near

ity

Abb

revi

atio

ns:

CL

IA,

Cli

nica

l L

abor

ator

y Im

prov

emen

ts A

men

dmen

ts o

f 19

88;

IMD

RF

, In

tern

atio

nal

Med

ical

Dev

ice

Reg

ulat

ors

For

um;

LL

OQ

, lo

wer

lim

it of

qua

ntif

icat

ion;

34

68

LO

B, l

imit

of b

lank

; LO

D, l

imit

of d

etec

tion;

QA

, qua

lity

ass

uran

ce.

3469

3470

34

71

H

62

133

“DR

AF

T D

OC

UM

EN

T. T

his

draf

t CL

SI d

ocum

ent i

s no

t to

be re

prod

uced

or c

ircu

late

d fo

r an

y pu

rpos

e ot

her

than

rev

iew

and

com

men

t. It

is n

ot to

be

cons

ider

ed e

ithe

r fi

nal o

r pu

blis

hed

and

may

not

be

quot

ed o

r re

fere

nced

. 18

Sep

tem

ber

201

9.”

Tab

le A

7. F

ull

Val

idat

ion

Typ

e 1

Par

amet

er

Com

men

ts

Sam

ple

s R

epli

cate

s A

nal

ytic

al R

un

s A

ccu

racy

/tru

enes

s R

efer

to S

ubch

apte

rs 3

.2.1

and

6.

1.2.

1 fo

r m

ore

info

rmat

ion.

20 s

ampl

es

5

leve

ls if

pos

sibl

e (l

ow, m

ediu

m, h

igh,

an

d L

LO

Q)

1 or

mor

e 1

run

per

sam

ple

Sp

ecif

icit

y

Doc

umen

tati

on s

houl

d in

clud

e a

sum

mar

y of

the

assa

y de

velo

pmen

t wit

h re

gard

to s

peci

fici

ty, s

uch

as

the

refe

renc

es ju

stif

ying

the

choi

ce o

f an

tige

ns a

nd th

e ga

ting

str

ateg

y.

T

he la

bora

tory

sho

uld

prov

ide

docu

men

tati

on to

ju

stif

y th

at th

e de

tect

ion

anti

body

is s

peci

fic.

Ref

er to

Sub

chap

ters

5.1

and

6.

1.2.

2 fo

r m

ore

info

rmat

ion.

Add

itio

nal e

valu

atio

n is

no

t req

uire

d du

ring

va

lida

tion

.

Add

itio

nal e

valu

atio

n is

no

t req

uire

d du

ring

va

lida

tion

.

Add

itio

nal e

valu

atio

n is

no

t req

uire

d du

ring

va

lida

tion

.

Sel

ecti

vity

Req

uire

d on

ly if

test

ing

in

the

pres

ence

of

a bi

olog

ic

that

inte

rfer

es w

ith

the

assa

y.

R

efer

to S

ubch

apte

r 6.

1.2.

2 fo

r m

ore

info

rmat

ion.

6–10

sam

ples

3

1 ru

n pe

r sa

mpl

e

Sen

siti

vity

L

OD

Req

uire

d

Ref

er to

Sub

chap

ter

6.1.

2.3

for

mor

e in

form

atio

n.

10

1 or

mor

e 1

or m

ore

LL

OQ

Req

uire

d

Ref

er to

Sub

chap

ter

6.1.

2.3

for

mor

e in

form

atio

n.

3 (5

leve

ls e

ach)

3

1 ru

n pe

r sa

mpl

e

3472

H62

13

4

“DR

AF

T D

OC

UM

EN

T. T

his

draf

t CL

SI d

ocum

ent i

s no

t to

be re

prod

uced

or c

ircu

late

d fo

r an

y pu

rpos

e ot

her

than

rev

iew

and

com

men

t. It

is n

ot to

be

cons

ider

ed e

ithe

r fi

nal o

r pu

blis

hed

and

may

not

be

quot

ed o

r re

fere

nced

. 18

Sep

tem

ber

201

9.”

Tab

le A

7. (

Con

tin

ued

) 34

73

Par

amet

er

Com

men

tsS

amp

les

Rep

lica

tes

An

alyt

ical

Ru

ns

Pre

cisi

on

R

epea

tab

ilit

y

(In

tra-

assa

y)

R

equi

red

P

reci

sion

exp

erim

ents

sh

ould

be

repe

ated

by

each

op

erat

or o

n ea

ch

inst

rum

ent.

R

efer

to F

igur

e 17

and

S

ubch

apte

r 6.

1.2.

4 fo

r m

ore

info

rmat

ion.

Mus

t be

mat

rix

sam

ples

(d

isea

se-s

tate

and

/or

heal

thy

dono

r) (

n =

6)

3

6 to

tal r

uns

4

runs

per

sam

ple

(1 b

y ea

ch o

f 2

oper

ator

s on

eac

h of

2 in

stru

men

ts)

R

epro

du

cib

ilit

y

(In

tera

ssay

)

Req

uire

d

Pre

cisi

on e

xper

imen

ts

shou

ld b

e re

peat

ed b

y ea

ch

oper

ator

on

each

in

stru

men

t.

4 ru

ns p

er s

ampl

e

Ref

er to

Fig

ure

17 a

nd

Sub

chap

ter

6.1.

2.4

for

mor

e in

form

atio

n.

Mat

rix

sam

ples

(di

seas

e-st

ate

and/

or h

ealt

hy d

onor

) (n

= 6

) an

d Q

C (

n =

2)

mus

t be

used

3

6 to

tal r

uns

4

runs

per

sam

ple

(1 b

y ea

ch o

f 2

oper

ator

s on

eac

h of

2 in

stru

men

ts)

2

addi

tion

al r

uns

wit

h Q

C

or o

ther

sam

ples

I

nte

rop

erat

or

D

ata

gene

rate

d fr

om in

tra-

an

d in

tera

ssay

exp

erim

ents

m

ay b

e us

ed f

or th

e ca

lcul

atio

ns.

A

ddit

iona

l exp

erim

ents

are

no

t req

uire

d.

D

ata

gene

rate

d fr

om in

tra-

an

d in

tera

ssay

exp

erim

ents

m

ay b

e us

ed f

or th

e ca

lcul

atio

ns.

A

ddit

iona

l exp

erim

ents

are

no

t req

uire

d.

In

teri

nst

rum

ent

D

ata

gene

rate

d fr

om in

tra-

an

d in

tera

ssay

exp

erim

ents

m

ay b

e us

ed f

or th

e ca

lcul

atio

ns.

A

ddit

iona

l exp

erim

ents

are

no

t req

uire

d.

D

ata

gene

rate

d fr

om in

tra-

an

d in

tera

ssay

exp

erim

ents

m

ay b

e us

ed f

or th

e ca

lcul

atio

ns.

A

ddit

iona

l exp

erim

ents

are

no

t req

uire

d.

34

74

H

62

135

“DR

AF

T D

OC

UM

EN

T. T

his

draf

t CL

SI d

ocum

ent i

s no

t to

be re

prod

uced

or c

ircu

late

d fo

r an

y pu

rpos

e ot

her

than

rev

iew

and

com

men

t. It

is n

ot to

be

cons

ider

ed e

ithe

r fi

nal o

r pu

blis

hed

and

may

not

be

quot

ed o

r re

fere

nced

. 18

Sep

tem

ber

201

9.”

Tab

le A

7. (

Con

tin

ued

) 34

75

Par

amet

er

Com

men

ts

Sam

ple

s R

epli

cate

s A

nal

ytic

al R

un

s Q

C e

valu

atio

n

and

QC

ran

ges

Q

C r

ange

s ne

ed to

be

esta

blis

hed

befo

re

vali

dati

on.

2

QC

leve

ls a

re r

equi

red

for

prim

ary

vali

dati

on.

R

efer

to S

ubch

apte

rs 5

.1,

7.2.

4, a

nd 7

.3.3

for

mor

e in

form

atio

n.

2 3

Min

imum

of

18 d

ata

poin

ts o

ver

6 ru

ns

Lin

eari

ty

R

equi

red

L

inea

r re

gres

sion

ana

lysi

s of

dat

a fr

om L

LO

Q

expe

rim

ents

Ref

er to

Sub

chap

ter

6.1.

3.5

for

mor

e in

form

atio

n.

D

ata

from

LL

OQ

ex

peri

men

ts

A

lter

nati

vely

, dil

utio

nal

line

arit

y m

ay b

e do

ne b

y us

ing

high

-lev

el s

ampl

es

dilu

ting

into

the

posi

tive

ce

ll-d

eple

ted

sam

ples

.

Dat

a fr

om L

LO

Q

expe

rim

ents

can

be

used

.

Dat

a fr

om L

LO

Q

expe

rim

ents

can

be

used

.

Sta

bil

ity

Sp

ecim

en

Ref

er to

Sub

chap

ter

6.1.

2.6.

1 fo

r m

ore

info

rmat

ion.

6 2

1 ru

n pe

r ti

me

poin

t

Pro

cess

ed s

amp

le

Ref

er to

Sub

chap

ter

6.1.

2.6.

2 fo

r m

ore

info

rmat

ion.

3 2

1 ru

n pe

r ti

me

poin

t

Coc

kta

il s

tab

ilit

y R

efer

to S

ubch

apte

r 6.

1.2.

6.3

for

mor

e in

form

atio

n.3–

4

1 ru

n pe

r ti

me

poin

t

Cri

tica

l rea

gen

t st

abil

ity

An

appr

oach

sim

ilar

to

cock

tail

sta

bili

ty s

houl

d be

fo

llow

ed, a

s de

scri

bed

in

Sub

chap

ter

6.1.

2.6.

3.

3 1

1 ru

n pe

r ti

me

poin

t

QC

sta

bil

ity

An

appr

oach

sim

ilar

to

cock

tail

sta

bili

ty s

houl

d be

fo

llow

ed, a

s de

scri

bed

in

Sub

chap

ter

6.1.

2.6.

3.

3 1

1 ru

n pe

r ti

me

poin

t

3476

H62

13

6

“DR

AF

T D

OC

UM

EN

T. T

his

draf

t CL

SI d

ocum

ent i

s no

t to

be re

prod

uced

or c

ircu

late

d fo

r an

y pu

rpos

e ot

her

than

rev

iew

and

com

men

t. It

is n

ot to

be

cons

ider

ed e

ithe

r fi

nal o

r pu

blis

hed

and

may

not

be

quot

ed o

r re

fere

nced

. 18

Sep

tem

ber

201

9.”

Tab

le A

7. (

Con

tin

ued

) 34

77

Par

amet

er

Com

men

ts

Sam

ple

s R

epli

cate

s A

nal

ytic

al R

un

s C

arry

over

L

ow-

and

high

-sam

ples

sh

ould

be

eval

uate

d se

quen

tial

ly.

2 or

mor

e (l

ow a

nd h

igh)

3

(alt

erna

ting

low

and

hi

gh)

1

Ref

eren

ce in

terv

als

Ref

er to

Sub

chap

ters

3.2

.8

and

6.1.

2.8

for

mor

e in

form

atio

n.

Not

req

uire

d N

ot r

equi

red

Not

req

uire

d

Doc

um

enta

tion

Val

idat

ion

pla

n

R

equi

red

R

efer

to S

ubch

apte

r 6.

1 fo

r m

ore

info

rmat

ion.

Val

idat

ion

rep

ort

R

equi

red

R

efer

to S

ubch

apte

r 6.

3 fo

r m

ore

info

rmat

ion.

QA

rev

iew

R

equi

red

A

bbre

viat

ions

: LL

OQ

, low

er li

mit

of

quan

tific

atio

n; L

OD

, lim

it of

det

ecti

on; Q

A, q

uali

ty a

ssur

ance

; QC

, qua

lity

cont

rol.

3478

3479

34

80

H

62

137

“DR

AF

T D

OC

UM

EN

T. T

his

draf

t CL

SI d

ocum

ent i

s no

t to

be re

prod

uced

or c

ircu

late

d fo

r an

y pu

rpos

e ot

her

than

rev

iew

and

com

men

t. It

is n

ot to

be

cons

ider

ed e

ithe

r fi

nal o

r pu

blis

hed

and

may

not

be

quot

ed o

r re

fere

nced

. 18

Sep

tem

ber

201

9.”

Tab

le A

8. F

ull

Val

idat

ion

Typ

e 2

3481

P

aram

eter

C

omm

ents

S

amp

les

Rep

lica

tes

An

alyt

ical

Ru

ns

Acc

ura

cy/t

ruen

ess

Com

pari

son

wit

h co

nfir

med

cl

inic

al d

iagn

osis

and

/or

prev

ious

ly a

ppro

ved

(pre

dica

te)

assa

y or

dev

ice

shou

ld b

e co

nduc

ted.

T

he n

umbe

r sh

ould

be

dete

rmin

ed d

urin

g di

scus

sion

s w

ith

regu

lato

ry a

genc

ies

and

can

rang

e fr

om 2

0–20

0.

M

ulti

ple-

site

stu

dies

are

re

com

men

ded.

1 1

per

sam

ple

Sp

ecif

icit

y

Doc

umen

tati

on s

houl

d in

clud

e a

sum

mar

y of

the

assa

y de

velo

pmen

t wit

h re

gard

to s

peci

fici

ty, s

uch

as th

e re

fere

nces

just

ifyi

ng

the

choi

ce o

f an

tige

ns a

nd

the

gati

ng s

trat

egy.

Man

ufac

ture

rs n

eed

to a

lso

prov

ide

docu

men

tati

on o

f th

e sp

ecif

icit

y co

nfir

mat

ion

of e

ach

mA

b.

R

efer

to S

ubch

apte

rs 5

.1

and

6.1.

2.2

for

mor

e in

form

atio

n.

N/A

N

/A

N/A

Sel

ecti

vity

M

anuf

actu

rers

sho

uld

test

for

th

e ef

fect

of

inte

rfer

ing

subs

tanc

es th

at m

ay o

ccur

in

poor

-qua

lity

sam

ples

, suc

h as

bu

t not

lim

ited

to, h

emol

ysis

, li

pem

ic s

ampl

es, a

nd ic

teri

c sa

mpl

es.

3 pe

r co

ndit

ion

3 1

or m

ore

3482

H62

13

8

“DR

AF

T D

OC

UM

EN

T. T

his

draf

t CL

SI d

ocum

ent i

s no

t to

be re

prod

uced

or c

ircu

late

d fo

r an

y pu

rpos

e ot

her

than

rev

iew

and

com

men

t. It

is n

ot to

be

cons

ider

ed e

ithe

r fi

nal o

r pu

blis

hed

and

may

not

be

quot

ed o

r re

fere

nced

. 18

Sep

tem

ber

201

9.”

Tab

le 8

A. (

Con

tin

ued

) 34

83

Par

amet

er

Com

men

tsS

amp

les

Rep

lica

tes

An

alyt

ical

Ru

ns

Sen

siti

vity

LO

D/L

OB

Req

uire

d

Ref

er to

Sub

chap

ter

6.1.

2.3.

1 fo

r m

ore

info

rmat

ion.

10 (

5 ne

gati

ve a

nd 5

low

-po

siti

ve s

ampl

es)

5 (e

ach

repl

icat

e ca

n be

ac

quir

ed 5

tim

es to

in

crea

se th

e st

atis

tica

l po

wer

)

A

ll th

e sa

mpl

es s

houl

d no

t be

test

ed o

n th

e sa

me

day.

Tes

ting

sho

uld

be s

prea

d ou

t ove

r a

min

imum

of

3

days

and

/or

mul

tipl

e si

tes.

L

LO

Q

R

equi

red

R

efer

to S

ubch

apte

r 6.

1.2.

3.2

for

mor

e in

form

atio

n.

≥ 5

sam

ples

nea

r th

e L

LO

Q in

trip

lica

te

3 1

or m

ore

R

epea

tab

ilit

y

(In

tra-

assa

y)

R

equi

red

P

reci

sion

exp

erim

ents

sh

ould

be

repe

ated

by

each

op

erat

or o

n ea

ch

inst

rum

ent.

R

efer

to F

igur

e 17

and

S

ubch

apte

r 6.

1.2.

4 fo

r m

ore

info

rmat

ion.

M

atri

x sa

mpl

es

(dis

ease

-sta

te a

nd/o

r he

alth

y do

nor)

(n

=10

−

20)

In

clud

ing

leve

ls a

t or

near

LL

OQ

is a

lso

impo

rtan

t if

assa

y is

in

tend

ed to

det

ect r

are

even

ts.

3 4

runs

per

sam

ple

(1 b

y ea

ch o

f 2

oper

ator

s on

ea

ch o

f 2

inst

rum

ents

)

R

epro

du

cib

ilit

y

(In

tera

ssay

)

Req

uire

d

Ref

er to

Sub

chap

ter

6.1.

2.3

for

mor

e in

form

atio

n.

≥ 5

sam

ples

nea

r th

e L

LO

Q in

trip

lica

te

3 1

or m

ore

I

nte

rop

erat

or

R

epea

tabi

lity

exp

erim

ents

sh

ould

be

repe

ated

by

each

op

erat

or o

n ea

ch

inst

rum

ent.

4

runs

per

sam

ple

D

ata

gene

rate

d fr

om

intr

a- a

nd in

tera

ssay

ex

peri

men

ts m

ay b

e us

ed f

or th

e ca

lcul

atio

ns.

A

ddit

iona

l exp

erim

ents

ar

e no

t req

uire

d.

D

ata

gene

rate

d fr

om

intr

a- a

nd in

tera

ssay

ex

peri

men

ts m

ay b

e us

ed f

or th

e ca

lcul

atio

ns.

A

ddit

iona

l exp

erim

ents

ar

e no

t req

uire

d.

D

ata

gene

rate

d fr

om

intr

a- a

nd in

tera

ssay

ex

peri

men

ts m

ay b

e us

ed f

or th

e

calc

ulat

ions

.

Add

itio

nal e

xper

imen

ts

are

not r

equi

red.

3484

H

62

139

“DR

AF

T D

OC

UM

EN

T. T

his

draf

t CL

SI d

ocum

ent i

s no

t to

be re

prod

uced

or c

ircu

late

d fo

r an

y pu

rpos

e ot

her

than

rev

iew

and

com

men

t. It

is n

ot to

be

cons

ider

ed e

ithe

r fi

nal o

r pu

blis

hed

and

may

not

be

quot

ed o

r re

fere

nced

. 18

Sep

tem

ber

201

9.”

Tab

le 8

A. (

Con

tin

ued

) 34

85

Par

amet

er

Com

men

ts

Sam

ple

s R

epli

cate

s A

nal

ytic

al R

un

s

In

teri

nst

rum

ent

R

epea

tabi

lity

ex

peri

men

ts s

houl

d be

re

peat

ed b

y ea

ch

oper

ator

on

each

in

stru

men

t.

4 ru

ns p

er s

ampl

e

D

ata

gene

rate

d fr

om in

tra-

and

in

tera

ssay

exp

erim

ents

may

be

used

for

the

calc

ulat

ions

.

Add

itio

nal e

xper

imen

ts a

re n

ot

requ

ired

.

D

ata

gene

rate

d fr

om in

tra-

and

in

tera

ssay

ex

peri

men

ts

may

be

used

for

th

e ca

lcul

atio

ns.

A

ddit

iona

l ex

peri

men

ts a

re

not r

equi

red.

D

ata

gene

rate

d fr

om in

tra-

an

d in

tera

ssay

exp

erim

ents

m

ay b

e us

ed f

or th

e ca

lcul

atio

n.

A

ddit

iona

l exp

erim

ents

are

no

t req

uire

d.

QC

eva

luat

ion

an

d

QC

ran

ges

2

QC

leve

ls a

re r

equi

red

R

efer

to S

ubch

apte

rs 5

.1,

7.2.

4, a

nd 7

.3.3

for

mor

e in

form

atio

n.

2 or

mor

e 3

Min

imum

of

18 d

ata

poin

ts

over

6 r

uns

Lin

eari

ty

R

equi

red

L

inea

r re

gres

sion

an

alys

is

R

efer

to S

ubch

apte

r 6.

1.3

for

mor

e in

form

atio

n.

The

re a

re s

ever

al a

ppro

ache

s:

D

ata

from

LL

OQ

Dis

ease

-sta

te s

ampl

es s

pike

d in

to h

ealt

hy s

ampl

es

A

lter

nati

vely

, dil

utio

nal

line

arit

y m

ay b

e co

nduc

ted

by

usin

g hi

gh-l

evel

sam

ples

di

luti

ng in

to th

e po

siti

ve c

ell-

depl

eted

sam

ples

Oth

er a

ppro

ache

s m

ay b

e ac

cept

able

.

3 1

or m

ore

Mea

sure

men

t/

rep

orta

ble

ran

ge

T

he u

pper

and

low

er

lim

it in

put c

ell n

umbe

r or

vo

lum

e sh

ould

be

esta

blis

hed

duri

ng a

ssay

de

velo

pmen

t.

Dat

a fr

om L

LO

Q a

nd

prec

isio

n ev

alua

tion

sh

ould

be

used

to

esta

blis

h th

e up

per

and

low

er r

epor

ting

lim

its.

N/A

N

/A

N/A

H62

14

0

“DR

AF

T D

OC

UM

EN

T. T

his

draf

t CL

SI d

ocum

ent i

s no

t to

be re

prod

uced

or c

ircu

late

d fo

r an

y pu

rpos

e ot

her

than

rev

iew

and

com

men

t. It

is n

ot to

be

cons

ider

ed e

ithe

r fi

nal o

r pu

blis

hed

and

may

not

be

quot

ed o

r re

fere

nced

. 18

Sep

tem

ber

201

9.”

Tab

le 8

A. (

Con

tin

ued

) 34

86

Par

amet

er

Com

men

ts

Sam

ple

s R

epli

cate

s A

nal

ytic

al R

un

s S

tab

ilit

y S

pec

imen

The

tim

e po

int s

houl

d ex

tend

1

poin

t bey

ond

whe

re

stab

ilit

y is

acc

epta

ble.

Ref

er to

Sub

chap

ter

6.1.

2.6.

1 fo

r m

ore

info

rmat

ion.

6 1

1 ru

n pe

r ti

me

poin

t

Pro

cess

ed s

amp

le

T

he ti

me

poin

t sho

uld

exte

nd

1 po

int b

eyon

d w

here

st

abil

ity

is a

ccep

tabl

e.

R

efer

to S

ubch

apte

r 6.

1.2.

6.2

for

mor

e in

form

atio

n.

3 1

1 ru

n pe

r ti

me

poin

t

Coc

kta

il s

tab

ilit

y

Ref

er to

Sub

chap

ter

6.1.

2.6.

3 fo

r m

ore

info

rmat

ion.

10

1

run

per

tim

e po

int

Cri

tica

l rea

gen

t st

abil

ity

A

n ap

proa

ch s

imil

ar to

co

ckta

il s

tabi

lity

sho

uld

be

foll

owed

, as

desc

ribe

d in

S

ubch

apte

r 6.

1.2.

6.3.

10

1 1

run

per

tim

e po

int

QC

sta

bil

ity

A

n ap

proa

ch s

imil

ar to

co

ckta

il s

tabi

lity

sho

uld

be

foll

owed

, as

desc

ribe

d in

S

ubch

apte

r 6.

1.2.

6.3.

10

1 1

run

per

tim

e po

int

Car

ryov

er

L

ow a

nd h

igh

sam

ples

sh

ould

be

eval

uate

d se

quen

tial

ly.

2 or

mor

e (l

ow a

nd h

igh)

3

(alt

erna

ting

low

and

hig

h)

1

3487

H

62

141

“DR

AF

T D

OC

UM

EN

T. T

his

draf

t CL

SI d

ocum

ent i

s no

t to

be re

prod

uced

or c

ircu

late

d fo

r an

y pu

rpos

e ot

her

than

rev

iew

and

com

men

t. It

is n

ot to

be

cons

ider

ed e

ithe

r fi

nal o

r pu

blis

hed

and

may

not

be

quot

ed o

r re

fere

nced

. 18

Sep

tem

ber

201

9.”

Tab

le 8

A. (

Con

tin

ued

) 34

88

Par

amet

er

Com

men

ts

Sam

ple

s R

epli

cate

s A

nal

ytic

al R

un

s R

efer

ence

inte

rval

s

Req

uire

d

Ref

er to

Sub

chap

ter

6.1.

2.8

and

CL

SI

docu

men

t EP

28.2

12

0 (6

0 m

ale,

60

fem

ale)

Age

s re

late

to th

e in

tent

of

the

test

(p

edia

tric

or

adul

t)

A

ddit

iona

l sam

ples

ar

e re

quir

ed if

di

ffer

ence

s ar

e ob

serv

ed b

etw

een

grou

ps.

1 1

run

per

sam

ple

Cli

nic

al v

alid

atio

n

Doc

umen

tati

on th

at th

e te

st

resu

lts

are

cons

iste

nt w

ith

clin

ical

dia

gnos

is d

eter

min

ed

by o

ther

test

res

ults

sho

uld

be

prov

ided

.

If n

eces

sary

, tes

t res

ults

sh

ould

dem

onst

rate

that

they

ar

e ap

plic

able

to th

e sp

ecif

ic

clin

ical

dia

gnos

is b

ut n

ot

othe

rs.

T

he n

umbe

r sh

ould

be

det

erm

ined

dur

ing

disc

ussi

ons

wit

h re

gula

tory

age

ncie

s an

d ca

n ra

nge

from

20

–200

.

Mul

tipl

e-si

te s

tudi

es

are

reco

mm

ende

d.

S

ampl

es m

ust i

nclu

de

inte

nded

use

(d

isea

se-s

tate

sp

ecif

ic)

and

heal

thy

or p

atie

nt s

ampl

es

from

a d

iffe

rent

di

seas

e st

ate.

1 or

mor

e 1

run

per

sam

ple

Doc

um

enta

tion

V

alid

atio

n p

lan

Req

uire

d

Ref

er to

Sub

chap

ter

6.1

for

mor

e in

form

atio

n.

Val

idat

ion

rep

ort

R

equi

red

R

efer

to S

ubch

apte

r 6.

3 fo

r m

ore

info

rmat

ion.

QA

rev

iew

R

equi

red

Abb

revi

atio

ns: L

LO

Q, l

ower

lim

it o

f qu

antif

icat

ion;

LO

B, l

imit

of b

lank

; LO

D, l

imit

of

dete

ctio

n; m

Ab,

mon

oclo

nal a

ntib

ody;

QA

, qua

lity

ass

uran

ce.

3489

H62


References for Appendix A 3490 3491 1 CLSI. User Protocol for Evaluation of Qualitative Test Performance; Approved Guideline—Second 3492

Edition. CLSI document EP12-A2. Wayne, PA: Clinical and Laboratory Standards Institute; 2008. 3493 3494 2 CLSI. Defining, Establishing, and Verifying Reference Intervals in the Clinical Laboratory; Approved 3495

Guideline—Third Edition. CLSI document EP28-A3c. Wayne, PA: Clinical and Laboratory Standards 3496 Institute; 2008. 3497

3498 3499

H62

143


Appendix B. Critical Reagents Beyond Antibodies 3500 3501 Abbreviations for Appendix B 3502 3503 7-AAD 7-amino-actinomycin D 3504 DAPI 4′6-diamidino-2-phenylindole 3505 3506 In addition to antibodies, ancillary reagents such as viability dyes and permeabilization buffers are used. 3507 Commonly used reagents are summarized below.1,2,3,4,5 3508 3509 B1 7-amino-actinomycin D 3510 3511 7-amino-actinomycin D (7-AAD) is unable to cross intact cell membranes but binds to DNA in dead/dying 3512 cells that have lost membrane integrity. 7-AAD exclusion allows for the discrimination between viable cells 3513 and dead cells. This assessment is important to understand the integrity of the sample to be tested. 7-AAD 3514 cannot be used for live/dead cell discrimination if samples are to be fixed or permeabilized as part of the 3515 assay procedure.6 3516 3517 B2 Fixable Viability Dyes 3518 3519 This class of viability dyes includes protein-reactive dyes that enter compromised cells and bind covalently 3520 to intracellular proteins. Such dyes also bind to cell surface proteins, but with much lower intensities, 3521 allowing discrimination between live and dead cells as long as they are added to the sample before fixation. 3522 They are commercially available in a wide variety of colors and provide greater flexibility in panel design. 3523 They cannot be used for nucleated identification, however, because they do not bind to DNA. 3524 3525 B3 4′6-diamidino-2-phenylindole 3526 3527 4′6-diamidino-2-phenylindole (DAPI) is unable to cross intact cell membranes and can be used to provide 3528 percent viability in combination with other markers to provide immunophenotypic cell characteristics.7 3529 3530 B4 Propidium Iodide 3531 3532 Propidium iodide is excluded by cells with intact membranes and can, therefore, be used for viability 3533 assessment in a manner similar to 7-AAD or DAPI. Additionally, in early studies, it was used for the 3534 assessment of DNA content.8 3535 3536 B5 Detergents and Solvents 3537 3538 Permeabilization reagents are necessary for the identification and measurement of antigens located within 3539 the cell. The permeabilization reagent works in tandem with the fixation process to allow fluorescent-3540 labeled antibodies to infiltrate the cellular membrane and bind to intracellular antigens. There are two 3541 classes of permeabilization reagents. Detergents such as saponin, NP-40, Triton X-100, or Tween-20 may 3542 be used for samples that require lysing. Alternatively, organic solvents such as methanol, acetone, or ethanol 3543 dissolve the lipids from the cell membrane allowing for permeabilization.9 3544 3545 3546

H62

144


References for Appendix B 3547 3548 1 Dezern AE, Borowitz MJ. ICCS/ESCCA consensus guidelines to detect GPI-deficient cells in 3549

paroxysmal nocturnal hemoglobinuria (PNH) and related disorders part 1 - clinical utility. Cytometry 3550 B Clin Cytom. 2018;94(1):16-22. 3551

3552 2 Illingworth A, Marinov I, Sutherland DR, Wagner-Ballon O, DelVecchio L. ICCS/ESCCA consensus 3553

guidelines to detect GPI-deficient cells in paroxysmal nocturnal hemoglobinuria (PNH) and related 3554 disorders part 3 - data analysis, reporting and case studies. Cytometry B Clin Cytom. 2018;94(1):49-66. 3555

3556 3 Marinov I, Illingworth AJ, Benko M, Sutherland DR. Performance characteristics of a non-fluorescent 3557

aerolysin-based paroxysmal nocturnal hemoglobinuria (PNH) assay for simultaneous evaluation of 3558 PNH neutrophils and PNH monocytes by flow cytometry, following published PNH guidelines. 3559 Cytometry B Clin Cytom. 2018;94(2):257-263. 3560

3561 4 Oldaker T, Whitby L, Saber M, Holden J, Wallace PK, Litwin V. ICCS/ESCCA consensus guidelines 3562

to detect GPI-deficient cells in paroxysmal nocturnal hemoglobinuria (PNH) and related disorders part 3563 4 - assay validation and quality assurance. Cytometry B Clin Cytom. 2018;94(1):67-81. 3564

3565 5 Sutherland DR, Illingworth A, Marinov I, et al. ICCS/ESCCA consensus guidelines to detect GPI-3566

deficient cells in paroxysmal nocturnal hemoglobinuria (PNH) and related disorders part 2 - reagent 3567 selection and assay optimization for high-sensitivity testing. Cytometry B Clin Cytom. 2018;94(1):23-3568 48. 3569

3570 6 Sutherland DR, Anderson L, Keeney M, Nayar R, Chin-Yee I. The ISHAGE guidelines for CD34+ cell 3571

determination by flow cytometry. International Society of Hematotherapy and Graft Engineering. J 3572 Hematother. 1996;5(3):213-226. 3573

3574 7 Davis BH, Dasgupta A, Kussick S, Han JY, Estrellado A; ICSH/ICCS Working Group. Validation of 3575

cell‐based fluorescence assays: practice guidelines from the ICSH and ICCS - Part II - preanalytical 3576 issues. Cytometry B Clin Cytom. 2013:84(5):286-290. 3577

3578 8 Krishan A. Rapid flow cytofluorometric analysis of mammalian cell cycle by propidium iodide 3579

staining. J Cell Biol. 1975;66(1):188-193. 3580 3581 9 Tanqri S, Vall H, Kaplan D, et al. Validation of cell-based fluorescence assays: practice guidelines 3582

from the ICSH and ICCS - part III - analytical issues. Cytometry B Clin Cytom. 2013;84(5):291-308. 3583 3584

H62

145


Appendix C. Temperature Monitoring Options 3585 3586 Table C1. Examples of Temperature Monitoring Options 3587

Technology Features Advantages (+)/Disadvantages (−) Temperature indicators

Irreversible dye change that indicates exposure outside of a temperature range or duration

Flat profile (+) Inexpensive (+) Lightweight (< 30g) (+) Shows only prolonged temperature excursions (−) Single use (−)

Temperature recorders

Portable chart recorder that generates a temperature record paper strip

Chart documents complete temperature history (+) Multi- and single-use options (+) Moderately priced (−) Typically weighs several ounces (−)

Temperature loggers

Digitally records temperatures at frequent intervals throughout the shipment

Flat profile (+) Lightweight (< 30g) (+) Multi- and single-use options (+) Reader interface: data are downloaded to a

computer for storage and analysis (a specialized reader and analysis software is needed) (+/−)

USB interface: data can be downloaded in PDF format (+/−)

Moderately priced (−) Abbreviations: PDF, portable document format; USB, universal serial bus. 3588

H62

146


Appendix D. Precision Statistical Discussion 3589 3590 Abbreviations for Appendix D 3591 3592 ANOVA analysis of variance 3593 SD standard deviation 3594 SDerror standard deviation of within-run error 3595 SDwl standard deviation of total error 3596 3597 D1 Additional Details on the Mixed-Effect ANOVA Model 3598 3599 As an example, assuming that the factorial design has only two factors: instrument and operator, the mixed 3600 model is given as shown in equation (D1): 3601 3602

𝑌 𝜇 𝛼 𝛽 𝜀 (D1) 3603

3604 Where: 3605 𝑌 is the observation from ith instrument, jth operator, kth replicate; 3606 𝜇 is the grand sample mean, 𝛼 is the instrument effect; 3607 𝛽 is the operator effect, 𝜀 is the repeatability error. 3608 3609 The number and type of factors in the factorial design can vary; this two-factor design is an example to 3610 explain the methodology. 3611 3612 The ANOVA model allows the determination of the variance components contributed by different factors; 3613 it also allows the determination of repeatability variation. From this model, the repeatability SD of within-3614 run error (SDerror) corresponds directly to the residual or error-term variance component, while the 3615 reproducibility SD of total error (SDwl) corresponds to the sum of all three variance components that are 3616 due to operator, instruments, and repeatability error, as shown in equation (D2). 3617 3618

𝑆𝐷 𝑉 𝑉 𝑉 (D2) 3619

3620 These variance components may be re-expressed as % CVs by dividing by the grand mean of all 3621 measurement results for the sample and then multiplying by 100, as shown in equation (D3). 3622 3623

𝐶𝑉 100% (D3) 3624

3625 3626

H62

147


D2 Variance Component Analysis 3627 3628 Figure D1 is a graphic display of sample data using variance component analysis. The sample data using 3629 variance component analysis are displayed in Table D1 according to the study design. 3630 3631

3632 Figure D1. Variance Component Analysis 3633 3634 Table D1. Variance Component Analysis 3635

Sample Variance Component Variance % Total SD % CV 1 Total 6.42 100.00 2.53 33.04 1 Instrument 0.00 0.00 0.00 0.00 1 Operator 4.33 67.50 2.08 27.15 1 Error 2.09 32.50 1.44 18.84 2 Total 16.09 100.00 4.01 18.46 2 Instrument 0.00 0.00 0.00 0.00 2 Operator 9.79 60.86 3.13 14.40 2 Error 6.30 39.14 2.51 11.55 3 Total 2.87 100.00 1.69 12.01 3 Instrument 0.00 0.00 0.00 0.00 3 Operator 0.23 8.16 0.48 3.43 3 Error 2.63 91.84 1.62 11.51

Abbreviations: % CV, coefficient of variation expressed as a percentage; SD, standard deviation. 3636 3637

H62

148


D3 An Alternative Approach 3638 3639 A simple alternative approach is explained as follows. For each sample, the intrarun variance (repeatablity 3640 variance) is the pooled average of the four variances of the triplicates, as shown in equation (D4). 3641

3642

𝑆𝐷3 1 𝑠𝑑 3 1 𝑠𝑑 3 1 𝑠𝑑 3 1 𝑠𝑑

12 1 3643

3644 Where sd , i 1, 2, 3, 4 is the SD of the three replicates within each analytical run. The repeatability CV 3645 is shown in equation (D3). 3646 3647 The above calculation is performed for each sample. The CV is calculated for each sample, and all the CVs 3648 will be reported. Each CV value should be no greater than the predefined threshold. Table D2 provides the 3649 raw data from which the analysis results in section D1 were derived. 3650 3651 Table D2. Two-Factor Precision Study 3652

Sample ID Run Instrument Operator Replicate 1 Replicate 2 Replicate 3 1 1 1 1 5.7 4.8 5.9 1 2 1 2 8 8.2 10.2 1 3 2 1 6.7 5.2 8.2 1 4 2 2 9.1 12 8 2 1 1 1 18 17.9 21.6 2 2 1 2 22.8 20.2 24.7 2 3 2 1 18 19.2 21.6 2 4 2 2 22 29 25.8 3 1 1 1 15.2 15.9 12.9 3 2 1 2 13.2 12.7 12.9 3 3 2 1 14.9 17.2 12.9 3 4 2 2 15.9 12.5 13

Abbreviation: ID, identification. 3653

(D4)

H62

149


Appendix E. Evaluation of Cocktail Stability 3654 3655 Table E1. Evaluation of Cocktail Stability 3656

Sample ID/Time Point Individually Pipetted

Antibodies Cocktail Date % CV Pass/Fail

Baseline Fresh Fresh

A

B

C

D

Week 1 Fresh 1 week stored at 4C A

B

C

D

Week 2 Fresh 2 weeks stored at 4C A

B

C

D


B

C

D


B

C

D Abbreviations: % CV, coefficient of variation expressed as a percentage; ID, identification. 3657

H62

150


Appendix F. Additional Tables for Qualitative Validation 3658 3659 In the example shown in Table F1, concordance is 100%. 3660 3661 Table F1. Example of Qualitative Reproducibility 3662

Sample ID Run 1 Run 2 Operator 1 Operator 2

Concordant or

Discordant 1 Positive Positive Positive Positive Concordant

2 Negative Negative Negative Negative Concordant

3 Positive Positive Positive Positive Concordant



Abbreviation: ID, identification. 3663 3664

Concordance C C D 100% (F1) 3665 3666 Where: 3667 C = false positives 3668 D = true negatives 3669 3670 In the example shown in Table F2, concordance is 100%. 3671 3672 Table F2. Example of Qualitative Reproducibility 3673

Replicate Results

Positive Negative

Ori

gin

al R

esu

lt

Pos

itiv

e

12 0

Neg

ativ

e

0 8

3674

Concordance A D A B C D 100% (F2) 3675 3676 Where: 3677 A = true positives 3678 B = false negatives 3679 C = false positives 3680 D = true negatives 3681 3682 3683

H62

151


In the example shown in Table F3, concordance is 60% because three of five samples are concordant. See 3684 equation (F1). 3685 3686 Table F3. Example of Qualitative Reproducibility 3687

Sample ID

Operator 1, Run 1

Operator 1, Run 2

Operator 2, Run 1

Operator 2, Run 2

Concordant or

Discordant 1 Positive Positive Positive Positive Concordant

2 Negative Negative Negative Positive Discordant


4 Positive Negative Positive Positive Discordant


Abbreviation: ID, identification. 3688 3689 In the example shown in Table F4, concordance is 90% because 18 of 20 samples are concordant. See 3690 equation (F2). 3691 3692 Table F4. Example of Qualitative Reproducibility 3693

Replicate Results

Positive Negative

Ori

gin

al R

esu

lt

Pos

itiv

e

11 1

Neg

ativ

e

1 7

3694 In the example shown in Table F5, the readout is CD19+, CD5+ clonal lymphocytes, and the concordance 3695 is 100%. See equation (F1). 3696 3697 Table F5. Example of Concordance at the Population Level 3698

Diagnosis Operator 1,

Run 1 Operator 1,

Run 2 Operator 2,

Run 1 Operator 2,

Run 2

Concordant or

Discordant CLL Positive Positive Positive Positive Concordant

Healthy donor Negative Negative Negative Negative Concordant

Abbreviation: CLL, chronic lymphoblastic leukemia. 3699 3700

3701

H62

152


The Quality Management System Approach 3702 3703 Clinical and Laboratory Standards Institute (CLSI) subscribes to a quality management system (QMS) approach in 3704 the development of standards and guidelines that facilitates project management, defines a document structure using 3705 a template, and provides a process to identify needed documents. The QMS approach applies a core set of “quality 3706 system essentials” (QSEs), basic to any organization, to all operations in any health care service’s path of workflow 3707 (ie, operational aspects that define how a particular product or service is provided). The QSEs provide the framework 3708 for delivery of any type of product or service, serving as a manager’s guide. The QSEs are: 3709 3710 Organization and Leadership Customer Focus Facilities and Safety

Management Personnel Management

Supplier and Inventory Management

Equipment Management Process Management Documents and Records

Management

Information Management Nonconforming Event Management Assessments Continual Improvement

3711

Org

aniz

atio

n an

d L

eade

rshi

p

Cus

tom

er

Foc

us

Fac

ilit

ies

and

Saf

ety

Man

agem

ent

Per

sonn

el

Man

agem

ent

Sup

plie

r an

d In

vent

ory

Man

agem

ent

Equ

ipm

ent

Man

agem

ent

Pro

cess

M

anag

emen

t

Doc

umen

ts a

nd

Rec

ords

M

anag

emen

t

Info

rmat

ion

Man

agem

ent

Non

conf

orm

ing

Eve

nt

Man

agem

ent

Ass

essm

ents

Con

tinu

al

Impr

ovem

ent

X AUTO13 AUTO13 AUTO13 AUTO13 AUTO13 AUTO13 AUTO13 AUTO13 AUTO13 AUTO13 AUTO13

EP05

EP06

EP07

EP12

EP14 EP23

EP28

H42 H42

H43

H52

M29

NBS05

QMS01 QMS01 QMS01 QMS01 QMS01 QMS01 QMS01 QMS01 QMS01 QMS01 QMS01 QMS01

QMS03

3712 3713

H62

153


Path of Workflow 3714 3715 A path of workflow is the description of the necessary processes to deliver the particular product or service that the 3716 organization or entity provides. A laboratory path of workflow consists of the sequential processes: preexamination, 3717 examination, and postexamination and their respective sequential subprocesses. All laboratories follow these 3718 processes to deliver their services, namely quality laboratory information. 3719 3720 H62 covers the medical laboratory path of workflow process indicated by an “X.” For a description of the other 3721 documents listed in the grid, please refer to the Related CLSI Reference Materials section. 3722 3723

Preexamination Examination Postexamination

Exa

min

atio

n or

deri

ng

Spe

cim

en

coll

ecti

on

Spe

cim

en

tran

spor

t

Spe

cim

en r

ecei

pt,

acce

ssio

ning

, and

pr

oces

sing

Exa

min

atio

n m

etho

d se

lect

ion

Exa

min

atio

n pe

rfor

man

ce

Res

ults

rev

iew

an

d fo

llow

-up

Lab

orat

ory

resu

lts

inte

rpre

tati

on

Com

mun

icat

ion

of

aler

t val

ues

and

issu

ance

of

prel

imin

ary

repo

rts

Rel

ease

of

fina

l re

port

s

Spe

cim

en

man

agem

ent

H42 H43

QMS01

H42 H43 H52

NBS05 QMS01

H42 H43

NBS05 QMS01

H42 H43 H52

QMS01

QMS01

X EP07

EP23 H42 H43 H52

NBS05 QMS01

EP23 H42 H43 H52

NBS05 QMS01

EP14

EP23 H42 H43 H52

NBS05 QMS01

QMS01

H42 H43

NBS05 QMS01

QMS01 3724

3725

H62

154


Related CLSI Reference Materials 3726 3727 AUTO13 Laboratory Instrument and Data Management Systems: Design of Software User Interfaces and End-

User Software Systems Validation, Operation, and Monitoring. 2nd ed., 2003. This document identifies important factors that designers and laboratory managers should consider when developing new software-driven systems and selecting software user interfaces. Also included are simple rule to help prepare validation protocols for assessing functionality and dependability of software.

EP05 Evaluation of Precision of Quantitative Measurement Procedures. 3rd ed., 2014. This document provides

guidance for evaluating the precision performance of quantitative measurement procedures. It is intended for manufacturers of quantitative measurement procedures and for laboratories that develop or modify such procedures.

EP06 Evaluation of the Linearity of Quantitative Measurement Procedures: A Statistical Approach. 1st ed.,

2003. This document provides guidance for characterizing the linearity of a method during a method evaluation; for checking linearity as part of routine quality assurance; and for determining and stating a manufacturer’s claim for linear range.

EP07 Interference Testing in Clinical Chemistry. 3rd ed., 2018. This guideline provides background information,

guidance, and experimental procedures for investigating, identifying, and characterizing the effects of interferents on clinical chemistry test results.

EP12 User Protocol for Evaluation of Qualitative Test Performance. 2nd ed., 2008. This document provides a

consistent approach for protocol design and data analysis when evaluating qualitative diagnostic tests. Guidance is provided for both precision and method-comparison studies.

EP14 Evaluation of Commutability of Processed Samples. 3rd ed., 2014. This document provides guidance for

evaluating the commutability of processed samples by determining if they behave differently than unprocessed patient samples when two quantitative measurement procedures are compared.

EP23™ Laboratory Quality Control Based on Risk Management. 1st ed., 2011. This document provides guidance

based on risk management for laboratories to develop quality control plans tailored to the particular combination of measuring system, laboratory setting, and clinical application of the test.

EP28 Defining, Establishing, and Verifying Reference Intervals in the Clinical Laboratory. 3rd ed., 2010. This

document contains guidelines for determining reference values and reference intervals for quantitative clinical laboratory tests.

H42 Enumeration of Immunologically Defined Cell Populations by Flow Cytometry. 2nd ed., 2007. This

document provides guidance for the immunophenotypic analysis of non-neoplastic lymphocytes by immunofluorescence-based flow cytometry; sample and instrument quality control; and precautions for acquisition of data from lymphocytes.

H43 Clinical Flow Cytometric Analysis of Neoplastic Hematolymphoid Cells. 2nd ed., 2007. This document

provides performance guidelines for the immunophenotypic analysis of neoplastic hematolymphoid cells using immunofluorescence-based flow cytometry; for sample and instrument quality control; and precautions for acquisition of data from neoplastic hematolymphoid cells.

H52 Red Blood Cell Diagnostic Testing Using Flow Cytometry, 2nd ed., 2014. This guideline addresses the

diagnostic red blood cell (RBC) assays performed as fluorescence-based assays on a flow cytometry platform; including testing procedures for fetomaternal hemorrhage detection, paroxysmal nocturnal hematuria screening, membrane defect anemia testing for hereditary spherocytosis, and nucleated RBC counting. Points of validation and quality control, and caveats of interpretation are also discussed.

M29 Protection of Laboratory Workers From Occupationally Acquired Infections, 4th ed., 2014. Based on

US regulations, this document provides guidance on the risk of transmission of infectious agents by aerosols, droplets, blood, and body substances in a laboratory setting; specific precautions for preventing the laboratory transmission of microbial infection from laboratory instruments and materials; and recommendations for the management of exposure to infectious agents.

CLSI documents are continually reviewed and revised through the CLSI consensus process; therefore, readers should refer to the most current editions.

H62

155


NBS05 Newborn Screening for Cystic Fibrosis. 1st ed., 2011. This document describes the use of newborn screening laboratory tests for detecting risk for cystic fibrosis from newborn dried blood spots (DBS) and addresses both the primary screening tests and the reflex tests performed on DBS.

QMS01 A Quality Management System Model for Laboratory Services. 5th ed., 2019. This guideline provides a

model for medical laboratories to organize the implementation and maintenance of an effective quality management system.

QMS03 Training and Competence Assessment. 4th ed., 2016. This guideline provides a structured approach for

developing effective laboratory personnel training and competence assessment programs. 3728

H62 Validation of Assays Performed by Flow Cytometry · associated with this use. This guideline is...

Documents

Transcript of H62 Validation of Assays Performed by Flow Cytometry · associated with this use. This guideline is...