CDRH OTC Review Practice, BPAC, 051103

1

CDRH Review Practice for Over-the-Counter in Vitro Diagnostic Tests

Center for Devices and Radiological Health (CDRH)

Office of in Vitro Diagnostic Device Evaluation and Safety

Arleen PinkosScientific Reviewer

2

Overview of Presentation

A. Background

B. Three Elements of OTC Review Accuracy and Reliability Labeling Risk vs. Benefit

C. Conclusion

3

OTC History

817 OTC products cleared for home use• Diabetic Products• Hormones• Drugs of Abuse• Urine Dipsticks• Fecal Occult Blood• Lipids, PT

4

OTC Infectious Disease Devices

None cleared or approved Devices reviewed for OTC- Self diagnosis:

• Group A Streptococcus, and Influenza A and B

In professional hands: • Sensitivity averaged 55-80%, dependent on specimen matrix • Specificity averaged 80-90%

False negatives – risks are life threatening False positives – risk of masking serious underlying disease

5

Indications for Use: What Drives the Review

Need to know who, what, when, and why • Special conditions for use that impact performance• Contraindications

Drives:• Protocol requirements• Questions raised• Data required

6

Three Review Elements

1. Is the device accurate and reliable in the hands of lay-users?

2. Is the device adequately labeled?

3. Do benefits outweigh the risks?

7

First Review Element

Is the device accurate and reliable in the hands of the user?

1. Can the test provide the right answer?2. Can the lay user get the right answer?3. Human Factors (HF)4. Risk Mitigation & Safeguards5. Stress Studies

8

Can the test provide the right answer?

Analytical and Clinical Performance

From initial professional-use submission Analytical and Clinical Sensitivity and

SpecificityAccurate (true) result• Quantitative -Bias, total error• Qualitative- Clinical Sensitivity and Specificity,

Predictive Values

9

Can the test provide theright answer?

Other Important Parameters

Cross-reactivity & Interference

Reproducibility

Reference intervals

Environmental factors, stability, reportable range, calibration, linearity, lot to lot variability, etc.

10

Can the lay user get the right answer?

Consumer Study GoalDemonstrate comparable performance when

used as intended

Representative Lay Users• Enrollment criteria, demographics

“Home” environment

Evaluate entire test system• Consumers perform all steps, unassisted• Labeling – Representative labeling, or according to marketing plan.

11

Consumer StudySample Considerations

Clinical or spiked – Challenging and span expected range

Prospectively collected if:• establish clinical sensitivity and specificity• demonstrate ability to collect the sample, or• sample is not stable

Biohazard

Properly characterized- WHO, CDC, reference method, or to another device

12

Consumer StudyProtocol Considerations

Comprehensive

Enough participants and samples

Correct Statistical Tools

Masking and randomizing- avoid bias

Written prior to study

13

Human Factors (HF)

A discipline concerned with how people interact with the device

Why are HF so important?

Users- varying abilities and behaviorsImpact on safety and effectiveness

14

Human Factors

“Errors” occur when users:

Fail to perform necessary actionsAdd actionsChoose the wrong actionPerform actions poorly

15

Human Factors Who is responsible for “user” errors?

“The instructions clearly state that results are to be read from right to left.”

“How was I to know they’d read it left to right?”

16

ACME

Toenail Cutter

All user errors cannot be preventedbut some can be avoided

17

Risk Mitigation & Safeguards What do we expect regarding errors?

Effective procedures to address HF and other potential problems:

1. Hazard Analysis: Identify potential errors and estimate their risk

2. Mitigate Risks: Eliminate or reduce risks to acceptable level

3. Verify effectiveness of safeguards

18

Risk Mitigation & Safeguards Examples

Built-in features to detect:• Short sample • Wrong sample• Reagent integrity

Internal QC- often limited scope

Lock-out mechanisms?

Labeling- techniques, but when is it adequate?

Goal- alert before results are compromised

19

Risk Mitigation & SafeguardsQuality Control

Is External QC• Available and practical?• Adequate?• Needed?

Will consumers do it ?!!

20

Stress Studies Demonstrating Effective Mitigation

Need to verify effectiveness of safeguards

What are stress studies?

• Challenges to the system in order to characterize performance under various conditions of use

21

Stress Studies How are they performed?

Artificially simulate potential errorsConcentrate on areas prone to errors or

failures• Technique• Follow directions• Deviations in environmental conditions

Observe results

22

Stress Study Examples

Instructions Type of Study

Procedure- store kit between 60-80 °F

Store kit as directed. Store the kit with intermittent freezing,

50 and 90 °F then run samples.

Procedure- add 3 drops of sample.

Perform the test by adding 2, 3, 4, and 5 drops of sample.

23

Second Review ElementIs the device adequately labeled?

Are the step-by-step instructions simple, concise, and easy to follow?• 7th grade reading level?

Do consumers know who should use the device and when?

Clear instructions for interpretation and follow-up? • Limitations?• Go to a doctor? • What if it didn’t work? • Repeat test?

24

LabelingHow do you know its adequate?

Verified-Consumer StudyValue is Limited“Trial effect”Will they read it?

QuestionnaireCompleted by consumers after testingMust be challenging:

Who, what, where, when, how & why?

25

Third Review ElementRisk vs. Benefit Consideration

Lay users are unique:

Unregulated environment May lack medical and technical

training: • evaluate other medical information • collect and transport samples• read and follow directions

May or may not carry out follow-up actions

Highly variable

26

Risk vs. Benefit

Risk

1. What is the impact of a false-positive or false-negative result on the user and

public health e.g., no follow-up, adverse medical conditions,

unnecessary treatment?

2. What are the medical risks to the user and public health if there is a delay in obtaining a professional examination and treatment, or if the device gives an equivocal result?

27

Risk vs. Benefit

Benefit

1. What is the clinical benefit to the patient and public health in terms of screening, diagnosing, or monitoring a particular disease or condition, or risk factor?

2. What are the medical benefits to the patient and public health in having the test available for home-use as opposed to having the test performed only by health care professionals?

28

CONCLUSIONThe Final Decision

Is the device accurate and reliable in the hands of the lay user?

Does the Labeling adequately convey all information necessary to use the device safely and effectively?

Do benefits outweigh risks?

If yes, generally found suitable for home-use