Validation of A Proposed Testing

Strategy using FDA-

approved Rapid Tests

Eugene Martin, Ph.D.

APHA 2008 Annual MeetingSan Diego, CA

October 25-29, 2008

Rapid-rapid Verification Programs

• Factors to consider:• How is your program organized?

• Is it centrally organized or groups of independent labs?• How much confidence do you have in each labs

ability to handle multiple assays?• How much experience do your laboratories have in

figuring out ‘discordant results’?• What will happen if there is a problem?• How prevalent is HIV where you are testing? As

prevalence DECREASES…False Positives results REMAIN constant while true positives decrease. Are you prepared to deal with discordant results?

NJ Rapid HIV Testing• One of the largest, most centralized rapid

HIV testing programs in the country:• County health departments • Sexually transmitted disease clinics, • Family planning programs, • Federally qualified healthcare centers,• TB clinics,• Prisons,• Hospital-based programs – 13 ERs (8 counties),• Prenatal clinics, and • Outreach through mobile vans.

New Jersey ‘sRapid HIV Sites

AIDS Coalition of Southern New JerseyAtlantic City Health DepartmentBergen County Health DepartmentBurlington County Health DepartmentCamden AHECCamden County Health DepartmentCheck-MateEast Orange Health DepartmentEric B. Chandler Health CenterFamCareHenry J. Austin Health CenterHope HouseHorizon Health CenterHunterdon County Health DepartmentHyacinth FoundationMartin Luther King OutreachMorristown Memorial HospitalNewark Community Health CenterNJCRIOcean County Health DepartmentPaterson Health DepartmentPlainfield Community Health CenterProceedRobert Wood Johnson Medical SchoolTrinitas Hospital

6/5/2006

UMDNJ-RWJMS/ NJ DHSS AIDS PREVENTION GRANTEES

Primary Satellite fixed mobile

Pale colors indicate pending sites

AIDS Coalition of Southern New JerseyAtlantic City Health DepartmentBergen County Health DepartmentBurlington County Health DepartmentCamden AHECCamden County Health DepartmentCheck-MateEast Orange Health DepartmentEric B. Chandler Health CenterFamCareHenry J. Austin Health CenterHope HouseHorizon Health CenterHunterdon County Health DepartmentHyacinth FoundationMartin Luther King OutreachMorristown Memorial HospitalNewark Community Health CenterNJCRIOcean County Health DepartmentPaterson Health DepartmentPlainfield Community Health CenterProceedRobert Wood Johnson Medical SchoolTrinitas Hospital

6/5/2006

UMDNJ-RWJMS/ NJ DHSS AIDS PREVENTION GRANTEES

Primary Satellite fixed mobile

Pale colors indicate pending sites

UMDNJ-RWJMS/ NJ DHSS AIDS PREVENTION GRANTEES

Primary Satellite fixed mobile

Pale colors indicate pending sites

Rapid HIV Testing in NJTesting Began 2003

• 23 primary sites• 32 satellite licenses• Western Blot confirmation

at state lab (PHEL) in Trenton

Over 70 CTS sites, including:

• Hospitals/EDs• FQHCs• CBOs• Health departments• Mobile vans• Prisons

Why a Centralized Program?

STEPS• Specialized skills are centralized• Testing and processes are

organized • Expenses are optimized• Problems are identified more

quickly• Solutions are distributed to all:

• http://www.njhiv1.org

Why Move to Rapid Confirmation?

• Problem• Preliminary Positive

clients fail to return for results (25.2%)

• NAP succeeds ONLY 20% of the time in locating these clients

• Solution• Confirmatory testing

on-site, same day

326

244

82

47

11

0

50

100

150

200

250

300

350

Number

Disposition of Confirmed HIV + Clients

Confirmed HIV + Result retuned to client Did Not Receive ResultsReferred to NAP Found by NAP

Validation of a Testing Algorithm

Can a second rapid HIV test confirm preliminary positives as effectively as a Western Blot?

“Validation is the process of demonstrating

that an analytical procedure is suitable for

its intended use” – CBER

• The use of other rapid tests to confirm a rapid HIV test is not new or novel - WHO recommends this approach for countries in which the prevalence of HIV exceeds 10% for a number of years

• What is new and novel is using this approach in sites with 2% or lower prevalence

What does any rapid test algorithm need to do?

NEED TO DO• Demonstrate Sensitivity• Identify True Positives• Not Identify False Positives • Demonstrate

Reproducibility• Demonstrate Robustness -

remain unaffected by small, but deliberate, variations in method parameters

DATA• Sensitivity & Specificity:

• Manufacturer’s Claims• CDC Post Marketing Survey

• Follow-up Investigations:• Existing confirmations• Alternative testing

Questions for a Rapid Testing Algorithm (RTA):

1. Are there false negative screening tests? i.e Screening test says NEG, but the client is infected.

2. Can a positive client result be confirmed by a second rapid test?

3. Can false positive screening tests be detected by running a second rapid test… or a third? What is the advantage? Disadvantage?

4. How can inconclusive “second-round” test results (eg, WB vs a second rapid test) be resolved?

5. What is the impact on the linkage to care?

NJ - A Two-Test Algorithm

HIV-1/2 Rapid HIV Test (Blood) STAT-Pak

Or

HIV-1/2 Rapid HIV Test (Oral) Oraquick

Unique Characteristics

• Area: New Jersey: 7,836 sq. mi> Los Angeles: 469.1 sq mi> San Francisco: 47 sq. miles

• Population: Greater LA (2007) ~17.78 million > NJ 8.69 million > San Francisco ~ 4.18 million

• Scale: Drive End to End in NJ 3 hrs. (WE 1 ½ hours)

• A mixture of urban/suburban and rural communities • North – urban• South – rural

• Many different venues perform rapid testing

Validation of the NJ algorithm

• Three Data Sets:• 2004• January, 2006 – October, 2007• 2008

2004 Ability of a Rapid to Confirm a

Rapid

• ALL confirmatory specimens sent to NJ PHEL during 8 month period:

1. Re-ran the PHEL specimen using Oraquick again

2. Re-ran the specimen with other rapid tests

3. Confirmed negative result by repeat Western blot and Viral Load

Rapid confirmation trial

Negative WB Pos Discordant

• __ ____ _____ _________

• ______ ________ _______ __ _____ ___ ___ ____________ _______• ______ ___ ________• ______ ___ ________

• 15,923 OraQuick tests statewide

• 363 prelim positive samples to state lab for confirmatory testing• 355 Western Blot

positive• 8 Western Blot

negative

July 1, 2004 through April 19, 2005

2004 NJ Data

363 Specimens at PHEL

8 Negative Western blot

355 Positive Western blot

Orasure Oraquick 8 POS 355 POS

Trinity UniGold 8 NEG 355 POS

Biorad Multispot 7 NEG 1 POS 354 POS 1 QNS

MedMira Reveal 8 NEG 340 POS 15 sample interference

Follow-up > 2mos

6/6 neg Western blot6/6 neg viral load

Rapid confirmation trial

2004-5 Rapid Testing Evaluation• All testing in 2004 involved fingerstick

rapid Oraquick HIV tests• All 8 Western Blot negative clients:

• Negative on follow-up at least 4 weeks later, by both antibody and nucleic acid testing

• 4 of 7 tested reacted with non-viral components of OraQuick device

• ALL were true Oraquick false positives

• A CLIA-waived rapid test matched Western Blot confirmatory results in 100% of the HIV + cases.

• Every false positive was identified by a proposed rapid confirmation algorithm between 2004-2006!

• Potential consequences using rapid-rapid confirmation:• Eliminate the non-returners• Effective sensitivity would approach 99-100%• Counseling, contact elicitation and referral for

treatment could be Done Immediately• In NJ, at least 200 additional HIV +

individuals would definitively know their status!!

2004 TAKE HOME MESSAGES:

Ability of a Rapid HIV Test to Confirm a Rapid HIV Result

January 2006- October 2007 NJ Data Set

Jan 2006 – Oct 2007 dataBACKGROUND• Oral HIV testing had become the predominant means of rapid HIV

testing in New Jersey.• The rate of discordant results had increased with oral testing

METHODS• We used retained specimens from follow-up testing of clients that

were Rapid Test (+), but Western Blot (-)

• Testing done on serum• Confirmed discordants and indeterminates (if they had follow-up).• DID NOT confirm true positives from this data set.• Samples were not from the same time as the screening

OraQuick.

• Used CLIA-waived tests ONLY:• Repeated OraQuick on blood• Trinity Uni-Gold• Clearview StatPak

Follow the data!

Total Rapid HIV Testing

Fingerstick Oral Testing

NJ Rapid HIV Testing 20056• Oral Testing Introduced More

False Positive Confirmations

Follow-up Information

• All follow-ups on negative Western Blot specimens were also NAAT negative (i.e.True OraQuick false positive). • All were "second rapid" negative.

• Oral Discordants• OraQuick followed by Blood OraQuick:

• 3 tested at CTS site on False Positives:• 2 blood negative; 1 blood positive • 56 tested on follow-up blood specimen• all were blood negative

• SUMMARY: 58 specimens were truly negative; 1 specimen was positive

• Blood Discordants• 11 tested on follow-up blood specimen

• 7 negative • 4 repeat positive

Observations

• Indeterminate Western Blot:• 12 total:

• 4 no follow-up; 3 QNS• 3 NAAT negative were "second rapid" negative• 2 NAAT positive were "second rapid" positive

2008 Data

• Randomly sampled serum specimens sent to NJ PHEL some for confirmatory testing; some for standard testing. We didn’t know their identity

• Ran: • Oraquick • UniGold• StatPak

• Discordant – 2 of the 3 rapids agreed – one did not • UniGold twice• Oraquick three times

All Pos

ALL Neg UniGold Discordant

Oraquick Discordant

TOTAL

149

26 2 3 180

Conclusive 175 97.2%

Inconclusive 5 2.8%

Potential Issues:

ID OraquickTrinity Unigold StatPak EIA (s/co) West. Blot

OQ80031 + - + 0.824/0.263REACTIVE ALL BANDS EXCEPT:66,17=0

Falsely negative 2nd rapid

Frequency 1:180

Falsely positive 2nd rapid• None using Trinity Unigold as the 2nd rapid• Three using StatPak as the 2nd rapid

ID OraquickTrinity Unigold StatPak EIA (s/co) West. Blot

OQ800131 + - + 0.046/0.270 NO BANDSOQ800169 + - + 0.091/0.272 NO BANDSOQ80032 + NS + 0.063/0.276 NO BANDS

• A second, different rapid HIV test can confirm a preliminary rapid resultas reliably as a Western Blot.

• 98% of time the conclusion will be correct

• 25% of individuals who would never have received their final result will now!!

• New Jersey will implement rapid-rapid verification and immediate linkage to care

• Following meetings with Department of Health it was decided to implement rapid-rapid verification statewide

Conclusions:

Predictive Value as a Function of Return for Results

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0% 5% 10% 15%

HIV Prevalence

Po

siti

ve P

red

icti

ve V

alu

e

100%

90%

80%

70%

60%

NJ - 70% get results

Standard PPV

New Jersy HIV Prevalence at CTS centers

Implementation

• PLAN:• 3 pilot sites have been identified to begin the ‘roll-out’ • 1 site is up, trained and running. The other 2 - within the month• Policies, Procedures, Counseling Messages and Forms are

completed for the entire system

• EXPECTATIONS: • It will not eliminate Western blot confirmation, BUT it will

provide the basis for immediate linkage to care!• Less than 1 in 100 will be later removed from care because of a

failure to confirm

• UNKNOWNS: What will be the real world performance of a rapid test in a confirmatory setting?

• Does reducing the delay really improve the linkage to care?• Does post-testing counseling impact positively on prevention

messages?

Thanks To:

RWJMS• Evan Cadoff, MD • Eugene Martin,

Ph.D.• Gratian Salaru, MD

• Sharon Holswade, MBA

• Franchesca Jackson, BS

• Nisha Intwala, MT• Claudia Carron, RN• Lisa May• Karen Williams

NJDHSS/DHAS• Sindy Paul, MD, MPH*• Linda Berezny, RN• Maureen Wolski, BS• Aye Maung Maung

NJDHSS/PHEL• Kenneth Earley• Kanjana Garcia• Bruce Wolf, Ph.D.

Site coordinators and counselors throughout New Jersey

THE END