2016 Metrics on Human Research Protection Program Performance Documents/2016 All... ·...

2016 Metrics on Human

Research Protection

Program Performance

Updated April 24, 2017

About the Metrics____________________________ Improving the quality of human research protection programs (HRPP) is a top priority of AAHRPP. Effective and efficient systems of oversight with organizations provide better protections for research participants and produce higher quality research. And collectively, they raise the bar globally to ensure research participants are safe and respected. AAHRPP is pleased to present the 2016 metrics for HRPP performance.

The metrics are collected from annual reports and new applications from our current clients. From data supplied by our client organizations in 2016, AAHRPP has compiled an information database to help research organizations, researchers, sponsors, government agencies, and participants identify and support high-performing practices for HRPPs. The data range from types of research and conformance with regulations and guidance to financial and personnel resources and IRB review times. For the 2016 data, the metrics are represented as the median, except where indicated as the mean.

Table of Contents

General Description of Organizations Slide 1. All Accredited Organizations in 2016 4

Slide 2. Responding Organizations in 2016 4

General Description of the Research Conducted or Overseen by Organizations Figure 1. Where Organizations Conduct Research 5 Figure 2. Type of Research Organizations Conduct or Review 5

Selected Types of Research Conducted or Overseen by Organizations Figure 3: Selected Types of Research that Organizations Conduct or Review 6

Figure 4: Organizations that Conduct or Review Research Involving Vulnerable Populations 6

Sponsors and Regulators of Research Figure 5: Sponsors of Research 7

Figure 6: Federal Sponsors of Research 7

Figure 7: Regulations and Guidance Followed by Organizations 8

Regulatory Oversight of Research Figure 8: Checking the Boxes on the Federalwide Assurance 9

Figure 9: Comparison of AAHRPP Organizations and Those Registered with OHRP on

Checking the Boxes on the Federalwide Assurance 9

Reliance on the IRB Figure 10: Use of IRBs 10

Figure 11: Use of External IRBs 10

Figure 12: Number of IRBs Per Organizations 11

Compensation of IRB Members Figure 13: Compensation of IRB Members by Organizations 12

Figure 14: Financial Compensation of IRB Members 12

Figure 15: Percentage of Organizations that Compensate IRB Chairs and Vice Chairs Over Time 13

Characteristics of IRBs Figure 16: Median Number of Active Protocols Organizations Oversee 14

Figure 17: Mean Number of Active Protocols Organizations Oversee 15

Figure 18: Median Number of Active Protocols Overseen by an IRB Based on the Number of

IRBs Per Organization 16 Figure 19: Mean Number of Active Protocols Overseen by an IRB Based on the Number of IRBs Per Organizations 17

IRB Review Times Figure 20: IRB Review Times by Type of Review 18

Figure 21: Four-Year Trends of Median IRB Review Times from Submission to Approval 19

Use of Technology Figure 22: Technology Use by Organizations with an IRB 20

Resources for the IRB Table 1: IRB Staffing and Funding Levels 21

Figure 23: Four-Year Trends in Median IRB Budgets 21

Audits of the HRPP Conducted by Organizations Table 2: Number of Internal Audits Organizations Conducted within the Past Year 22

Figure 21: Four Trends in Mean Number of Audits Organizations Conducted 22

Complaints Reported to the IRB Figure 22: Five-Year Trends in Mean Numbers of Complaints Reported 23

Non-Compliance Reported to the IRB Table 3: Number of Cases of Non-Compliance Reported to the IRB in the Past Year 24

Figure 26: Five-Year Trends in Mean Number of Reported Cases of Non-Compliance 24

AAHRPP Survey Results Figure 27: Survey Results Regarding AAHRPP Accreditation 25

General Description of Organizations

Metrics on HRPP Performance 4

All Accredited Organizations 2016

05

101520253035404550

Perc

ent

of O

rgan

izat

ions

Slide 1: 43% of all accredited organizations are academic institutions, 38% are hospitals, 9% are independent IRBs, 4% are VA facilities, 3% are dedicated research sites or research institutes, 2% are government organizations, 1% are contract research organizations, and 0.5% are sponsors.

Responding Organizations 2016

05

101520253035404550

Perc

ent

of O

rgan

izat

ions

Slide 2: 43% of organizations that responded in 2016 were academic institutions, 39% were hospitals, 10% were independent IRBs, 4% were dedicated research sites or research institutes, 2% were VA facilities, and 2% were government organizations.

General Description of the Research Conducted or

Overseen by Organizations


Figure 1: Where Organizations Conduct Research

0 20 40 60 80 100

In home state/province/region pluscountries other than home country

In home state/province/region only

In home state/province/region plusother state/province/regions within

home country

In home state/province/region, otherstate/province/regions, and othercountries outside of home country

Percent

Figure 1: 50% of responding organizations conduct research home state/province/region, other states/provinces/regions within their home country, and countries other than their home country, 24% conduct research in their home state/province/region and other states/provinces/regions within their home country, 23% of responding organizations conduct research in their home state/province/region only, and 4% conduct research in their home state/province/region and countries other than their home countries.

Figure 2: Type of Research Organizations Conduct or Review

0 20 40 60 80 100

Other

Social Science/Behavioral

Biomedical

Percent

Figure 2: 100% of responding organizations conduct biomedical research, 91% conduct social/behavioral research, and 30% conduct research falling into other categories.

Selected Types of Research Conducted or Overseen

by Organizations


Figure 3: Selected Types of Research that Organizations Conduct or Review

0 20 40 60 80 100

Planned Emergency Research withoutConsent

Investigational Devices

Investigational Drugs

Percent

Figure 3: 90% of organizations conduct research involving investigational drugs, 84% conduct research involving investigational devices, and 16% of organizations conduct planned emergency research without consent.

Figure 4: Organizations that Conduct or Review Research Involving Vulnerable Populations

0 20 40 60 80 100

Other

Prisoners

Pregnant Women

Students

Adults with Diminished Capacity

Employees

Children

Percent

Figure 4: 98% of all organizations conduct research with children, 95% conduct research with employees, 95% conduct research with adults with diminished capacity, 94% conduct research with students, 83% conduct research with pregnant women, 50% conduct research with prisoners, and 15% conduct research with other vulnerable populations.

Sponsors and Regulators of Research


Figure 5: Sponsors of Research

0

5

10

15

20

25

30

35

40

45

50

Internal Industry Federal External

Perc

ent

Figure 5: For all organizations, a median of 43% of research is internally sponsored, a median of 20% of research is industry sponsored, a median of 19% of research portfolios is federally sponsored, and a median of 7% of research portfolios is externally sponsored.

Figure 6: Federal Sponsors of Research

0

5

10

15

20

25

30

35

40

US Government Only Both US and Non-USGovernment

Non-US GovernmentOnly

Perc

ent

Figure 6: For all organizations receiving federal funds, a median of 25% of research is sponsored by the US Government only, a median of 22% of research portfolios is sponsored by both US and Non-US governments, and a median of 12% of research is sponsored by only Non-US Governments.

Sponsors and Regulators of Research


Figure 7: Regulations and Guidance Followed by Organizations

0 20 40 60 80 100

Deparment of Energy

Enviornmental Protection Agency

Department of Veterans Affairs

Department of Justice

Country-specific Law

Department of Education

ICH-GCP when requested by the sponsor

ICH-Good Clinical Practice

Department of Defense

State

Department of Health & Human Services

Food & Drug Administration

Percent

Figure 7: 83% of all organizations follow FDA regulations, 80% follow Department of Health and Human Services regulations, 77% follow regulations or guidelines of their state, 57% follow Department of Defense requirements, 38% follow the ICH- Good Clinical Practice guideline, 29% follow the ICH- Good Clinical Practice guideline when requested by the sponsor, 28% follow Department of Education requirements, 22% follow Country-specific regulations or guidelines, 12% follow Department of Justice requirements, 13% follow Department of Veterans Affairs requirements, 11% follow Environmental Protection Agency requirements, and 10% follow Department of Energy requirements.


Regulatory Oversight of Research

Figure 8: Checking the Boxes on the Federalwide Assurance

7%

22%

71%

Checked Box ApplyingSubpart A

Checked Boxes ApplyingSubparts A, B, C, D

Did Not Check BoxesApplying Subparts A, B, C,or D

Figure 8: 71% of all organizations did not check boxes applying Subparts A, B, C, or D, 22% of all organizations check boxes applying all Subparts (A, B, C, D), and 7% checked the box only applying Subpart A.

29%

34%

37%

Checked BoxApplyingSubpart A

Checked BoxApplyingSubparts A,B, C, D

Did NotCheck BoxesApplying toSubparts A,B, C, or D

Figure 9: Comparison of AAHRPP Organizations and Those Registered with OHRP on Checking the Boxes on the

Federalwide Assurance

From Office for Human Research Protections (2015)

Accredited Organizations

7%

22%

71%

Checked BoxApplyingSubpart A

Checked BoxesApplyingSubparts A, B,C, D

Did Not CheckBoxesApplyingSubparts A, B,C, or D

Figure 9: Compared to percentages provided by OHRP, more organizations accredited by AAHRPP or applying for AAHRPP accreditation unchecked the box.

Reliance on the IRB


Figure 10. Use of IRBs

Does not have own IRB, 4%

Have own IRB, 96%

Figure 10: 96% of all organizations have their own IRB; 4% do not have their own IRB.

Figure 11. Use of External IRBs

Rely on External IRB for no more than 10%, 71%

Rely on External IRB for more

than 10%, 26%

Rely on External IRB for 100%,

3%

Figure 11: 71% of all organizations use external IRBs for a maximum of 10% of total protocols, 26% of organizations rely on external IRBs for more than 10% of total protocols, and 3% of all organizations rely on external IRBs for 100% of total protocols.

Reliance on the IRB


Figure 12: Number of IRBs per Organization

1 IRB, 44%

2 IRBs, 16%

3 IRBs, 18%

4 IRBs, 9%

5 or more IRBs, 13%

Figure 12: 44% of all organizations have one IRB, 16% have two IRBs, 18% have three IRBs, 9% have four IRBs, and 13% have five or more IRBS.

Compensation of IRB Members


Figure 13: IRB Member Compensation

0

10

20

30

40

50

60

70

80

90

100

Compensate IRBChairs

Compensate Non-AffiliatedMembers

Compensate IRBVice Chairs

CompensateAffiliatedMembers

Perc

ent

of O

rgan

izat

ions

Figure 13: 83% of all organizations compensate IRB chairs, 61% compensate non-affiliated members, 56% compensate IRB vice chairs, and 38% compensate affiliated members.

Figure 14: Financial Compensation of IRB Members

75

80

85

90

95

100

Compensate Non-AffiliatedMembers

Compensate IRBVice Chairs

Compensate IRBChairs

CompensateAffiliatedMembers

Perc

ent

of O

rgan

izat

ions

Figure 14: Of organizations that compensate IRB members, 94% compensate non-affiliated members financially, 93% compensate IRB vice chairs financially, 89% compensate IRB chairs financially, and 85% compensate affiliated members financially.

Compensation of IRB Members


Figure 15: Percentage of Organizations that Compensate IRB Chairs and Vice Chairs Over Time

0

10

20

30

40

50

60

70

80

90

100

2012 2013 2014 2015 2016

Pe

rce

nt

of

Org

an

iza

tio

ns

Chairs

Vice Chairs

Figure 15: The percentage of organizations that compensate IRB chairs and vice-chairs has remained relatively consistent since 2012.

Characteristics of IRBs


Figure 16: Median Number of Active Protocols Organizations Oversee

41

366

254

780

0

200

400

600

800

1000

1200

ExemptProtocols

Reviewed bythe Expedited

Procedure

Reviewed bythe Convened

IRB

Total

Nu

mb

er

of

Act

ive

Pro

toco

ls

95% CI Upper Bound

95% CI Lower Bound

Median

Figure 16: A median of 41 protocols were deemed exempt by all organizations*, a median of 366 protocols were reviewed by the expedited procedure, a median of 254 protocols were reviewed by the convened IRB, and a median of 780 total protocols were overseen by all organizations. *Exempt determinations made within 12 months of an organization’s submission to AAHRPP.



Figure 17: Mean Number of Active Protocols Organizations Oversee

143

787

526

1401

0

200

400

600

800

1000

1200

1400

1600

1800

ExemptProtocols

Reviewed bythe Expedited

Procedure

Reviewed bythe Convened

IRB

Total

Nu

mb

er

of

Act

ive

Pro

toco

ls

95% CI Upper Bound

95% CI Lower Bound

Mean

Figure 17: A mean of 142 protocols were deemed exempt by all organizations*, a mean of 787 protocols were reviewed by the expedited procedure, a mean of 526 protocols were reviewed by the convened IRB, and a mean of 1401 total protocols were overseen by all organizations. *Exempt determinations made within 12 months of an organization’s submission to AAHRPP.



Figure 18: Median Number of Active Protocols Overseen by an IRB Based on the Number of IRBs Per Organization

0

500

1000

1500

2000

2500

3000

3500

4000

All One IRB Two IRBs Three IRBs Four IRBs Five orMore IRBs

Nu

mb

er

of

Act

ive

Pro

toco

ls

Figure 18: A median of 509 protocols were overseen by all organizations, a median of 254 protocols were overseen by organizations with one IRB, a median of 438 protocols were overseen by organizations with two IRBs, a median of 1352 protocols were overseen by organizations with three IRBs, a median of 1179 protocols were overseen by organizations with four IRBs, and a median of 3708 were overseen by organizations with five or more IRBs.



Figure 19: Mean Number of Active Protocols Overseen by an IRB Based on the Number of IRBs Per Organization

0

500

1000

1500

2000

2500

3000

3500

4000

All One IRB Two IRBs Three IRBs Four IRBs Five orMore IRBs

Nu

mb

er

of

Act

ive

Pro

toco

ls

Figure 19: A mean of 1054 protocols were overseen by all organizations, a mean of 384 protocols were overseen by organizations with one IRB, a mean of 648 protocols were overseen two IRBs, a mean of 1468 protocols were overseen by organizations with three IRBs, a mean of 1131 protocols were overseen by organizations with four IRBs, and a mean of 3593 protocols were overseen by organizations with five or more IRBs.

IRB Review Times


Figure 20: IRB Review Times by Type of Review

16

37

8

19

10

0

5

10

15

20

25

30

35

40

45

Time fromSubmission toReview by theConvened IRB

Time fromSubmission to

Approval by theConvened IRB

Time fromSubmission toReview by the

ExpeditedProcedure


Approval by theExpeditedProcedure


ExemptDetermination

Ca

len

da

r D

ay

s

Figure 20: The time from submission to review by the convened IRB is a median of 16 calendar days, the time from submission to approval by the convened IRB is a median of 37 calendar days, the time from submission to review by the expedited procedure is a median of 8 calendar days, the time from submission to approval by the expedited procedure is a median of 19 calendar days, and the time from submission to exempt determination is a median of 10 calendar days.

IRB Review Times


Figure 21: Four-Year Trends of Median IRB Review Times from Submission to Approval

0

10

20

30

40

50

60

2013 2014 2015 2016

Ca

len

da

r D

ay

s Reviewed by the Convened IRB

Reviewed by the Expedited Procedure

Exempt Determinations

Figure 21: Median review times have remained consistent since 2014.

Use of Technology


Figure 22: Technology Use by Organizations with an IRB

0 20 40 60 80 100

Use of an Online System for IRB Review

Use of an Online IRB Application

Use of an Electronic System for theDistribution of Materials

Use of a Database to Track IRB Protocols

Percent

Figure 22: 94% of all organizations use a database to track IRB protocols, 91% use an electronic system for the distribution of materials, 74% use an online IRB application, and 74% use an online system for IRB review functions.

Resources for the IRB


Table 1: IRB Staffing and Funding Levels

Protocol Category

MedianNumber of Staff

MedianNumber of Protocols

Median Protocols per FTE

Median Dollars Budgeted for IRB

All 7 865 123.6 $485,436

1-100 2 37.5 18.75 $100,800

101-500 4 271.5 67.9 $352,288

501-1000 5.5 729.5 132.6 $457,500

1001-2000 9 1379.5 156.8 $596,013

2001-4000 18 2839 157.7 $1,063,264

4000+ 27 6344 235 $2,215,500

Figure 23: Four-Year Trends in Median IRB Budgets

$300,000

$400,000

$500,000

$600,000

$700,000

$800,000

2013 2014 2015 2016

Ave

rage

Dol

lars

Figure 23: Median IRB budgets have slightly increased since 2015. .

Audits of the HRPP Conducted by Organizations


Table 2. Number of Internal Audits Organizations Conducted within the Past Year

For-Cause Audits of

Researchers

Random Audits of Researchers

For-Cause Audits of IRBs

Random Audits of IRBs

Mean 6 44 3 22

Median 1 15 0 4

Min 0 0 0 0

Max 149 860 162 506

Figure 24: Five-Year Trends in Mean Number of Audits Organizations Conducted

0

5

10

15

20

25

30

35

40

45

50

For-Cause Auditsof Researchers

Random Audits ofResearchers

For-Cause Auditsof IRBs

Random Audits ofIRBs

Mea

n N

umbe

r of

Aud

its

2012 2013 2014 2015 2016

Figure 24: For-cause audits of researchers and random audits of researchers have reached an all-time high in 2016. Random audits of IRBs and for-cause audits of IRBs have remained consistent since 2014.

Complaints Reported to the IRB


Figure 25: Five-Year Trends in Mean Numbers of Complaints Reported

0

2

4

6

8

10

12

Complaints

Nu

mb

er

of

Re

po

rts

2012 2013 2014 2015 2016

Figure 25: The number of complaints from research participants reached its highest total since 2012. .

Non-Compliance Reported to the IRB

Metrics on HRPP Performance 24 Metrics on HRPP Performance 24

Table 3. Number of Cases of Non-Compliance Investigated in the Past Year

Protocol Category

Median Number of Allegations of Non-

Compliance

Median Number of Determinations of Serious Non-

Compliance

Median Number of Determinationsof Continuing Non-Compliance

All 3 1 0

1-500 1 0 0

501-1,000 1 1 1

1,001-2,000 4 2.5 0.5

2,001-4,000 11 4 1

>4,000 16 4 2.5

Figure 26: Five-Year Trends in Mean Number of Reported Cases of Non-Compliance

0

5

10

15

20

25

30

35

40

Non-Compliance Allegations Serious Non-Compliance Determinations Continuing Non-Compliance Determinations

Num

ber

of R

epor

ts

2012 2013 2014 2015 2016

Figure 26: Allegations of non-compliance, determinations of serious non-compliance, and determinations of continuing non-compliance have slightly increased since 2015.

AAHRPP Survey Results


Figure 27: Survey Results Regarding AAHRPP Accreditation

0

10

20

30

40

50

60

70

80

90

100

Improves Human ParticipantProtection

Imrpoves Quality of ResearchReview

Improves RegulatoryCompliance

Provides Value

Strongly Agreed Agreed Neutral Opinion Disagreed

Figure 27: 58% of responding organizations strongly agreed that AAHRPP Accreditation improves human participant protection, 35% agreed, and 7% had a neutral opinion. 63% of responding organizations strongly agreed that AAHRPP Accreditation improves the quality of research review, 30% agreed, and 7% had a neutral opinion. 60% of responding organizations strongly agreed that AAHRPP Accreditation improves regulatory compliance, 33% agreed, 6% had a neutral opinion, and less than 1% disagreed. 69% of responding organizations strongly agreed that AAHRPP Accreditation provides value to their organization, 25% agreed, 5% had a neutral opinion, and less than 1% disagreed.

2301 M Street, NW (202) 783-1112 phone

Suite 500 (202) 783-1113 fax

Washington, DC 20037 www.aahrpp.org

http://www.aahrpp.org/

2016 Metrics on Human Research Protection Program Performance Documents/2016 All... ·...

Documents

Transcript of 2016 Metrics on Human Research Protection Program Performance Documents/2016 All... ·...