Office of the DirectorOffice of the Director

National Instituteon Alcohol Abuseand Alcoholism

National Instituteon Alcohol Abuseand Alcoholism

National Instituteof Arthritis andMusculoskeletal

and Skin Diseases

National Instituteof Arthritis andMusculoskeletal

and Skin Diseases

National CancerInstitute

National CancerInstitute

National Instituteof Diabetes andDigestive and

Kidney Diseases

National Instituteof Diabetes andDigestive and

Kidney Diseases

National Instituteof Dental andCraniofacial

Research

National Instituteof Dental andCraniofacial

Research

National Instituteon Drug Abuse

National Instituteon Drug Abuse

National Instituteof Environmental Health Sciences

National Instituteof Environmental Health Sciences

National Instituteon Aging

National Instituteon Aging

National Instituteof Child Health

and HumanDevelopment

National Instituteof Child Health

and HumanDevelopment

National Institute onDeafness and Other

CommunicationDisorders

National Institute onDeafness and Other

CommunicationDisorders

National EyeInstitute

National EyeInstitute

National HumanGenome Research

Institute

National HumanGenome Research

Institute

National Heart,Lung, and Blood

Institute

National Heart,Lung, and Blood

Institute

National Instituteof Mental Health

National Instituteof Mental Health

National Instituteof NeurologicalDisorders and

Stroke

National Instituteof NeurologicalDisorders and

Stroke

National Instituteof General

Medical Sciences

National Instituteof General

Medical Sciences

National Instituteof Nursing Research

National Instituteof Nursing Research

National Libraryof Medicine

National Libraryof Medicine

Center for InformationTechnology

Center for InformationTechnology

Center for Scientific Review

Center for Scientific Review

National Centerfor Complementary

and AlternativeMedicine

National Centerfor Complementary

and AlternativeMedicine

National Instituteof Allergy and

Infectious Diseases

National Instituteof Allergy and

Infectious Diseases

FogartyInternational

Center

FogartyInternational

Center

National Centerfor ResearchResources

National Centerfor ResearchResources

National Institutes of HealthNational Institutes of Health

Clinical Center

Clinical Center

National Center on Minority Health andHealth Disparities

National Center on Minority Health andHealth Disparities

National Institute of Biomedical Imagingand Bioengineering

National Institute of Biomedical Imagingand Bioengineering

A Typical A Typical Institute/CenterInstitute/Center

Office of the ICOffice of the ICDirectorDirector

NationalNationalAdvisoryAdvisoryCouncilCouncil

Board ofBoard ofScientificScientific

CounselorsCounselors

ExtramuralExtramural

ScientificScientificProgramsPrograms

GrantsGrants ContractsContracts

IntramuralIntramural

LaboratoryLaboratoryStudiesStudies

ClinicalClinicalStudiesStudies

NIH Extramural NIH Extramural AwardsAwards

GrantGrant PatronPatron (assistance, (assistance,

encouragement)encouragement)

CooperativeCooperative PartnerPartnerAgreementAgreement (assistance but substantial(assistance but substantial program involvement)program involvement)

ContractContract PurchaserPurchaser (procurement)(procurement)

NIH Funding in FY 2004: By NIH Funding in FY 2004: By MechanismMechanismTotal = $27BTotal = $27B

Research Project Grants54%

Research Centers9%

Other Research Grants6%

Research Training3%

R&D Contracts10%

Research Management4%

Nat. Lib. Of Medicine1%

Cancer Prev. & Control

2%

Construction & Facilities

1%

Intramural Research10%

Program Announcement Program Announcement (PA)(PA) Invites grant applications in a given Invites grant applications in a given

research arearesearch area

May describe new or expanded interest May describe new or expanded interest in a particular extramural programin a particular extramural program

May be a reminder of a continuing May be a reminder of a continuing interest in a particular extramural interest in a particular extramural programprogram

Generally has no funds set asideGenerally has no funds set aside

Applications reviewed in CSR along with Applications reviewed in CSR along with unsolicited grant applicationsunsolicited grant applications

Request for Applications Request for Applications (RFA)(RFA)

Invites grant applications in a defined Invites grant applications in a defined research area for a one-time competitionresearch area for a one-time competition

Usually targets a perceived area of Usually targets a perceived area of scientifically needed information for scientifically needed information for which the technology is availablewhich the technology is available

Single receipt date designatedSingle receipt date designated

Specifies amount of funds set aside and Specifies amount of funds set aside and expected number of awardsexpected number of awards

Applications reviewed by an Institute Applications reviewed by an Institute SEPSEP

Request for Proposals Request for Proposals (RFP)(RFP) Invites contract proposals in a given Invites contract proposals in a given

research arearesearch area

Describes specific need(s) to be met by Describes specific need(s) to be met by contractor and lists milestones for contractor and lists milestones for progressprogress

Single receipt date designatedSingle receipt date designated

Projected costs are estimated and Projected costs are estimated and funds are set asidefunds are set aside

Proposals are reviewed by Institute Proposals are reviewed by Institute Contract Review CommitteeContract Review Committee

Dual Review System for Grant Dual Review System for Grant ApplicationsApplications

Second Level of Second Level of ReviewReview

National Advisory CouncilNational Advisory Council Assesses Quality of SRGAssesses Quality of SRG Review of Grant ApplicationsReview of Grant Applications Makes Recommendation toMakes Recommendation to Institute Staff on FundingInstitute Staff on Funding Evaluates Program PrioritiesEvaluates Program Priorities and Relevanceand Relevance Advises on PolicyAdvises on Policy

First Level of ReviewScientific Review Scientific Review

GroupGroup Provides Initial Scientific Provides Initial Scientific

MeritMerit Review of Grant ApplicationsReview of Grant Applications Rates Applications and Rates Applications and

Makes Recommendations Makes Recommendations for Appropriate Level of for Appropriate Level of Support and Duration of Support and Duration of AwardAward

GROUPSGROUPS

CSR IRGsCSR IRGsStudy SectionsStudy Sections

Special Emphasis PanelsSpecial Emphasis Panels

INSTITUTESINSTITUTESScientific Review GroupsScientific Review Groups

Contract Review CommitteesContract Review Committees

APPLICATIONS REVIEWEDAPPLICATIONS REVIEWEDResearch ProjectsResearch Projects

Academic Research Academic Research Enhancement AwardsEnhancement Awards

Postdoctoral FellowshipsPostdoctoral FellowshipsSmall Business Innovation Small Business Innovation

ResearchResearchShared InstrumentationShared Instrumentation

Program ProjectsProgram ProjectsCentersCenters

Institutional Training GrantsInstitutional Training GrantsConference GrantsConference GrantsCareer AwardsCareer Awards

Types of Scientific Review Types of Scientific Review GroupsGroupsWhere are Applications Where are Applications Reviewed?Reviewed?

Small GrantsSmall GrantsRFAsRFAsContractsContracts

Center for Scientific Center for Scientific ReviewReview Serves as central receipt point for most Serves as central receipt point for most

PHS grant applicationsPHS grant applications Assigns applications to CSR Integrated Assigns applications to CSR Integrated

Review Groups/Study Sections or Review Groups/Study Sections or Institute Scientific Review GroupsInstitute Scientific Review Groups

Assigns applications to NIH Institute(s) Assigns applications to NIH Institute(s) as potential funding component(s)as potential funding component(s)

Conducts initial scientific merit review of Conducts initial scientific merit review of most research applications submitted to most research applications submitted to the NIH in more than 100 Study Sections the NIH in more than 100 Study Sections

Applications Submitted to Applications Submitted to NIHNIH

Over 60,000 grant Over 60,000 grant applications are applications are submitted to NIH submitted to NIH each year, of each year, of which 25-30% are which 25-30% are fundedfunded

Competing grant Competing grant applications are applications are received for three received for three review cycles per review cycles per yearyear

Competing Applications Competing Applications Reviewed By:Reviewed By:

NOTE: Starting in FY93, NIDA, NIAAA, and NIMH are included in NIH totals

Fiscal Year PHS NIH CSR

1976 27,213 19,509 16,268

1985 31,807 28,407 23,626

1995 44,342 41,899 29,203

1996 38,579 38,012 27,253

1997 38,767 38,065 26,516

1998 39,474 37,855 26,074

1999 44,395 42,383 31,837

2000 45,705 43,814 32,091

2001 46,925 43,411 30,665

2002 52,367 48,677 34,135

2003 64,187 59,440 41,899

Applications are Applications are Assigned to:Assigned to:

Scientific review groups Scientific review groups based on:based on:–Specific review guidelines for Specific review guidelines for each scientific review groupeach scientific review group

Institutes based on:Institutes based on:–Overall mission of the InstituteOverall mission of the Institute–Specific programmatic Specific programmatic mandates and interests of the mandates and interests of the InstituteInstitute

Assignment to CSR Study Assignment to CSR Study Sections Sections

Applications are assigned for review toApplications are assigned for review to

Standing Study Sections Standing Study Sections when the subject when the subject matter of the application matches the matter of the application matches the referral guidelines for the study sectionreferral guidelines for the study section

Ad Hoc Special Emphasis Panels (SEPs) Ad Hoc Special Emphasis Panels (SEPs) when the subject matter does not fit into when the subject matter does not fit into any study section, or when assignment of any study section, or when assignment of an application to the most appropriate an application to the most appropriate study section would create a conflict of study section would create a conflict of interest. Also used for special interest. Also used for special mechanisms (e.g., fellowships, SBIRs, mechanisms (e.g., fellowships, SBIRs, AREAS)AREAS)

Peer Review in CSR Peer Review in CSR Study SectionsStudy Sections

CSR Study Sections are managed by a CSR Study Sections are managed by a Scientific Review Administrator (SRA) who is a Scientific Review Administrator (SRA) who is a professional, usually at the Ph.D. level, whose professional, usually at the Ph.D. level, whose scientific background is close to the expertise scientific background is close to the expertise of the study sectionof the study section

Each CSR standing study section has 12 - 24 Each CSR standing study section has 12 - 24 members who are primarily from academiamembers who are primarily from academia

As many as 60 - 100 applications are reviewed As many as 60 - 100 applications are reviewed at each study section meetingat each study section meeting

Scientific Review Scientific Review AdministratorAdministrator

Performing administrative and technical review of applications to ensure completeness and accuracy

Selecting reviewers based on broad input

Managing study section meetings Preparing summary statements Providing any requested information

about study section recommendations to Institutes and National Advisory Councils/Boards

Designated Federal official with overall responsibility for the review process, including:

Criteria For Selection of Peer Criteria For Selection of Peer ReviewersReviewers

Demonstrated Scientific ExpertiseDemonstrated Scientific Expertise Doctoral Degree or EquivalentDoctoral Degree or Equivalent Mature Judgment Mature Judgment Work Effectively in a Group Work Effectively in a Group

ContextContext Breadth of PerspectiveBreadth of Perspective ImpartialityImpartiality Interest in ServingInterest in Serving Adequate Representation of Adequate Representation of

Women and Minority ScientistsWomen and Minority Scientists

Certification of No Conflict of Certification of No Conflict of InterestInterest

This will certify that in the review of applications This will certify that in the review of applications and proposals by and proposals by (study section(study section)) on on (date)(date), I did , I did not participate in the evaluation of any grant or not participate in the evaluation of any grant or fellowship applications from (1) any fellowship applications from (1) any organization, institution or university system in organization, institution or university system in which a financial interest exists to myself, which a financial interest exists to myself, spouse, parent,child, or collaborating spouse, parent,child, or collaborating investigators; (2) any organization in which I investigators; (2) any organization in which I serve as officer, director, trustee, employee or serve as officer, director, trustee, employee or collaborating investigator; or (3) any collaborating investigator; or (3) any organization which I am negotiating or have organization which I am negotiating or have any arrangements concerning prospective any arrangements concerning prospective employment or other such associations.employment or other such associations.

__________________ ____________________________________ __________________ __________________ ____________________________________ __________________ __________________ ____________________________________ __________________ __________________ ____________________________________ __________________

SIGNATURESSIGNATURES

ConfidentialityConfidentiality Review materials and proceedings of Review materials and proceedings of

review meetings represent privileged review meetings represent privileged information to be used only by information to be used only by consultants and NIH staff.consultants and NIH staff.

At the conclusion of each meeting, At the conclusion of each meeting, consultants will be asked to destroy or consultants will be asked to destroy or return all review-related material.return all review-related material.

Consultants should not discuss review Consultants should not discuss review proceedings with anyone except the SRA.proceedings with anyone except the SRA.

Questions concerning review proceedings Questions concerning review proceedings should be referred to the SRA.should be referred to the SRA.

Review of Research Review of Research GrantsGrants

REVIEW CRITERIA:REVIEW CRITERIA: Significance Significance ApproachApproach InnovationInnovation InvestigatorInvestigator EnvironmentEnvironment Overall EvaluationOverall Evaluation

Review Criteria Review Criteria (defined)(defined) Significance: Significance: Does the study address an important Does the study address an important

problem? How will scientific knowledge be problem? How will scientific knowledge be advanced?advanced?

Approach: Approach: Are design and methods well-developed Are design and methods well-developed and appropriate? Are problem areas addressed?and appropriate? Are problem areas addressed?

Innovation: Innovation: Are there novel concepts or Are there novel concepts or approaches? Are the aims original and innovative?approaches? Are the aims original and innovative?

Investigator: Investigator: Is the investigator appropriately Is the investigator appropriately trained?trained?

Environment: Environment: Does the scientific environment Does the scientific environment contribute to the probability of success? Are there contribute to the probability of success? Are there unique features of the scientific environment?unique features of the scientific environment?

Research Involving Human Research Involving Human SubjectsSubjects

Important Important ConsiderationsConsiderations

Is the proposed study exempt from Is the proposed study exempt from human subject review?human subject review?

Are there any apparent risks* to the Are there any apparent risks* to the human subjects?human subjects?

Are the protections adequate?Are the protections adequate? What are the potential benefits to What are the potential benefits to

the subjects and to mankind?the subjects and to mankind? Are the inclusions of minorities and Are the inclusions of minorities and

both genders adequately addressed?both genders adequately addressed?*”Risks” include the possibility of physical, psychological, or social injury resulting from research.

Common Problems in Common Problems in ApplicationsApplications Lack of new or original ideasLack of new or original ideas Absence of an acceptable scientific rationaleAbsence of an acceptable scientific rationale Lack of experience in the essential methodologyLack of experience in the essential methodology Questionable reasoning in experimental Questionable reasoning in experimental

approachapproach Uncritical approachUncritical approach Diffuse, superficial, or unfocused research planDiffuse, superficial, or unfocused research plan Lack of sufficient experimental detailLack of sufficient experimental detail Lack of knowledge of published relevant workLack of knowledge of published relevant work Unrealistically large amount of workUnrealistically large amount of work Uncertainty concerning future directionsUncertainty concerning future directions

Scientific Review Group Scientific Review Group or Study Section Actionsor Study Section Actions

Scored, Scientific Merit Scored, Scientific Merit Rating (priority scores Rating (priority scores and percentiles)and percentiles)

Unscored (lower half; Unscored (lower half; Streamlined)Streamlined)

DeferralDeferral

ActionAction Scored - Scientific Merit Rating 1.0 Scored - Scientific Merit Rating 1.0

to to approximately 3.0 approximately 3.0

Based on the relevant review criteria, Based on the relevant review criteria, the application is judged to be in the the application is judged to be in the upper half of applications reviewed by upper half of applications reviewed by the study section or scientific review the study section or scientific review group. The recommendation can be group. The recommendation can be for the requested time and amount or for the requested time and amount or for an adjusted time and amount. A for an adjusted time and amount. A priority score is provided, and a priority score is provided, and a summary statement prepared that summary statement prepared that incorporates the written critiques plus incorporates the written critiques plus a resume and summary of the a resume and summary of the discussion.discussion.

ActionAction UnscoredUnscored

Application is unanimously judged Application is unanimously judged to be in the lower half of to be in the lower half of applications reviewed by the applications reviewed by the study section or scientific review study section or scientific review group. No priority score is group. No priority score is assigned. The summary assigned. The summary statement provided to the statement provided to the applicant is a compilation of applicant is a compilation of reviewers’ comments prepared reviewers’ comments prepared prior to the meeting.prior to the meeting.

Calculation of Calculation of PercentilesPercentiles

Based on the priority score, the percentile is Based on the priority score, the percentile is the rank of an application relative to others the rank of an application relative to others reviewed over three cycles. It indicates the reviewed over three cycles. It indicates the percentage of applications with better priority percentage of applications with better priority scores.scores.

Percentile = (Percentile = (relative rank – 0.5)relative rank – 0.5) x x 100100

# of applications# of applications____________________

For Special Emphasis Panels, the percentile values For Special Emphasis Panels, the percentile values are intercalated on a table of all-CSR revieware intercalated on a table of all-CSR review

Summary Summary StatementStatement Once applications have been Once applications have been

reviewed, the results are documented reviewed, the results are documented by the SRA in a summary statement by the SRA in a summary statement and released to the NIH Commons and released to the NIH Commons (available to Institutes and (available to Institutes and applicants)applicants)

The summary statement contains:The summary statement contains:•Resume and Summary of DiscussionResume and Summary of Discussion•Minimally-edited CritiquesMinimally-edited Critiques•Priority Score and Percentile RankingPriority Score and Percentile Ranking•Budget RecommendationsBudget Recommendations•Administrative NotesAdministrative Notes

Dual Review System for Grant Dual Review System for Grant ApplicationsApplications

Second Level of Second Level of ReviewReview

National Advisory CouncilNational Advisory Council Assesses Quality of SRGAssesses Quality of SRG Review of Grant ApplicationsReview of Grant Applications Makes Recommendation toMakes Recommendation to Institute Staff on FundingInstitute Staff on Funding Evaluates Program PrioritiesEvaluates Program Priorities and Relevanceand Relevance Advises on PolicyAdvises on Policy

First Level of ReviewScientific Review Scientific Review

GroupGroup Provides Initial Scientific Provides Initial Scientific

MeritMerit Review of Grant ApplicationsReview of Grant Applications Rates Applications and Rates Applications and

Makes Recommendations Makes Recommendations for Appropriate Level of for Appropriate Level of Support and Duration of Support and Duration of AwardAward

Extramural Program Extramural Program MissionMission

Identify scientific opportunities

Foster the best science

Ensure proper stewardship

Promote effective communication

Manage a portfolio of investments toimprove health through science

Council ActionsCouncil Actions

Concurrence with study Concurrence with study section actionsection action

Modification of study Modification of study section actionsection action

Deferral for re-reviewDeferral for re-review

What Determines What Determines Which Awards Are Which Awards Are Made?Made?

Scientific meritScientific merit

Program considerationsProgram considerations

Availability of fundsAvailability of funds