Research Position Evaluation System (RPES) or How you get what you deserve New SY Orientation...

Research Position Evaluation Research Position Evaluation SystemSystem(RPES)(RPES)

oror

How you get what you How you get what you deservedeserve

New SY Orientation October 15-18, 2007

OutlineOutline

PurposePurpose

ProcessProcess

Panel OperationsPanel Operations

Results and OutcomesResults and Outcomes

ResourcesResources

PurposePurpose

Exercise delegated authority from the Exercise delegated authority from the Administrator to accurately determine the Administrator to accurately determine the grade levels of ARS research scientist grade levels of ARS research scientist positions using peer panels.positions using peer panels.Not a “Promotion System”Not a “Promotion System”Not a “Tenure System”Not a “Tenure System”Based on “person-in-the-job” concept. Based on “person-in-the-job” concept. Open-ended promotion potential based on Open-ended promotion potential based on personal research & leadership personal research & leadership accomplishments.accomplishments.Only for Category 1 positionsOnly for Category 1 positions

ProcessProcess

RL is notified that a SY will be RL is notified that a SY will be reviewed.reviewed.

SY, with RL support, constructs the SY, with RL support, constructs the case write-upcase write-up

Case is sent to Area OfficeCase is sent to Area Office

Louise Dalton checks the case for Louise Dalton checks the case for adherence to format.adherence to format.

ProcessProcess

Mickey McGuire (AAD) reviews the Mickey McGuire (AAD) reviews the case for content.case for content.

Case is returned to SY for revision.Case is returned to SY for revision.

Final case is sent to RPES staff Final case is sent to RPES staff through Area Office.through Area Office.

Case is then sent to panel for Case is then sent to panel for evaluationevaluation..

TimingTiming

GS 11 – Every 3 yearsGS 11 – Every 3 years





ST – Every 5 yearsST – Every 5 years

Approved ExceptionsApproved Exceptions

Early ReviewEarly Review

Delayed ReviewDelayed Review

ReevaluationReevaluation

Poor PerformerPoor Performer

Anatomy of a Case Write-up Anatomy of a Case Write-up

Must be accurateMust be accurate– RL/Supervisor assistsRL/Supervisor assists

Career long accomplishmentsCareer long accomplishments

Recency matters!Recency matters!

Must be well documentedMust be well documented

In depth contact list is very importantIn depth contact list is very important

Peer group is importantPeer group is important

ARS Peer GroupsARS Peer Groups

Animal Bioscience (ABS)Animal Bioscience (ABS) Animal Health (AHL)Animal Health (AHL) Chemistry & Chemical Engineering (CCE)Chemistry & Chemical Engineering (CCE) Entomology (ENT)Entomology (ENT) Food Science & Human Nutrition (FHN)Food Science & Human Nutrition (FHN) Plant Bioscience (PBS)Plant Bioscience (PBS) Plant Health (PHL)Plant Health (PHL) Pre/Post Harvest Science & Engineering (PHS)Pre/Post Harvest Science & Engineering (PHS) Soil & Water Science (SWS)Soil & Water Science (SWS) Systematics & Taxonomy (STX)Systematics & Taxonomy (STX)

Research Grade Evaluation GuideResearch Grade Evaluation Guide

Four evaluation factorsFour evaluation factors– 1 - Research Assignment1 - Research Assignment– 2 - Supervision Received2 - Supervision Received– 3 - Guidelines & Originality3 - Guidelines & Originality– 4 - Qualifications and Contributions4 - Qualifications and Contributions

Factor 4 is double-weighted... to Factor 4 is double-weighted... to implement the person-in-the-job implement the person-in-the-job conceptconcept

Factor 1 – Factor 1 – Research AssignmentResearch Assignment

Assigned ResponsibilityAssigned Responsibility– Mention formal leadership responsibility (Lead Scientist, Research Mention formal leadership responsibility (Lead Scientist, Research

Leader, Director)Leader, Director)

Research Objectives & MethodologyResearch Objectives & Methodology– (next 3-5 yr)(next 3-5 yr)

Expected Results - Expected Results - impact on science/technologyimpact on science/technologyKnowledge RequiredKnowledge Required

Supervisory ResponsibilitiesSupervisory Responsibilities

Factor 2 – Factor 2 – Supervision ReceivedSupervision Received

Assigned Responsibility - Assigned Responsibility - freedom to controlfreedom to control

Technical Guidance ReceivedTechnical Guidance Received

Review of Results - manuscript review/approvalReview of Results - manuscript review/approval

General Supervision - frequency/nature/authorityGeneral Supervision - frequency/nature/authority

Factor 3 – Factor 3 – Guidelines and OriginalityGuidelines and Originality

Available Literature - applies to research areaAvailable Literature - applies to research areaOriginality Required – to obtain expected resultsOriginality Required – to obtain expected results

(End of the Position Description; Limited to 3 pages; all (End of the Position Description; Limited to 3 pages; all gender neutral)gender neutral)

Demonstrated Originality – What is it about you that Demonstrated Originality – What is it about you that enabled the accomplishments. Use examples of enabled the accomplishments. Use examples of accomplishments but don’t simply recite them.accomplishments but don’t simply recite them.

Factor 4 - Qualifications and Factor 4 - Qualifications and Contributions: Entire CareerContributions: Entire Career

Demonstrated Accomplishments Demonstrated Accomplishments – What was accomplished?What was accomplished?– What was your role?What was your role?– Impact (So What? Who Cares?)Impact (So What? Who Cares?)

Stature and Recognition (Awards, Stature and Recognition (Awards, Invitations (even those you could not Invitations (even those you could not accept), Professional Society Activities)accept), Professional Society Activities)

Factor IV - Qualifications and Factor IV - Qualifications and Contributions: Entire CareerContributions: Entire Career

Advisory and Consultant Activities Advisory and Consultant Activities

Other (Education, Experience, Other Significant Other (Education, Experience, Other Significant Information)Information)

PublicationsPublications• Peer reviewed journal articles and patentsPeer reviewed journal articles and patents• Non Peer reviewed (proceedings, etc.)Non Peer reviewed (proceedings, etc.)• Abstracts not listedAbstracts not listed

Demonstrated AccomplishmentsDemonstrated Accomplishments

GS 11 and 12 – Allowed 5 accomplishment GS 11 and 12 – Allowed 5 accomplishment statementsstatements

GS 13 and above – Allowed 8GS 13 and above – Allowed 8

Limited to a half page eachLimited to a half page each

Each accomplishment must be documented with Each accomplishment must be documented with two exhibits, typically senior authored, peer two exhibits, typically senior authored, peer reviewed manuscripts. Can be letters from reviewed manuscripts. Can be letters from stakeholders (through the AD)stakeholders (through the AD)

IDR list should contain at least one individual IDR list should contain at least one individual knowledgeable about each accomplishmentknowledgeable about each accomplishment

Demonstrated AccomplishmentsDemonstrated Accomplishments

What was accomplished? What was accomplished? – Created, Discovered, Solved a problem, etc.Created, Discovered, Solved a problem, etc.

What role did you play?What role did you play?– Conceived, Led, Wrote Manuscripts, Grant Conceived, Led, Wrote Manuscripts, Grant

Proposals, etc.Proposals, etc.

What is the impact (so what and who cares)?What is the impact (so what and who cares)?– Who has adopted, How has it impacted Who has adopted, How has it impacted

science, What recognition was received science, What recognition was received (awards, grants funded, etc.)(awards, grants funded, etc.)

Types of AccomplishmentsTypes of Accomplishments

ResearchResearch

Special Assignments or ProjectsSpecial Assignments or Projects

Technology TransferTechnology Transfer

Systems Research and IntegrationSystems Research and Integration

Leadership (RL and Scientific)Leadership (RL and Scientific)

Accomplishment Statement Accomplishment Statement ExamplesExamples

Accomplishment:Accomplishment: Adjuvants for use with microbial and Adjuvants for use with microbial and chemical pesticides must ensure protection from sunlight chemical pesticides must ensure protection from sunlight and rainfall and be useful in relatively small amounts. and rainfall and be useful in relatively small amounts. Wheat gluten was chemically modified to make an Wheat gluten was chemically modified to make an adjuvant that protects adjuvant that protects Bacillus thuringiensisBacillus thuringiensis from sunlight from sunlight and washoff by rainfall when added to a spray tank at and washoff by rainfall when added to a spray tank at 1% solids. The adjuvant also helps a corn rootworm 1% solids. The adjuvant also helps a corn rootworm insecticidal formulation called SLAM adhere to corn insecticidal formulation called SLAM adhere to corn plants and the adjuvant, when combined with feeding plants and the adjuvant, when combined with feeding stimulants, can be used to stick other corn rootworm stimulants, can be used to stick other corn rootworm pesticides to plants thus enabling reduction of active pesticides to plants thus enabling reduction of active ingredient. ingredient.


Role:Role: Dr. X led the team and co- Dr. X led the team and co-conceived, co-developed and co-patented conceived, co-developed and co-patented the technology. He supervised writing the the technology. He supervised writing the manuscripts and patent applications. manuscripts and patent applications.


Impact: Impact: This technology created another option for managers This technology created another option for managers charged with controlling corn rootworm adults on corn. SLAM charged with controlling corn rootworm adults on corn. SLAM is the only product with reduced rates of active ingredient is the only product with reduced rates of active ingredient currently on the market. SLAM does not adhere well to currently on the market. SLAM does not adhere well to plants and must be reapplied. By mixing the gluten adjuvant plants and must be reapplied. By mixing the gluten adjuvant with SLAM, the product remains on the plants longer and can with SLAM, the product remains on the plants longer and can resist wash off. The Area-Wide Corn Rootworm Adult resist wash off. The Area-Wide Corn Rootworm Adult Management Project funded Dr. X’s work for ca. $100,000 Management Project funded Dr. X’s work for ca. $100,000 from 1997-2000. Microflo, the company that produces SLAM, from 1997-2000. Microflo, the company that produces SLAM, licensed the use of gluten and reformulated 30,000 pounds licensed the use of gluten and reformulated 30,000 pounds for use in the summer of 2000. In addition, Trécé for use in the summer of 2000. In addition, Trécé Corporation licensed the use of gluten to combine it with a Corporation licensed the use of gluten to combine it with a potent feeding stimulant as an adjuvant that can be used with potent feeding stimulant as an adjuvant that can be used with reduced rates of other pesticides registered for use against reduced rates of other pesticides registered for use against corn rootworm adults. The product was launched in Summer corn rootworm adults. The product was launched in Summer 2000. (Exhibit 3a, #24; Exhibit 3b, #25, and #71, #41)2000. (Exhibit 3a, #24; Exhibit 3b, #25, and #71, #41)


Accomplishment/Role: Dr. Y postulated Accomplishment/Role: Dr. Y postulated that direct mechanical inoculation of the that direct mechanical inoculation of the vascular tissues in seeds will bypass the vascular tissues in seeds will bypass the need for vectors to transmit maize viruses. need for vectors to transmit maize viruses. This elegant, unconventional, and simple This elegant, unconventional, and simple approach resulted in a highly efficient approach resulted in a highly efficient method for mechanical transmission of (6 method for mechanical transmission of (6 viruses listed) into maize.viruses listed) into maize.


Impact: Among other benefits, this research Impact: Among other benefits, this research provided a unique solution to study viruses provided a unique solution to study viruses without the confounding effect of vectors, without the confounding effect of vectors, eliminated or reduced the intensive labor eliminated or reduced the intensive labor requirements of insect rearing, expedited tests requirements of insect rearing, expedited tests on infectivity of virus preparations, provided a on infectivity of virus preparations, provided a means to study the mechanism of resistance to means to study the mechanism of resistance to systemic virus movement and to study virus systemic virus movement and to study virus resistance independently from vector resistance, resistance independently from vector resistance, and facilitated studies that manipulate and facilitated studies that manipulate recombinant viral clones (Exhibit 8a, #85, Exhibit recombinant viral clones (Exhibit 8a, #85, Exhibit 8b #92, and #87)8b #92, and #87)


Comprised of 5 scientists GS 13 and above Comprised of 5 scientists GS 13 and above representing two peer groups; a personnel representing two peer groups; a personnel representative, and a panel chair. Panel representative, and a panel chair. Panel members are anonymous.members are anonymous.One panel member (within your peer group) is One panel member (within your peer group) is assigned to do an in depth review of your case assigned to do an in depth review of your case (panel member may have additional cases to do (panel member may have additional cases to do the in depth review)the in depth review)– Reads manuscriptsReads manuscripts– Calls in depth contacts and others (a minimum of 5 Calls in depth contacts and others (a minimum of 5

contacts, including supervisor, are required)contacts, including supervisor, are required)– Prepares draft panel reportPrepares draft panel report


All members score each case prior to All members score each case prior to meetingmeetingPanel meets face to face to discuss proper Panel meets face to face to discuss proper grade according to the RGEG grade according to the RGEG – Consensus approachConsensus approach

Panels will cover up to 12 cases in a day Panels will cover up to 12 cases in a day and a half; surprisingly most lower graded and a half; surprisingly most lower graded cases require the most amount of time so cases require the most amount of time so it is essential the case is well written.it is essential the case is well written.

Grade DeterminationGrade Determination

DegreeDegree

FactorsFactors AA BB CC DD EE

11 22 44 66 88 1010

22 22 44 66 88 1010

33 22 44 66 88 1010

44 44 88 1212 1616 2020

Grade DeterminationGrade Determination

GS 11GS 11 8-148-14

GS 12GS 12 16-2416-24

GS 13GS 13 26-3426-34

GS 14GS 14 36-4436-44

GS 15GS 15 46-5446-54

RPES Panel Decision RPES Panel Decision OptionsOptions

Upgrade (UPG)Upgrade (UPG)

Remain in Grade (RIG)Remain in Grade (RIG)

Insufficient Factual Basis (IFB)Insufficient Factual Basis (IFB)

Grade/Category Problem (GCP)Grade/Category Problem (GCP)

Refer to Supergrade (REF)Refer to Supergrade (REF)

Split (SPL)Split (SPL)

Factor 1: Research AssignmentFactor 1: Research Assignment

Degree C: Degree C: – The incumbent is responsible for formulating The incumbent is responsible for formulating

and conducting a systematic research attack and conducting a systematic research attack on a problem area of considerable scope and on a problem area of considerable scope and complexity…..complexity…..

– Characteristically, research studies of this Characteristically, research studies of this scope will result in a series of publishable scope will result in a series of publishable contributions to knowledge….contributions to knowledge….

Factor 2: Supervision ReceivedFactor 2: Supervision Received

Degree CDegree C– In programmed or applied research, the researcher is In programmed or applied research, the researcher is

typically assigned a broad problem area….typically assigned a broad problem area….– The researcher is responsible, with little or no The researcher is responsible, with little or no

supervisory assistance, for formulating hypotheses, supervisory assistance, for formulating hypotheses, for developing and carrying out the plan of attack….for developing and carrying out the plan of attack….

– The supervisor is kept informed, through occasional The supervisor is kept informed, through occasional discussions, of general plans and progress of the discussions, of general plans and progress of the work….work….

Factor 3: Guidelines and OriginalityFactor 3: Guidelines and Originality

Degree CDegree C– In basic research, available guides and In basic research, available guides and

precedents, e.g. existing literature in the field, precedents, e.g. existing literature in the field, are limited in usefulness or largely lacking due are limited in usefulness or largely lacking due to the novel character of the work being done.to the novel character of the work being done.

– In applied research, this degree typically In applied research, this degree typically involves development and application of new involves development and application of new techniques and original methods of attack to techniques and original methods of attack to the solution of important problems.the solution of important problems.

Factor 4: Qualifications and Factor 4: Qualifications and Scientific ContributionsScientific Contributions

Degree CDegree C– At this degree, the researcher has demonstrated his At this degree, the researcher has demonstrated his

or her ability as a mature, competent and productive or her ability as a mature, competent and productive worker. He/she will have typically authored one or worker. He/she will have typically authored one or more publications of considerable interest and value more publications of considerable interest and value to the field.to the field.

– Contribution involves leadership of a productive Contribution involves leadership of a productive research team or leadership in the conception and research team or leadership in the conception and formulation of productive research ideas … with such formulation of productive research ideas … with such soundness and value as to have marked him/her as a soundness and value as to have marked him/her as a significant contributor to their professional field.significant contributor to their professional field.

RESEARCH POSITION EVALUATION WORKSHEET Name: ===================================================================================================== Initial Score Summary: F1 ____ + F2 ____ + F3 ____ + F4 ____ = ____ Points

FACTOR 4 - CONTRIBUTIONS, IMPACT, AND STATUREA. Demonstrated Accomplishments:In brief, complete sentences, summarize significance and impact of the 3 highest-rated accomplishments, and explain incumbent's role in each:B. Stature and Recognition:Incumbent’s stature is evidenced by the following 3-5 honors, awards, and/or invitations:Incumbent is [ ] locally [ ] regionally [ ] nationally [ ] internationally recognized for research in: Incumbent has a [ ] poor, [ ] fair, [ ] good, [ ] excellent record of participation in scientific meetings.C. Advisory and Consultant Activities:Incumbent's 3-5 most significant advisory and consultant activities include:The panel assigned Level ____ for this factor because:Concerns/deficiencies noted for criteria of this factor:[ ] Recency of accomplishment[ ] Diminished stature/recognition[ ] Diminished advisory/consultant activityARS Form 516 (9/06)

FACTOR 1 - RESEARCH ASSIGNMENT

A. Assigned Responsibility:Incumbent occupies a [ ] Lead Scientist [ ] Research Leader [ ] Center/Laboratory Director position, or [ ] has no formal leadership role.Incumbent's current assignment involves research to (identify assignment limits/boundaries):In team research, incumbent usually functions as [ ] leader or [ ] member, providing expertise in:Assignment is [ ] continuing or [ ] new (since _____) [NLT 4 years back; not applicable for first post-hire panel review].Incumbent has a substantial [ ] service [ ] technology transfer [ ] other special assignment (identify ________________________________________________________________________________________) requiring approximately____% of duty time.B. Objectives and Methodology: (for next 3-4 years) Objectives are to:Of particular significance are current studies on:Methods and approaches employed are best characterized as [ ] routine [ ] unusual [ ] novel [ ] sophisticated [ ] requiring considerable modification, because:C. Expected Results:Successful research should result in:The panel assigned Level ____ for this factor because:Concerns/deficiencies noted for criteria of this factor:[ ] Results are not reflected as accomplishments and/or advisory/consultant activities.[ ] There is little evidence that substantial progress is being made in this continuing assignment.ARS Form 516 (9/06) (p. 2)

FACTOR 2 - SUPERVISORY CONTROLS

A. Assigned Authority: (freedom to make decisions within scope of assignment)Within the [ ] specific [ ] identifiable [ ] broad [ ] unusually broad assignment, incumbent has the freedom to:Specific problems for study, such as work on _____________________________________________________________, are [ ] assigned by the supervisor [ ] selected by incumbent subject to approval by the supervisor [ ] the total responsibility of the incumbent.B. Technical Guidance Received: (extent of review by supervisor prior to conducting experiments)Technical guidance involves:C. Review of Results: (extent of review/verification of results/findings by the supervisor)[ ] Manuscripts and other reports are reviewed by the supervisor as required by ARS policy. OR[ ] Manuscripts and other reports are reviewed by the supervisor for: D. General Supervision:[ ] Any [ ] Major [ ] Only broad changes in research require the supervisor's approval.Incumbent has responsibility for:and freedom to:The panel assigned Level ____ for this factor because:Concerns/deficiencies noted for criteria of this factor:[ ] Extent of supervision is not consistent with the demonstrated capability of the scientist or progress being made.ARS Form 516 (9/06)

FACTOR 3 - GUIDELINES AND ORIGINALITY

A. Available Literature: (extent to which literature is available/relevant to the assignment)Literature on ____________________________ is available, but specific information on __________________________________________is lacking.Available techniques such as __________________________________________________________________________________________ [ ] are useful [ ] require minimal adaptation [ ] require major adaptation [ ] are unusable.B. Originality Required: (aspects which make planning/conducting research and interpreting results especially difficult)Originality is required to:The assignment is difficult because:C. Demonstrated Originality:Incumbent's originality is evidenced by:The panel assigned Level ____ for this factor because:Concerns/deficiencies noted for criteria of this factor:[ ] The originality required in this position is not consistent with originality demonstrated by incumbent to date. ARS Form 516 (9/06)

Quantity is Important...Quantity is Important...

...but it is the quantity of quality ...but it is the quantity of quality contributions--not mere numbers of contributions--not mere numbers of

publications--that matters.publications--that matters.

There are several degrees of activity:There are several degrees of activity:– Activity which maintains a career i.e. Activity which maintains a career i.e.

justifies current grade leveljustifies current grade level– Activity which limits or diminishes a career Activity which limits or diminishes a career

ie. mundane, insignificant or undirected ie. mundane, insignificant or undirected studiesstudies

– Activity which advances a career because Activity which advances a career because of its impact i.e. a program that addresses of its impact i.e. a program that addresses important problemsimportant problems

– Inactivity is the surest way to loseInactivity is the surest way to lose– Researchers are not “tenured in grade”Researchers are not “tenured in grade”– Must “keep running just to stay in place”Must “keep running just to stay in place”

Program vs StudiesProgram vs Studies

Helpful HintsHelpful Hints

Keep a fileKeep a file– Honors and AwardsHonors and Awards– Invitations (even those declined)Invitations (even those declined)– Meetings attended and your participationMeetings attended and your participation– Manuscripts that you reviewedManuscripts that you reviewed– Special AssignmentsSpecial Assignments

Have someone else critically read your Have someone else critically read your write-upwrite-upGet examples from successful colleaguesGet examples from successful colleagues

Helpful HintsHelpful Hints

Be sure your 115s are up to dateBe sure your 115s are up to date

Your research program should develop short Your research program should develop short term results as well as long term solutions to term results as well as long term solutions to major problems.major problems.

Start with the end in mindStart with the end in mind– How will the research I am starting result in an How will the research I am starting result in an

accomplishment/impact statement? Who will care accomplishment/impact statement? Who will care and why?and why?

If you are asked to serve on a panel, do so!If you are asked to serve on a panel, do so!

Some StatisticsSome Statistics

Study of new scientists hired at the GS 11 Study of new scientists hired at the GS 11 and GS12 levels in 1999 and 2000and GS12 levels in 1999 and 2000– A total of 135 scientists were hiredA total of 135 scientists were hired– 108 are still with ARS (5 were terminated 108 are still with ARS (5 were terminated

during probation, 1 was terminated after during probation, 1 was terminated after probation and 21 left “voluntarily”)probation and 21 left “voluntarily”)

– 77 scientists were promoted at first RPES, 33 77 scientists were promoted at first RPES, 33 received Remain in Grade scores.received Remain in Grade scores.

Effect of Publications on RPES SuccessEffect of Publications on RPES Success

00.5

11.5

22.5

33.5

Nu

mb

er

of

Pu

blic

ati

on

s/Y

ea

r

UPG 2X

RIG 2X

UPG 1X

RIG 1X

UPG 1X

Before ARS

After Hiring

Eval 2X Eval 1X

ResourcesResources

Research LeaderResearch Leader

Area OfficeArea Office

RPE StaffRPE Staff– Merle ColeMerle Cole– Pat HumphryPat Humphry

Manual 431.3Manual 431.3

Policies and Procedures 431.3Policies and Procedures 431.3

http://www.afm.ars.usda.gov/rpes/http://www.afm.ars.usda.gov/rpes/

Research Position Evaluation System (RPES) or How you get what you deserve New SY Orientation...

Documents

Transcript of Research Position Evaluation System (RPES) or How you get what you deserve New SY Orientation...