Reporting Reporting Results for Results for Pesticide Pesticide ProgramsProgramsRobin Powell, EMRobin Powell, EM

Pyramid Lake Paiute TribePyramid Lake Paiute Tribe

Environmental DepartmentEnvironmental Department

BackgroundBackground

General Performance Reporting Act General Performance Reporting Act (GPRA) Requirements(GPRA) Requirements

Program Assessment Rating Tool (PART)Program Assessment Rating Tool (PART) General Accountability Office (GAO)-General Accountability Office (GAO)-

investigative arm of Congress charged with investigative arm of Congress charged with examining mattersexamining mattersrelating to the receipt and payment of public relating to the receipt and payment of public funds.funds.

Office of Management and Budget (OMB)Office of Management and Budget (OMB)

Passed in 1993Passed in 1993 Phased implementation Phased implementation

Focusing on ALL Federal programsFocusing on ALL Federal programs Goals & Purpose:Goals & Purpose:

Improve program efficiency and effectiveness Improve program efficiency and effectiveness by focusing on resultsby focusing on results

Holding federal agencies accountable and Holding federal agencies accountable and improving the confidence of the American improving the confidence of the American peoplepeople

Improve Congressional rulemaking with Improve Congressional rulemaking with accurate informationaccurate information

GPRAGPRA

GPRAGPRA

As of 1997, all programs are As of 1997, all programs are required to submit an annual required to submit an annual strategic plan, annual report with strategic plan, annual report with results, and a performance report to results, and a performance report to OMBOMB

I.e., EPA Strategic Plan and GoalsI.e., EPA Strategic Plan and Goals Establish performance indicatorsEstablish performance indicators Verification and validation of resultsVerification and validation of results

Outcome MeasureOutcome Measure

““An assessment of the An assessment of the results of a program results of a program

activity compared to its activity compared to its intended purpose”intended purpose”

Output MeasureOutput Measure

““The tabulation, The tabulation, calculation, or recording calculation, or recording of activity or effort and of activity or effort and can be expressed in a can be expressed in a

quantitative or qualitative quantitative or qualitative manner”manner”

Performance GoalPerformance Goal

““AA performance goal means a performance goal means a target level of performance target level of performance

expressed as a tangible, expressed as a tangible, measurable objective, against measurable objective, against which actual achievement can which actual achievement can be compared, including a goal be compared, including a goal

expressed as a quantitative expressed as a quantitative standard, value, or rate”standard, value, or rate”

Performance MeasurePerformance Measure

““A way to measure the A way to measure the activities of a program in activities of a program in

relation to the relation to the performance goal set forth performance goal set forth

by the program.”by the program.”

Performance IndicatorPerformance Indicator

““A performance indicator A performance indicator means a particular value means a particular value or characteristic used to or characteristic used to

measure output or measure output or outcome”outcome”

PARTPART

““A systematic, consistent A systematic, consistent process for developing process for developing program performance program performance ratings and then using ratings and then using

that information to that information to make budget decisions”make budget decisions”

PARTPART

According to Bill Diamond/EPA: According to Bill Diamond/EPA:

PART was “developed to assess and improve program performance so that the Federal government can achieve better results.”

In 2004, Pesticide Field Programs (ES; Worker Protection; WQ; SAI) were assessed and received a score of 36 (results not demonstrated)

Poor score was largely attributable to lack of performance outcome measures and quantifiable data on contribution of field program activities to strategic agency goals.

Reporting Results: Why?Reporting Results: Why?

Will be necessary to remain Will be necessary to remain “aggressive” in budget allocation “aggressive” in budget allocation processprocess Regionally and nationallyRegionally and nationally

Maintain funding levelsMaintain funding levels Improve ability to defend the need for Improve ability to defend the need for

additional fundingadditional funding

Reporting Results: How?Reporting Results: How?

No single output or outcome can No single output or outcome can measure an entire programmeasure an entire program

Integrated approachIntegrated approach All outputs and outcomesAll outputs and outcomes Quantitative/QualitativeQuantitative/Qualitative StatisticalStatistical NarrativeNarrative Aggregated and disaggregatedAggregated and disaggregated

Reporting Results: Reporting Results: What?What?

OutputsOutputs Any numerical data & statistical Any numerical data & statistical

analysesanalyses OutcomesOutcomes

Did the X # of inspections prevent X# Did the X # of inspections prevent X# of unlicensed applicators applying of unlicensed applicators applying RUPs?RUPs?

NarrativeNarrative Qualitative and QuantitativeQualitative and Quantitative

Pesticides: GeneralPesticides: General

Can be data rich or deficient depending Can be data rich or deficient depending on programon program

Assessment resultsAssessment results Number of spray eventsNumber of spray events Comparison of applied GUPs & RUPs Comparison of applied GUPs & RUPs Quantity of pesticides used on tribal Quantity of pesticides used on tribal

landslands Analyses of any gathered sampling data Analyses of any gathered sampling data

NOT raw dataNOT raw data

Pesticides: C&TPesticides: C&T

Number of C&T eventsNumber of C&T events

Number of attendees at each eventNumber of attendees at each event

Comparison of C&T attendeesComparison of C&T attendees Pass vs. FailPass vs. Fail

Pesticides: EnforcementPesticides: Enforcement

Number of spray violationsNumber of spray violations Compliance Assistance activities Compliance Assistance activities

(limited)(limited) Number of inspections performed vs. Number of inspections performed vs.

number of violationsnumber of violations Number of licensed off-reservation Number of licensed off-reservation

applicators who apply on tribal landsapplicators who apply on tribal lands Number of cases initiatedNumber of cases initiated

Pesticides: WPSPesticides: WPS

Number of violation eventsNumber of violation events

Number of citationsNumber of citations

Comparison of violation typesComparison of violation types

Number of inspectionsNumber of inspections

Pesticides: LimitationsPesticides: Limitations Difficult to measure compliance Difficult to measure compliance

Very limited outputs and performance indicatorsVery limited outputs and performance indicators Very difficult to assess the progress in Very difficult to assess the progress in

achieving environmental goalsachieving environmental goals Environmental Educational indicatorsEnvironmental Educational indicators

Difficulty in assessing the compliance Difficulty in assessing the compliance assistance outcomesassistance outcomes Goal of EPAGoal of EPA How to measure the pollution reduction through How to measure the pollution reduction through

these activitiesthese activities

Pesticides: PossibilitiesPesticides: Possibilities

Can improve strategic planningCan improve strategic planning Forecast needs Forecast needs Identify needs and goalsIdentify needs and goals

Budgetary processBudgetary process Defensible requests for additional fundingDefensible requests for additional funding Provides guidelines on funding needsProvides guidelines on funding needs Forecasting future funding requirements Forecasting future funding requirements

to maintain program integrityto maintain program integrity

Reporting Results Example: Reporting Results Example: PesticidesPesticides

1Pesticides

A Comparison of the Frequencies of Pesticides Used on Pyramid Lake Indian

Reservation

2,4-D Glyphosate Carbaryl

Picloram Triclopyr Imazapic

Overall Overall Report Report

Example: Example: PesticidesPesticides

Other Media: Grants Other Media: Grants ManagementManagement

Demonstrate program capacity Demonstrate program capacity buildingbuilding

Demonstrates the need for additional Demonstrates the need for additional support for programs that are deficit support for programs that are deficit in fundingin funding

Enables the program and EPA to Enables the program and EPA to identify tribal needs for additional identify tribal needs for additional programs programs

Can forecast needsCan forecast needs

Reporting Results Example: Reporting Results Example:

Grants ManagementGrants Management

31

49

51

22

411

21

4

0 5 10

# of Grants per Program

1

En

viro

nm

enta

l P

rog

ram

s

FY 2004 Active Grants

ConferenceDoDAirEducationCWA 105Species MgmtMiningPesticides CWA 106Weed ControlER ResponseCWA 319 Lead OutreachSolid Waste

Overall Overall Report Report

Example: Example: Grants Grants MgmtMgmt

Not RecommendedNot Recommended

It is not needed to include full documents It is not needed to include full documents or reports with the results or progress or reports with the results or progress reportreport

Do not have more than 3 pages of Do not have more than 3 pages of narrative per project or programnarrative per project or program

It is not recommended to include It is not recommended to include ordinances or codes within the report.ordinances or codes within the report. Reference within the report the documentsReference within the report the documents

You can have a good results report while You can have a good results report while keeping it simple!!!keeping it simple!!!

ConclusionConclusion

It has been stated repeatedly It has been stated repeatedly that if programs are not that if programs are not producing verifiable producing verifiable “results” through “results” through illustration of outputs illustration of outputs and/or outcomes, those and/or outcomes, those programs will NOT be programs will NOT be funded in the futurefunded in the future

Thank You. Thank You.

Robin Powell, Environmental Robin Powell, Environmental ManagerManager

Environmental DepartmentEnvironmental Department

Pyramid Lake Paiute TribePyramid Lake Paiute Tribe

rpowell@plpt.nsn.usrpowell@plpt.nsn.us

775-574-0101 x12775-574-0101 x12