Performance Measures, Outcomes, and Impacts Sharon Stout Office of Planning and Accountability April...

Performance Measures, Outcomes, and Impacts

Sharon StoutOffice of Planning and Accountability

April 24, 2007

Objectives

● To discuss CSREES use of performance measures in the recent past and in the present;

● To review what CSREES OPA hopes to find in future performance measures, outputs, outcomes, impacts; and

● To encourage ongoing dialogue on performance measurement, measures and indicators within the CSREES Land-Grant University System.

Fundamental Principles

● Axiom 1: What is measured can be improved. ● Axiom 2: The above statement is true – only if

we are measuring the right things.

Critical Elements for Success

1) Select the right things to measure – both controllable and important to achieving success;

2) Measure those things in the right ways – with useable indicators and targets;

3) Employ measures in a framework to improve performance; and

4) Create an organizational culture valuing and encouraging use of measures.

Outline

● CSREES Past Use of Performance Data ● Using a Performance Measures Framework,

including Logic Models, to Improve ● CSREES Examples ● Developing and Using Performance Indicators and

Targets ● Using and Valuing Measures

CSREES Past Use of Performance Data

CSREES Program Leadership and Showcasing Results

CSREES and the Land-grant System are ● Addressing the national problems identified in the

Strategic Plan;● Choosing the right actions to respond; ● Achieving results; and ● Communicating with stakeholders, including

funders, the public, and the Land-grant System.

CSREES Accountability

● Defined as “the state of being accountable; liability to be called on to render an account; the obligation to bear the consequences for failure to perform as expected.”

● CSREES is accountable to the American people, the Administration, and Congress for the use of public funds.

Webster’s Dictionary, 1913

Outputs, Outcomes and Impacts

State Plans of Work and Annual Reports were used to provide examples for:

● Budget justification and narrative;● PART self-assessments; ● USDA Reports of Accomplishments;● Communication with Partners, other agencies, and

the public.

Success Extracting Results?

● CSREES has not been as successful as we would like to be in presenting a systematic report of results from State POW annual reports.

● External panels brought in to review CSREES accomplishments for the Portfolio Review Expert Process noted the lack of an adequate data system – and results.

Representative Comments by Experts in Reports on PREP…

● “The panel was disappointed with the evidence provided…”

● “The portfolio failed to present a complete picture of all the inputs, outputs and outcomes.”

● “There is a strong need to improve accountability showing measurable impacts, not just in CSREES, but throughout the system and down to individual investigators.”

Improved POW System is Raising Panelists’ Expectations

● “… Plan of Work and One Solution are expected to improve the documentation of significant findings.”

● “In 5 years, [the panel] expects to see the consistent information across knowledge areas necessary to evaluate the portfolio properly.”

Expectations of Panelists for CSREES OPA and NPLs

● CSREES OPA will work …”to improve data collection, performance measurement and reporting.”

● “CSREES and NPLs should have better communication with state partners in order to get significant evidence of outputs and impacts.”

What is Happening With CSREES POW and 1Solution?

● Exciting developments – as reported in other presentations here by Bart Hewitt – and CSREES ISTM.

● “Management dashboards” will enable CSREES deputies, NPLs, and P&A to examine trends – and drill down to specific projects.

● States can access information from other states.

Present Status, Future Hopes

● Exciting developments refers to changes in our information systems

● CSREES OPA, ISTM, and Budget Office work together to make better use of our information

● How can we jointly improve the information in our systems?

Reducing Burden, Increasing Use

● CSREES is no longer collecting ‘‘impacts” through a separate system.

● CSREES is relying on Plans of Work and annual reports for both “stories and statistics” for planning, reporting, and budget justification.

Using Performance Measurement

Frameworks to Improve

Performance Measurement Definition and Use

● Ongoing, regular collection of information for monitoring how a policy, program or initiative is doing at any point in time.

● Designed and used to report on the level of attainment of planned results and on performance trends over time.

Performance Measurement Framework

● Begins with design of a policy, program or initiative and evolves over time.

● Always is engaged as part of the ongoing management of a policy, program or initiative.

● Continues from the initial choices of performance measures and indicators, through performance monitoring to formative and summative evaluation.

Performance Measurement Framework Cycle

5

1

0

2 3

4

6

78

SpecifyMeasures

EstablishData collectionProcedures

DevelopInformation System

Measure& Report Performance

Performance MeasurementUnderstanding

FormativeEvaluation

SummativeEvaluation

ProgramProfileLogic model

Review AndModify

FE

ED

BA

CK

Criteria for Assessing PM Frameworks

● Useful for results management and accountability

● Shared ownership ● Transparent ● Decision- and action-oriented ● Credible and realistic ● Flexible

CSREES Use of PM Framework● Planning tools (‘strategic thinking’) -- to focus

attention on desired outcomes● Model to identify and track measures of interest --

inputs, outputs, and outcomes. ● Evaluability assessment – does this program on its

face make sense? (Identify questions to be addressed in an implementation plan)

PM Framework in CSREES Context …

Performance measurement framework is designed with organization

● Mission; ● Vision;● Strategic Goals;● Objectives; and ● Organizational and functional linkages.

CSREES Mission and Vision… Mission ● To advance knowledge for agriculture, the

environment, human health and well-being, and communities

Vision ● To improve the lives of people worldwide through

an agricultural knowledge system sustained by the innovation of scientists and educators.

Types of StrategiesO

rgan

izat

ion

al

Sti

mu

liIntended Strategy

Deliberate Strategy

Realized Strategy

UnrealizedStrategy

Source: Mintzberg, H. Ahlstrand, B. Lempel, J. (1998)

Emergent Stra

tegy

CSREES Strategic Goals ● Enhance international competitiveness of American agriculture;● Enhance competitiveness and sustainability of Rural and Farm

economies;● Support increased economic opportunity and improved quality

of life in rural America;● Enhance protection and safety of the Nation’s agriculture and

food supply;● Improve the Nation’s nutrition and health; and● Protect and enhance the Nation’s Natural Resource Base and

Environment.

CSREES Logic Model as “Roadmap”

Describethe problem, challenge, and opportunities

Situation Inputs Outputs

Occurs when there is a change in knowledgeor theparticipantsactuallylearn

Actions Conditions

Outcomes

$$$Other PartnersNIH, ARS

Describewhat comes out from funded activities

Activities

What we do in ResearchExtensionEducation

Knowledge

Occurs when there is a change in behavior or theparticipantsact on whatthey have learned

Occurs when a societalconditionis improveddue toa participantaction takenin the previous column

ASSUMPTIONS EXTERNAL FACTORS

Water Quality Program: Logic Model

Inputs OutputsActivities Participants

Outcomes- Impact

Knowledge Actions Conditions

SITUATION

PRIORITIES

Staff

Money

Materials

Partners

On-farm visits

Set up record keeping systems to track phosphorus

Educational workshops

Farmers at risk of overfeeding phosphorus

Increased knowledge of link between cattle diet and water quality

Increased understanding of recommended phosphorus levels

Make appropriate adjustments to cattle feed

Monitor phosphorus levels in feed, manure, and soil

Improved water quality

Feed cost savings

Reductions in phosphorus use

Increased knowledge of tracking phosphorus levels

Were the inputs sufficient, timely? Did they meet the program goals? Were they sufficient given the situation?

Did all activities occur as intended? Quality of intervention: appropriate content; usable record keeping system; well planned visits?

Did the targeted farmers participate? Who did not? What were their reactions? Who else was reached?

Did knowledge increase? Did know-ledge of P levels increase? What else happened?

Are farmers monitoring P levels?

Are they adjusting cattle feed? Anything else?

Is there a reduction in P use?

How much $ saved? Other benefits? Negative consequences?

Water Quality Program: Performance Measurement, Evaluation Questions


Outcomes


Indicators: How will you know it?

# of staff; $ invested; delivery timetable

# of workshops data systems, on-farm visits, Farm self-reports of quality

Actual vs. desired attendance. #, % attended per session

#, % with increased knowledge of … Additional outcomes: + & -

#, % monitoring P levels; making adjustments to cattle feed Additional outcomes: + & -

#, % reducing P use; level of reduction. # dollars saved/farmer. Additional outcomes: + & -

Water Quality Program: Indicators


Outcomes


Uses of the Logic Model● Clarifies the linkages between activities, outputs and

expected outcomes of the policy, program or initiative

● Communicates externally about the rationale, activities and expected results of the policy, program or initiative;

Uses of the Logic Model● Tests whether the policy, program or initiative

"makes sense" from a logical perspective; and

● Provides the fundamental framework on which the performance measurement and evaluation strategies are based (i.e., determining what would constitute success).

Performance Measures vs. Performance Indicators

● Performance measures are conceptual, and need to be operationalized

● Performance indicators are operationalized, to specify actual data (qualitative or quantitative) to be collected and used

What is a Performance Indicator?

An indicator is a quantitative or qualitative amount used to determine the extent to which progress is made toward an outcome.

● Quantitative Example: percent of firms exporting 10 percent of their produce

● Qualitative Example: distance between mother and child pairs in video of play activity

Outputs

Products, services and events that are:● Intended to lead to outcomes; and ● Linked to problems or issues to be addressed –

through the logical or causal chain of events depicted in the model.

Performance MeasuresExamples -- Outputs

● Number of training sessions held● Number of participants trained● Number of instructional hours● Number of publications● Number of patents

OutcomesPlanned results or changes for individuals,

groups, communities, organizations or systems, including:

● Changes in knowledge;● Changes in behavior; and ● Changes in conditions (impacts) -- resulting in

solution of the original problem or issue.

Examples of Outcomes

● Knowledge: Change in level of knowledge regarding plant production

● Behaviors: Change in farming practices● Conditions: Increased food security

CSREES ExamplesPerformance Measures

and Indicators

Examples …

● Soybean Rust ● CEAP ● Cryptosporidium ● Nutrition and Obesity

● Family Financial Security

What are We Seeking?

Quantitative and qualitative evidence that● Researchers are identifying possible solutions to

national problems;● Educators and extension faculty and staff are

helping the public learn; and ● The public is applying new knowledge and

addressing national problems, thus● Producing changes in conditions.

Soybean Rust● Devastating disease worldwide – some regions

lost 60-80% of their soybean crop● First found in November, 2004 by an Extension

Specialist at Louisiana State University trained by a CSREES program, SPDN

● Rapid response by USDA agencies and Land Grant partners

● CSREES and its National Plant Diagnostic Network (NPDN), provided disease recognition and pathogen diagnostic tools.

● Helped save $11 to $299 million in 2005 …

Penn State Soybean Rust Model

SBR Observation – 12-14-2005

Soybean Rust in 2005

Zooming to a state brings up county level resolution and specific guidelines for that state.

Producer Access to Information

Soy Bean Rust Outputs

● Research papers modeling spread of soy bean rust;

● Training for extension agents in identifying and treating soy bean rust;

● Development of Internet Pest Information Platform.

Soy Bean Rust Output Indicators

● # of research papers modeling spread of soy bean rust;

● # of training hours for extension agents● # of participants trained● Completed development of Pest Information

Platform

Soy Bean Rust Outcomes

● Knowledge – producers aware of spread of soy bean rust and access Pest Information Platform for advice on treatment;

● Behavior – producers reduce fungicide use (cease treating in advance of spread);

● Conditions – crops appropriately treated, cost savings, less fungicide in ground water.

Indicators

Outcomes:● Knowledge – number of producer consultations

with Extension agents;● Behavior – amounts and locations of applied

fungicide; claims filed for crop damage ● Conditions – TBD – ERS model of cost savings

Water

● Basic need and pressing national problem● Directly related to public health, food safety, and other

critical public concerns ● Requires research, education, and extension ● Model exemplar for CSREES and the Land Grant

System results under the new reporting system?

“Surf your watershed…”

http://cfpub.epa.gov/surf/locate/map2.cfm

CSREES Actions

● Listening sessions ● Cross-agency planning ● Partnerships● CSREES White Papers● Coordinated programming ● Funding, including Formula and Competitive

Water Situation … ● Of 147 countries ranked for water efficiency, the

United States ranked last … (NCSE, 2004).● Growing populations and changing values have

increased demands, increasing water use and management conflicts.

● Agricultural needs conflict with urban needs and with demands to sustain or improve ecosystem services, recreation, and tourism.

● Overuse of water resources adversely affects water quality.

Research

Program Development

Education/Outreach

Policy Development

Sustainable Water Resources

Management

Water Users: Industrial, Ag

Sector, Rural/Urban

Communities

Existing WQNetwork

Desire for Formula Funds

Limited Human Resources

Status Quo

Funding

Global Visibility

Oversight

Behavioral Science

Biological Science

Physical Science

Water Education Foundations

Outreach and Demonstrations

Adoption Education K-12

Water Security

CEAP – Research Outputs Indicators

Publications Water quality and agricultural management

parameters and data linking water quality and agricultural management trends.

Land use/water quality model (AnnAGNPS) of Rock Creek.

CEAP – Research and Extension Outcomes

● Knowledge of best management practices for agricultural producers

● Actions: Agricultural producers adopt best management practices

● Conditions: Reduced contamination of water and food supply by livestock pathogens

CEAP – Research and Extension Outcome Indicators

● Knowledge: Feedback from seminars for producers (e.g., evaluations, surveys, etc.)

● Actions: Documentation of adoption of best practices

● Conditions: Fewer pathogens identified in surface and ground water, reduced number of contaminants found in water and food supply

CEAP, Outcomes, Conditions

Discovered increased dissolved phosphorus concentrations in Lake Erie tributaries, associated with conservation tillage -- enriching the surface layer of the soil in phosphorus, which then leaches out into the water.

Ohio EPA convened a phosphorus task force to examine the extent and severity of the issue, and impact on Lake Erie.

Water Quality Outcomes Conditions…

“… improve the quality of the Nation’s surface water and groundwater resources in agricultural, rural, and urbanizing watersheds.”

National Integrated Water Quality Program, FY 2007 Request for Applications

Cryptosporidium, Situation

Cryptosporidium is one of the most common causes of waterborne disease within humans in the United States

May be found in drinking water and recreational water in every region of the United States and throughout the world

Parasite may be carried by agricultural animals, wildlife, and humans

Cryptosporidium, Situation

In 1993, an estimated 403,000 residents of Milwaukee, Wisconsin became ill with cryposporidiosis

Ineffective filtration process led to the inadequate removal of Cryptosporidium oocysts from the municipal water supply

CDC investigators, in a retrospective cost-of-illness analysis, found the total cost of illness was $96.2 million: $31.7 million in medical costs and $64.6 million in productivity losses.

EPA now requires screening for cryptosporidium.

Cryptosporidium Activities

Research – genome sequencing; transmission, survival and spread of oocysts;

analysis of spread through water shed and into water supply.

Extension – methods to prevent transmission through agricultural animals into surface and ground water

Cryptosporidium, Research Outputs Developed a stochastic model of pathogen transport

over the soil surface Developed a deterministic model including

environmental parameters Compared modeled oocyst transport data to

experiments Applied engineering risk assessment to expand

model to analysis of an entire water supply consisting of multiple watersheds.

Cryptosporidium, Output Indicators Stochastic model itself Deterministic model, and its environmental

parameters Oocyst transport data, as predicted by models and

collected experimentally Risk assessments, parameters and data values

from multiple watersheds.

Cryptosporidium, Research Outcomes

Knowledge – Provided information on how environmental factors in a watershed affect transport of oocysts before they reach a water supply system.

Behavior – Enabled water treatment managers to better determine risks of pollution in extreme hydrological events

Conditions – TBD, Prevention of treatment failures, Prevention of outbreaks of cryptosporidiosis.

Cryptosporidium, Research Outcome Indicators Knowledge – Parameters and data on

environmental factors in watershed affecting transport of oocysts, as provided to agricultural producers and water treatment managers

Behavior – Changes in best management practices by producers and water treatment managers

Conditions – TBD, Cryptosporidium identified in watersheds, water supply

Cryptosporidium, Outcomes,Extension

Knowledge – Farmers and ranchers learn of best management practices (BMPs) for adequate set back distances and buffers to safeguard leafy greens from livestock pathogens and contaminated rangeland runoff

Behavior – Producers adopt BMPs, with cost-share programs as needed

Condition – Reduce pathogens in water supply

Cryptosporidium, Outcomes,Extension Indicators

Knowledge – Feedback from seminars for producers

Behavior – Documented adoption of BMPs, participation in cost-share programs

Condition – Reductions in pathogens found in in water supply, cost savings in water treatment

1995

Obesity Trends* Among U.S. AdultsBRFSS, 1990, 1995, 2005

(*BMI 30, or about 30 lbs overweight for 5’4” person)

2005

1990

No Data <10% 10%–14% 15%–19% 20%–24% 25%–29% ≥30%

Nutrition and Obesity Situation

Nutrition and Obesity

• Between 1970-1980 & 2003-2004, overweight among adolescents (12-19 years) increased from 5% to 17%.

Nutrition and Obesity Among Adolescents, Outputs

● Completed 1st study to apply dietary pattern methodology to systematically evaluate the diets of adolescents.

● Presentation: National Nutrient Databank Conference, Wash DC 2003

● Publication: Journal of Nutrition, Feb 2007

Nutrition and Obesity

Outcomes, Knowledge● Four distinct dietary patterns were identified.

Healthy pattern -- high intake of fruit, vegetables, dairy, grains & low intake of sweets and fried foods --was followed by 12% of White girls.

● Healthy pattern was associated with more favorable nutrient intakes & a smaller waist circumference.

● None of the dietary patterns followed mitigated adiposity (reduced body fat) among Black girls.

Nutrition and Obesity,Outcome Indicators

● Four distinct dietary patterns were identified among adolescent girls.

● Healthy pattern -- high intake of fruit, vegetables, dairy, grains & low intake of sweets and fried foods --was followed by 12% of White girls.

● Healthy pattern was associated with more favorable nutrient intakes & a smaller waist circumference.

● Among Black girls, none of the dietary patterns were associated with mitigating adiposity (increased body fat).

Nutrition and Obesity, Outcomes

● Food intake pattern consistent with guidance for healthy eating (like the USDA/HHS Dietary Guidelines for Americans) appears to help prevent overweight – among those who followed it in a longitudinal study.

● Only a minority of adolescent girls followed healthy food intake pattern.

Nutrition and Obesity, Outcomes, Behavior

● Food intake pattern consistent with guidance for healthy eating (like the USDA/HHS Dietary Guidelines for Americans) appears to help prevent overweight – among those who followed it in a longitudinal study.

● Only a minority of adolescent girls followed healthy food intake pattern.

Nutrition and Obesity, Outcomes, Conditions

● Results of this study can be used to set goals for effective obesity prevention strategies for adolescent girls.

● Reductions in adiposity reduce risks for heart disease, diabetes, etc. in later life.

● Reductions in health care costs associated with obesity.

Financial Security

Creating ability to meet future needs while keeping pace with day-to-day obligations.

Preparing for retirement and potential long-term care costs

Requiring planning, saving, and debt control.

Financial Security Outputs

Toolkit of web-based and face-to-face programs; Guidebook to Help Late Savers Prepare for

Retirement, Training of educators, and Development of eXtension program,

http://www.extension.org/personal+finance

Financial Security Outcome Indicators, Behavior

41% percent reported using one or more recommended financial management practices from an initiative program;

46% reported they developed plans to achieve retirement and/or future income goals; and

57% reported that they had increased their financial security.

Financial Security Outcome Indicators, Knowledge

89% of program participants increased their financial knowledge related to later life issues

64% planned to use recommended financial management practices

56% planned to manage their use of credit, reduce debt, and/or reduce household spending in light of their long-term goals for later life

Financial Security Outcome Indicators, Conditions

7,663 individuals reported a total of $6,501,945 in dollars saved, debt reduced, and/or new dollars invested--an average of $848 per person.

Developing and Using Performance

Indicators and Targets

Performance Measurement Framework Cycle (revisited)

1

0

2 3

4

5

6

78

SpecifyMeasures

EstablishData collectionProcedures

DevelopInformation System

Measure& Report Performance

Performance MeasurementUnderstanding

FormativeEvaluation

SummativeEvaluation

ProgramProfileLogic model

Review AndModify

FE

ED

BA

CK

Critical Steps –Using Indicators to Improve

Create a target performance level for each indicator Assign someone to that indicator Derive targets carefully Check indicators against targets periodically Anticipate shortfalls and correct them Shortfalls generally of two types –

Design flaws – redesign process? Execution issues -- training, better equipment, etc.

Performance Measure Example

Cumulative number of expanded commercially adaptable processes that convert biomass to fuels through the development of cost effective biochemical or thermochemical technologies, and used commercially

Performance Indicator Example

Long Term Outcome Measure -- Cumulative number of expanded commercially adaptable processes that convert biomass to fuels through the development of cost effective biochemical or thermochemical technologies, and used commercially

Performance Indicator Criteria

Precision – How well-defined is the indicator? Accuracy – Actual indicators are only

approximations of ideal measure, but how good an approximation?

Simplicity – Easily measured and interpreted? Cost Robustness (balance) – use multiple indicators,

especially for potentially competing criteria

Challenges and Trade-offs (1)

In the private sector, competing dimensions of performance to be captured by indicators –

Cost Quality Speed

Challenges and trade-offs (2)

In the public sector, competing dimensions of performance to be captured by indicators –

Adequacy Cost Equity Efficiency Effectiveness

Challenges and trade-offs (3)

For Federal programs, including CSREES, competing dimensions of performance to be captured by indicators –

• Relevance • Quality

• Performance

LOGIC MODEL

ELEMENT Indicator

Data Source/

Method

Used

Collected

By …

Timing/Frequency of Measurement

Ongoing Measure

Process

Evaluation

Outcome

Evaluation

Outputs Output 1

Output 2

Output X

Outcomes Knowledge Outcome 1

Knowledge Outcome 2

Behavior Outcome 1

Behavior Outcome 2

Condition Outcome 1

Condition Outcome 2

Condition Outcome X

Using and

Valuing Measures

Critical Steps – Using Performance Measures

Using PM to enhance learning and improve performance

Cycling between and among research, education, and extension

Better use of limited funding Stimulating new and better questions, approaches

to programs – and performance measurement and evaluation…

Multiple Uses of PM and Indicators

Communicating Success – ‘stories and statistics’ Modeling Performance monitoring, including indicators and/or

milestones Analysis and planning Evaluation (formative, definitely, and possibly

summative)

Rolling up results…

Strategic Objective

Intermediate Result 1

IntermediateResult 2

Sub IR 1.1 Sub IR 1.2

Activity B

Why? So What?

How?

Assuming What?

What Else? What Else?

Activity A Activity C

Cryptosporidium example…

Improve water quality

Farmers adopt BMPImproved ability

to detect and treat

Sub IR 1.1 Sub IR 1.2

Green payments

Why? So What?

How?

Assuming What?

What Else? What Else?

Farm demo Improved models

Wetland Restoration for Sediment Yield Reduction:A Conceptual Model

Benefit of Wetland Restoration for Sediment Yield Reduction

Marginal Change in Total Downstream Sediment Yield per Restoration Dollar

Marginal Change in Restored Wetland Area per Restoration Dollar

Marginal Change in Hydrologic Response per Restored Wetland

Area

Marginal Change in Sediment Yield per

Change in Hydrologic Response

Key

Concept

Vellidis et al., 2003. Prioritizing wetland restoration for sediment yield reduction: A conceptual model. Environmental Management 31(2): 301-312.




Marginal Change in Restored Wetland Area per Restoration Dollar

Community Willingness

Purchasing Efficiency

Restorability

Presence of: Watershed Protection Activities;

Conservation Programs

Watershed & environmental protection groups; conservation easements; protected areas

Restorability

Wetland characteristics; disturbance

Property Value

Land Cost

Key

Concept

Descriptor

Indicator

Measurement Point




Marginal Change in Hydrologic Response Wetland Area

Headwater Response

Floodplain Response

Watershed Yield; Runoff Delivery; Runoff

Interception

Precipitation; topography, permeability; hydrologic connectivity; flow friction

Upstream Velocity; Flood Frequency;

Interception

Stream order; return period; structural modifications; hydrologic connectivity

Key

Concept

Descriptor

Indicator

Measurement Point




Marginal Change in Hydrologic Response Wetland Area

Headwater Sediment Sources

Floodplain Sediment Sources

Upland Runoff; Unpaved Road Runoff;

Development Site Runoff

Land use; RUSLE; unpaved road density; conversion rate

Channel Sources; Upland Runoff; Unpaved Road Runoff;

Development Site Runoff

Total flow; bank materials; channel slope; contribution watershed area; sinuosity; land use; RUSLE; unpaved road density; conversion rate

Key

Concept

Descriptor

Indicator

Measurement Point

Summary

“Doing it right…”

Develop a logical/causal framework – model Gather data Turn data into information – use it for analysis Continually refine the model Base actions on findings Assess outcomes

Back to Accountability…

“… The old saw that one cannot be held responsible for what one does not fully control has outlived its usefulness; it is both reasonable and necessary to hold everyone with some influence over a metric [measure] jointly accountable for it.”

Next Steps?

● Questions? ● Where do we go from here? ● Increased communication… ● Increased synthesis…

Contact Information ● Sharon Stout, Social Scientist, OPA, CSREES 202-

401-0183, [email protected] ● Kitty Cardwell, [email protected]● Mike O’Neill, [email protected] ● Mary Ann Rozum, [email protected]● Etta Saltos, [email protected]● Jane Schuchardt, [email protected]

Office of Planning and Accountability Websitehttp://www.csrees.usda.gov/about/strat_plan.html

Performance Measures, Outcomes, and Impacts Sharon Stout Office of Planning and Accountability April...

Documents

Transcript of Performance Measures, Outcomes, and Impacts Sharon Stout Office of Planning and Accountability April...