1 Creating Strong Reports New Mexico AmeriCorps April 20, 2006 Sue Hyatt, Project STAR Coach.
-
Upload
kaylee-bradshaw -
Category
Documents
-
view
213 -
download
0
Transcript of 1 Creating Strong Reports New Mexico AmeriCorps April 20, 2006 Sue Hyatt, Project STAR Coach.
11
Creating Strong Reports
New Mexico AmeriCorpsApril 20, 2006
Sue Hyatt, Project STAR Coach
22
AgendaAgenda
Learn some “do’s and don’ts” Learn some “do’s and don’ts” forforperformance measures (PM’s) in performance measures (PM’s) in the APR.the APR.
Reporting TipsReporting Tips
Review of Your Most Recent Review of Your Most Recent ReportReport
33
Project STAR Findings:Common PM and Reporting
Issues
Outputs versus OutcomesOutputs versus Outcomes Intermediate versus End OutcomesIntermediate versus End Outcomes ““Creative Writing” - Indicators and Creative Writing” - Indicators and
TargetsTargets Targets Not Related to InstrumentsTargets Not Related to Instruments Weak Community Need StatementsWeak Community Need Statements
44
REVIEW: Types of Performance Measures OutputsOutputs Intermediate OutcomesIntermediate Outcomes End OutcomesEnd Outcomes
55
OutputsOutputs
Are counts of the amount of services Are counts of the amount of services delivered, work, or products created that delivered, work, or products created that members or volunteers have completed.members or volunteers have completed.
Usually number of people servedUsually number of people served
Do Do notnot answer the question, “what answer the question, “what changed as a result of our work?” changed as a result of our work?”
Do Do notnot provide information on benefits or provide information on benefits or other changes in the lives of beneficiaries.other changes in the lives of beneficiaries.
Examples:Examples:30 students tutored in reading100 tress planted25 houses built benefiting 25 families
66
End OutcomesEnd Outcomes
Answer the question, “So what?”Answer the question, “So what?”
Changes that have occurred in the Changes that have occurred in the lives of beneficiaries that are lives of beneficiaries that are significantsignificantExamples:Examples:Students will not engage in drug or alcohol useIncreasing homeownership among clientsStudents demonstrate better in school behavior
77
Intermediate Intermediate Outcomes Outcomes
Answer the question, “what difference did it Answer the question, “what difference did it make”?make”?Do Do notnot represent the final result you hope to represent the final result you hope to achieve for beneficiariesachieve for beneficiariesLikely preconditions for longer-term gainsLikely preconditions for longer-term gains
When positive are usually a sign that your project When positive are usually a sign that your project is on track to achieve the related end outcomes.is on track to achieve the related end outcomes.
Examples:Examples:
Students report feeling safe and not pressured to engage in Students report feeling safe and not pressured to engage in risky behavior.risky behavior.
Decrease in volunteer attrition.Decrease in volunteer attrition.Students will complete homework assignments.Students will complete homework assignments.
88
Possible Beneficiary Possible Beneficiary ChangesChanges
Outputs:Outputs: Number of People Served Number of People Served Outcomes:Outcomes: How People Changed How People Changed
KnowledgeKnowledge
AttitudesAttitudes
Behavior or SkillsBehavior or Skills
IntermediateIntermediate EndEnd
99
What is an Indicator?What is an Indicator?
Information you collect to determineInformation you collect to determineprogress toward achieving resultsprogress toward achieving results
Concrete and measurableConcrete and measurable Answers the question, “What is myAnswers the question, “What is my
evidence that something has occurred?”that something has occurred?” Output indicator usually begins with Output indicator usually begins with
“Number of...”“Number of...” Outcome indicator usually begins with Outcome indicator usually begins with
“Percent of...”“Percent of...”
1010
Internal Alignment: End Outcome
Members will provide 1 on 1 reading tutoring to third graders at least 3 times a week for 20
minutes/session End Outcome Result: Students have End Outcome Result: Students have
improved proficiency in readingimproved proficiency in reading Indicator: Percentage of students tutored Indicator: Percentage of students tutored
who have improved proficiency in readingwho have improved proficiency in reading Target: 75% (263 of 350) students Target: 75% (263 of 350) students
tutored will have improved proficiency in tutored will have improved proficiency in readingreading
Instrument: Reading GradesInstrument: Reading Grades
1111
Internal Alignment: End Outcome (NOT!)
Members will provide 1 on 1 reading tutoring to third graders at least 3 times a week for 20
minutes/session End Outcome Result: Students have End Outcome Result: Students have
improved proficiency in readingimproved proficiency in reading Indicator: Teachers will report that 75% Indicator: Teachers will report that 75%
students have completed more reading students have completed more reading homework.homework.
Target: Students tutored will show Target: Students tutored will show improved behavior in the classroom and improved behavior in the classroom and be sent to the Principal’s office less be sent to the Principal’s office less often.often.
Instrument: logs and surveysInstrument: logs and surveys
1212
APR Do’s and Don’ts:Do create performance
measurement plans
1313
What is a Performance Measurement What is a Performance Measurement Plan?Plan?
Written description of steps you will take to obtain data to measure outputs and outcomes
The PM plan specifies:• Who will do each step• When they will do it
Also known as data collection plan PM plan is instrument-focused The PM plan is your roadmap during the
program year. It helps you obtain reliable, useful information about the effect of your program on beneficiaries.
1414
Performance Measurement Plans Performance Measurement Plans (continued)(continued)
1515
Performance Measurement Plans Performance Measurement Plans (continued)(continued)
1616
APR Do’s and Don’ts:Do include strong descriptions
of service activities
1717
APR Do’s and Don’ts:Do clearly identify instruments
1818
APR Do’s and Don’ts:APR Do’s and Don’ts:DoDo talk about talk about changechange in outcome in outcome
resultsresults
1919
APR Do’s and Don’ts:Don’t get carried away with
output codes
2020
Why Do Progress Reports?Why Do Progress Reports?
Enhances program accountabilityEnhances program accountability Opportunity for reflection and Opportunity for reflection and
strategic thinking—see the forest strategic thinking—see the forest for the treesfor the trees
Helps you become a “great Helps you become a “great communicator”—the more youcommunicator”—the more youdo it, the better you get at it.do it, the better you get at it.
Program staff turnover: Progress Program staff turnover: Progress report is a report is a permanent permanent document.document.
Helpful in identifying TA needs.Helpful in identifying TA needs.
2121
Calculating Targets: Calculating Targets: Who Should I Report Who Should I Report On?On?
Entire “needy
population”
Your beneficiarie
s
Beneficiaries you
measured
“Successful” beneficiaries
2222
Example of Target CalculationExample of Target Calculation
““Successful” Successful” Beneficiaries You Beneficiaries You
MeasuredMeasured
Beneficiaries You Beneficiaries You MeasuredMeasured
287287
428428
TargetTarget: : Sixty percent of neighborhood Sixty percent of neighborhood residents will report an increase in recycling.residents will report an increase in recycling.
67%67%
2323
Completing theCompleting theAPR Progress ReportAPR Progress Report
2424
Well-written Progress Well-written Progress Reports Answer These Reports Answer These
Questions:Questions:
How much work is complete?How much work is complete? What work is currently in What work is currently in
progress?progress? What work hasn’t started yet?What work hasn’t started yet? What challenges have arisen?What challenges have arisen? What steps are being taken to What steps are being taken to
address these challenges?address these challenges?
These same questions apply to reporting on performance
measurement!
2525
Describing Your Work Describing Your Work Before Results Are Before Results Are
AvailableAvailable
“...We don’t have any results yet.”
2626
Describing Your Work Describing Your Work Before Results Are Before Results Are
AvailableAvailable Report with respect to:Report with respect to:
– Developing logic models and Developing logic models and indicatorsindicators
– Refining data collection plansRefining data collection plans– Data access/confidentiality (human Data access/confidentiality (human
subjects)subjects)– Developing and pilot testing Developing and pilot testing
instruments instruments – Training data collectorsTraining data collectors
2727
Describing Your Describing Your ResultsResults
Tell the reader:Tell the reader:– How and when data were collectedHow and when data were collected– How data were analyzedHow data were analyzed
Account for any data that were Account for any data that were inaccessible or unusable.inaccessible or unusable.
Include numbers, not just Include numbers, not just percentages.percentages.
Don’t delay analyzing and reporting Don’t delay analyzing and reporting baseline data!baseline data!
2828
Describing Challenges & Describing Challenges & SolutionsSolutions
Worst thing to report: “No challenges”Worst thing to report: “No challenges”
Explain how challenges affect program Explain how challenges affect program operation.operation.
Report steps taken to address challenges.Report steps taken to address challenges.
Always have a backup plan (in case your Always have a backup plan (in case your proposed solution doesn’t work)proposed solution doesn’t work)
Bring past challenges to closureBring past challenges to closure
2929
ExerciseExercise
Review Your Recent Progress Review Your Recent Progress Report and Note What You Want Report and Note What You Want to Remember to Upgrade Next to Remember to Upgrade Next
Time You Do a ReportTime You Do a Report
3030
Additional Additional Resources::
Contact Project STARContact Project STAR• http://nationalserviceresources.org//siteshttp://nationalserviceresources.org//sites
/star//star/
• 1-800-548-36561-800-548-3656• [email protected]@aiweb.com