Presenter: Michael Thaler 21 st CCLC State Program Manager [email protected]
description
Transcript of Presenter: Michael Thaler 21 st CCLC State Program Manager [email protected]
![Page 1: Presenter: Michael Thaler 21 st CCLC State Program Manager mthaler@doe.k12.ga](https://reader036.fdocuments.in/reader036/viewer/2022062520/568159e9550346895dc7366a/html5/thumbnails/1.jpg)
Dr. John D. Barge, State School Superintendent“Making Education Work for All Georgians”www.gadoe.org
Evaluating your 21st CCLC Program and Lessons Learned by Veteran
Program DirectorsFY13 Program Training
August 7, 2012
Presenter: Michael Thaler21st CCLC State Program [email protected](404) 232-1197
![Page 2: Presenter: Michael Thaler 21 st CCLC State Program Manager mthaler@doe.k12.ga](https://reader036.fdocuments.in/reader036/viewer/2022062520/568159e9550346895dc7366a/html5/thumbnails/2.jpg)
Dr. John D. Barge, State School Superintendent“Making Education Work for All Georgians”www.gadoe.org
Goals of this Presentation
• Understand the importance of program evaluation• Discuss the components of the evaluation reports• Understand the roles and expectations of an external
evaluator• Learn about tools other 21st CCLC programs are using• Receive tips and knowledge that our most veteran
program directors wish they had known their first year
![Page 3: Presenter: Michael Thaler 21 st CCLC State Program Manager mthaler@doe.k12.ga](https://reader036.fdocuments.in/reader036/viewer/2022062520/568159e9550346895dc7366a/html5/thumbnails/3.jpg)
Dr. John D. Barge, State School Superintendent“Making Education Work for All Georgians”www.gadoe.org
“We are doing a great job of running our programs and we are effective, but we aren’t
telling our story and that’s hurting us.”
Dr. Sylvia LylesBeyond School Hours XIII Conference February, 2010
![Page 4: Presenter: Michael Thaler 21 st CCLC State Program Manager mthaler@doe.k12.ga](https://reader036.fdocuments.in/reader036/viewer/2022062520/568159e9550346895dc7366a/html5/thumbnails/4.jpg)
Dr. John D. Barge, State School Superintendent“Making Education Work for All Georgians”www.gadoe.org
Perception vs. Reality• Perception
- Daycare centers- Unorganized or ineffective activities- Minimal impact on schools, families and community- Adult employment agency
• Reality- Highly organized out-of-school programs- Utilize research-based activities that specifically
target ‘at-risk’ or ‘high opportunity’ students- Provide critical services to a number of stakeholders
![Page 5: Presenter: Michael Thaler 21 st CCLC State Program Manager mthaler@doe.k12.ga](https://reader036.fdocuments.in/reader036/viewer/2022062520/568159e9550346895dc7366a/html5/thumbnails/5.jpg)
Dr. John D. Barge, State School Superintendent“Making Education Work for All Georgians”www.gadoe.org
Our job is to tell the story of our 21st CCLC programs to our various stakeholders so we can change their perception and help educate them about positive impact of our programs. This is accomplished by developing and utilizing an effective evaluation plan and following the
principles of effectiveness.
![Page 6: Presenter: Michael Thaler 21 st CCLC State Program Manager mthaler@doe.k12.ga](https://reader036.fdocuments.in/reader036/viewer/2022062520/568159e9550346895dc7366a/html5/thumbnails/6.jpg)
Dr. John D. Barge, State School Superintendent“Making Education Work for All Georgians”www.gadoe.org
Why should you do it?
Principles of Effectiveness (Sec. 4205 (b)) states ‘the program or activity shall undergo a periodic evaluation to assess its progress toward achieving its goal of providing high quality opportunities for academic enrichment’
![Page 7: Presenter: Michael Thaler 21 st CCLC State Program Manager mthaler@doe.k12.ga](https://reader036.fdocuments.in/reader036/viewer/2022062520/568159e9550346895dc7366a/html5/thumbnails/7.jpg)
Dr. John D. Barge, State School Superintendent“Making Education Work for All Georgians”www.gadoe.org
Principles of Effectiveness (Sec. 4205 (b))
The program or activity should be based upon:1. An assessment of objective data
regarding the need for the program2. An established set of performance
measures3. Scientifically-based research that
demonstrates the activity will help the students meet State or local standards
![Page 8: Presenter: Michael Thaler 21 st CCLC State Program Manager mthaler@doe.k12.ga](https://reader036.fdocuments.in/reader036/viewer/2022062520/568159e9550346895dc7366a/html5/thumbnails/8.jpg)
Dr. John D. Barge, State School Superintendent“Making Education Work for All Georgians”www.gadoe.org
What can evaluations provide?
• In addition to letting you know how you are doing, effective evaluation plans can help:– Increase student and parent participation– Increase school support– Obtain additional partnerships– Obtain additional funding– Increase the effectiveness of your program
![Page 9: Presenter: Michael Thaler 21 st CCLC State Program Manager mthaler@doe.k12.ga](https://reader036.fdocuments.in/reader036/viewer/2022062520/568159e9550346895dc7366a/html5/thumbnails/9.jpg)
Dr. John D. Barge, State School Superintendent“Making Education Work for All Georgians”www.gadoe.org
Essential Steps for an Effective Evaluation Plan
1. Determine the needs of your students and families2. Establish clear goals, objectives, and activities based
on the identified needs3. Develop the timeline for the evaluation plan4. Collect and analyze baseline, formative, and
summative data5. Disseminate the evaluation results to your
stakeholders6. Refine the program and its activities for next year,
based upon the evaluation results
![Page 10: Presenter: Michael Thaler 21 st CCLC State Program Manager mthaler@doe.k12.ga](https://reader036.fdocuments.in/reader036/viewer/2022062520/568159e9550346895dc7366a/html5/thumbnails/10.jpg)
Dr. John D. Barge, State School Superintendent“Making Education Work for All Georgians”www.gadoe.org
Evaluation Requirements
• 3 Major Components:- Formative (mid-year) Assessment- Summative (end of year) Assessment- Cayen/Afterschool 21 data entry system
• Summer Evaluation, if your program will run one in 2013
![Page 11: Presenter: Michael Thaler 21 st CCLC State Program Manager mthaler@doe.k12.ga](https://reader036.fdocuments.in/reader036/viewer/2022062520/568159e9550346895dc7366a/html5/thumbnails/11.jpg)
Dr. John D. Barge, State School Superintendent“Making Education Work for All Georgians”www.gadoe.org
Formative Assessment• Must undergo at least one and it must be written by the external evaluator• Evaluates from the beginning of the program year to the time of the evaluation• At a minimum, the report must include:
– Student Attendance: Data on average daily attendance and enrollment at each site;
– Program Operation: Synopsis of current level of operation at each site;– Objective Assessment: Data analysis and indication of progress towards
achieving EACH objective (as required, all objectives must include measures that allow for continuous (formative) assessment); and
– Recommendations: Recommendations for programmatic refinement for all objectives where progress is not being achieved and/or where the program is not likely to achieve the stated objective by the end of the program year.
![Page 12: Presenter: Michael Thaler 21 st CCLC State Program Manager mthaler@doe.k12.ga](https://reader036.fdocuments.in/reader036/viewer/2022062520/568159e9550346895dc7366a/html5/thumbnails/12.jpg)
Dr. John D. Barge, State School Superintendent“Making Education Work for All Georgians”www.gadoe.org
Formative Assessment
• Due February 1, 2013• Important to continuously assess your
program throughout the year (i.e., grades, attendance)
• Effective programs make revisions throughout, based upon their formative assessments
• Share the results with stakeholders
![Page 13: Presenter: Michael Thaler 21 st CCLC State Program Manager mthaler@doe.k12.ga](https://reader036.fdocuments.in/reader036/viewer/2022062520/568159e9550346895dc7366a/html5/thumbnails/13.jpg)
Dr. John D. Barge, State School Superintendent“Making Education Work for All Georgians”www.gadoe.org
![Page 14: Presenter: Michael Thaler 21 st CCLC State Program Manager mthaler@doe.k12.ga](https://reader036.fdocuments.in/reader036/viewer/2022062520/568159e9550346895dc7366a/html5/thumbnails/14.jpg)
Dr. John D. Barge, State School Superintendent“Making Education Work for All Georgians”www.gadoe.org
![Page 15: Presenter: Michael Thaler 21 st CCLC State Program Manager mthaler@doe.k12.ga](https://reader036.fdocuments.in/reader036/viewer/2022062520/568159e9550346895dc7366a/html5/thumbnails/15.jpg)
Dr. John D. Barge, State School Superintendent“Making Education Work for All Georgians”www.gadoe.org
Summative Assessment
• Each program must submit a summative report by June 28th, 2012
• Evaluates the entire program year• The results of your summative report will be used to
help determine continuation of funding in subsequent years
• Must be written by your external evaluator and must follow the template on the following slides
![Page 16: Presenter: Michael Thaler 21 st CCLC State Program Manager mthaler@doe.k12.ga](https://reader036.fdocuments.in/reader036/viewer/2022062520/568159e9550346895dc7366a/html5/thumbnails/16.jpg)
Dr. John D. Barge, State School Superintendent“Making Education Work for All Georgians”www.gadoe.org
Summative Report Template• Each summative report must include, at minimum, the following major sections: • Overview and History: Provide a general overview of the 21st CCLC program,
including a history of previous operation and how the program has progressed and grown over the past year(s) of operation.
• Student Attendance and Enrollment: Total and Regularly-Participating Student Enrollment (per site); Student Demographics (e.g., gender, ethnicity); Average Daily Attendance (per site)
• Program Operation: The summative report must include information on operation for EACH site within the program (e.g., days, hours, and weeks of operation for each component).
• Quality of Staffing: The composition of site staff is one method for assessing the quality and breadth of 21st CCLC programs. For each site within the program, the summative report must include information about staffing (e.g., Staff Demographics, Student: Staff Ratio, Staff Training, and Use of Certified Teachers).
![Page 17: Presenter: Michael Thaler 21 st CCLC State Program Manager mthaler@doe.k12.ga](https://reader036.fdocuments.in/reader036/viewer/2022062520/568159e9550346895dc7366a/html5/thumbnails/17.jpg)
Dr. John D. Barge, State School Superintendent“Making Education Work for All Georgians”www.gadoe.org
Summative Report Template• Objective Assessment: Within the grant application, 21st CCLC programs
proposed both objectives and methods of evaluating progress towards achieving the objectives. The summative report must include detailed information for EACH OBJECTIVE approved by GaDOE (e.g., activities, data collected, timeline, analysis and results). Programs are NOT permitted to revise, reword, or otherwise change their objectives without specific written approval from GaDOE.
• Other Observations (Optional): If appropriate, provide other relevant findings pertaining to this 21st CCLC program. Other findings could include qualitative and/or quantitative data not related to specific objectives; quotes and/or statements from students, parents, and/or teachers; success stories of students within the program; and photographs demonstrating unique program activities and services.
![Page 18: Presenter: Michael Thaler 21 st CCLC State Program Manager mthaler@doe.k12.ga](https://reader036.fdocuments.in/reader036/viewer/2022062520/568159e9550346895dc7366a/html5/thumbnails/18.jpg)
Dr. John D. Barge, State School Superintendent“Making Education Work for All Georgians”www.gadoe.org
Summative Report Template
• Progress towards Sustainability: The summative report must provide information about program partnerships (e.g., partnership development, partner maintenance, and contributions to the program).
• Overall Recommendations: Provide an overall assessment of this 21st CCLC program, as well as any program-wide recommendations to enhance program quality.
![Page 19: Presenter: Michael Thaler 21 st CCLC State Program Manager mthaler@doe.k12.ga](https://reader036.fdocuments.in/reader036/viewer/2022062520/568159e9550346895dc7366a/html5/thumbnails/19.jpg)
Dr. John D. Barge, State School Superintendent“Making Education Work for All Georgians”www.gadoe.org
Major Outcomes/GaDOE’s Expectations
• GaDOE expects every 21st CCLC program to demonstrate the following:– Increase in grades from fall to spring (no less than 85% of
student grades entered)– Increase in CRCT/EOCT scores (no less than 85% of students
with scores in tested grade levels)
– Increase in school day and program attendance– Improvement in areas such as discipline, homework
completion, parent involvement, parent/student satisfaction, etc.
![Page 20: Presenter: Michael Thaler 21 st CCLC State Program Manager mthaler@doe.k12.ga](https://reader036.fdocuments.in/reader036/viewer/2022062520/568159e9550346895dc7366a/html5/thumbnails/20.jpg)
Dr. John D. Barge, State School Superintendent“Making Education Work for All Georgians”www.gadoe.org
Cayen/ Afterschool 21 Data Entry System
• Programs must enter data (grades, attendance, activities, etc.) into this system continuously throughout the life of the grant
• GaDOE uses this to fulfill its requirement of reporting data to US ED
• All new subgrantees must attend webinars and work closely with their assigned AS21 contractor
![Page 21: Presenter: Michael Thaler 21 st CCLC State Program Manager mthaler@doe.k12.ga](https://reader036.fdocuments.in/reader036/viewer/2022062520/568159e9550346895dc7366a/html5/thumbnails/21.jpg)
Dr. John D. Barge, State School Superintendent“Making Education Work for All Georgians”www.gadoe.org
External EvaluatorProvides a critical service to our programs and should be an experienced
professionalContact multiple evaluators and ‘shop around’
- 21st CCLC experience?- Speak with some of their current programs and see if they will provide you with one of the reports
- Ensure you hire one early in the program yearCan only use 3% of your budget for this and must enter in to one year
contractsCannot be the person who wrote the grantThe quality of your evaluator can have a significant impact on the quality
your program!
![Page 22: Presenter: Michael Thaler 21 st CCLC State Program Manager mthaler@doe.k12.ga](https://reader036.fdocuments.in/reader036/viewer/2022062520/568159e9550346895dc7366a/html5/thumbnails/22.jpg)
Dr. John D. Barge, State School Superintendent“Making Education Work for All Georgians”www.gadoe.org
Role of an External EvaluatorRemember - They work for you, you don’t work for them
They are there to provide guidance and support, but ultimately the success of the program relies on the program administration
They are somewhat limited by the quality of data you provide to them
Refer to the operations manual for more information requirements/recommendations for the external evaluator
![Page 23: Presenter: Michael Thaler 21 st CCLC State Program Manager mthaler@doe.k12.ga](https://reader036.fdocuments.in/reader036/viewer/2022062520/568159e9550346895dc7366a/html5/thumbnails/23.jpg)
Dr. John D. Barge, State School Superintendent“Making Education Work for All Georgians”www.gadoe.org
What must they provide?
– At a minimum:
- One formative assessment
- Summative assessment after the conclusion of the program year
- A list of recommended revisions based upon the results of the formative and summative assessments
![Page 24: Presenter: Michael Thaler 21 st CCLC State Program Manager mthaler@doe.k12.ga](https://reader036.fdocuments.in/reader036/viewer/2022062520/568159e9550346895dc7366a/html5/thumbnails/24.jpg)
Dr. John D. Barge, State School Superintendent“Making Education Work for All Georgians”www.gadoe.org
Suggestions: What to Request
• Conduct site visits at least twice a year– This can vary depending on the size of your grant or
location of your evaluator
• Assist with developing surveys and other measurement instruments
• Assist with the dissemination of evaluation results– i.e., One page flyer and/or brief PowerPoint presentation
![Page 25: Presenter: Michael Thaler 21 st CCLC State Program Manager mthaler@doe.k12.ga](https://reader036.fdocuments.in/reader036/viewer/2022062520/568159e9550346895dc7366a/html5/thumbnails/25.jpg)
Dr. John D. Barge, State School Superintendent“Making Education Work for All Georgians”www.gadoe.org
Suggestions: What to Request
• Participate and present at school/advisory/stakeholder meetings
• Conduct focus groups with students, staff, and/or parents
• Provide input in revising objectives and assessments
• Monthly meetings or teleconferences to ensure the evaluation plan is on track
![Page 26: Presenter: Michael Thaler 21 st CCLC State Program Manager mthaler@doe.k12.ga](https://reader036.fdocuments.in/reader036/viewer/2022062520/568159e9550346895dc7366a/html5/thumbnails/26.jpg)
Dr. John D. Barge, State School Superintendent“Making Education Work for All Georgians”www.gadoe.org
Helpful Hints
• Set up your contract so your payments align with deliverables, not just timeframes
• Clearly explain to them the evaluation requirements
• If possible, bring them to state/national conferences and state meetings
![Page 27: Presenter: Michael Thaler 21 st CCLC State Program Manager mthaler@doe.k12.ga](https://reader036.fdocuments.in/reader036/viewer/2022062520/568159e9550346895dc7366a/html5/thumbnails/27.jpg)
Dr. John D. Barge, State School Superintendent“Making Education Work for All Georgians”www.gadoe.org
Other Helpful Hints
• Disaggregate the results of your summative evaluation based on number of days attended:
• 0-29 days• 30-59 days• 60-89 days• 90+ days• Compare regularly participating students (30+ days) to non-
21st CCLC students• Track students for the life of your grant
![Page 28: Presenter: Michael Thaler 21 st CCLC State Program Manager mthaler@doe.k12.ga](https://reader036.fdocuments.in/reader036/viewer/2022062520/568159e9550346895dc7366a/html5/thumbnails/28.jpg)
Dr. John D. Barge, State School Superintendent“Making Education Work for All Georgians”www.gadoe.org
Additional Evaluation Tools
• Needs Assessments or Interest Surveys– Students - determine activities that would interest them– Consider pre and post surveys– Staff - determine the type of training that would benefit
them the most (example)– Staff - determine which types of enrichment activities they
would like to teach– Parents - not only what type of activities, but when and
where to hold them
![Page 29: Presenter: Michael Thaler 21 st CCLC State Program Manager mthaler@doe.k12.ga](https://reader036.fdocuments.in/reader036/viewer/2022062520/568159e9550346895dc7366a/html5/thumbnails/29.jpg)
Dr. John D. Barge, State School Superintendent“Making Education Work for All Georgians”www.gadoe.org
Professional Development Activity VI SI NI1. Transition school-day activities to After-School Programactivities2. Integrate related reading and math learning activities for enrichment and non-remediation classes.3. Evaluate and document learning in enrichment and non-remediation classes4. Develop course outlines and weekly lessons in accordance with program objectives5. Identify what teaching techniques promote learning in the After-School Program classroom. 6. Develop additional enrichment activities 7. Promote parental involvement and determine activities to keep them coming back 8. Discover supplemental teaching materials that work 9. Develop/refine classroom management skills for After-School Program10. Maintain a safe, secure classroom environment while promoting learning and creative expression11. Understand the influence of school, family and peers with delinquency12. Gain national, state and DJJ perspectives of After-School Programs13. How is the After-School Evaluated?14. Time to share After-School Program experiences with other after-school teachers15. How to get more community participation for speakers, classroom supplies, etc
DJJ AFTER-SCHOOL PROGRAMPROFESSIONAL DEVELOPMENT SURVEYINTRODUCTION: This professional development survey was developed for DJJ After-School Program teachers. Survey results will help determine future professional development activities. Read each statement carefully and place a check mark in the appropriate space that best represents your answer for Very Interested (VI), Somewhat Interested (SI) or Not Interested.
Comments Suggestions: ____________________________________________________________________________________________________________
![Page 30: Presenter: Michael Thaler 21 st CCLC State Program Manager mthaler@doe.k12.ga](https://reader036.fdocuments.in/reader036/viewer/2022062520/568159e9550346895dc7366a/html5/thumbnails/30.jpg)
Dr. John D. Barge, State School Superintendent“Making Education Work for All Georgians”www.gadoe.org
![Page 31: Presenter: Michael Thaler 21 st CCLC State Program Manager mthaler@doe.k12.ga](https://reader036.fdocuments.in/reader036/viewer/2022062520/568159e9550346895dc7366a/html5/thumbnails/31.jpg)
Dr. John D. Barge, State School Superintendent“Making Education Work for All Georgians”www.gadoe.org
Additional Evaluation Tools
• Student progress reports
• Classroom/teacher observation forms
• Parent satisfaction survey
![Page 32: Presenter: Michael Thaler 21 st CCLC State Program Manager mthaler@doe.k12.ga](https://reader036.fdocuments.in/reader036/viewer/2022062520/568159e9550346895dc7366a/html5/thumbnails/32.jpg)
Dr. John D. Barge, State School Superintendent“Making Education Work for All Georgians”www.gadoe.org
![Page 33: Presenter: Michael Thaler 21 st CCLC State Program Manager mthaler@doe.k12.ga](https://reader036.fdocuments.in/reader036/viewer/2022062520/568159e9550346895dc7366a/html5/thumbnails/33.jpg)
Dr. John D. Barge, State School Superintendent“Making Education Work for All Georgians”www.gadoe.org
![Page 34: Presenter: Michael Thaler 21 st CCLC State Program Manager mthaler@doe.k12.ga](https://reader036.fdocuments.in/reader036/viewer/2022062520/568159e9550346895dc7366a/html5/thumbnails/34.jpg)
Dr. John D. Barge, State School Superintendent“Making Education Work for All Georgians”www.gadoe.org
![Page 35: Presenter: Michael Thaler 21 st CCLC State Program Manager mthaler@doe.k12.ga](https://reader036.fdocuments.in/reader036/viewer/2022062520/568159e9550346895dc7366a/html5/thumbnails/35.jpg)
Dr. John D. Barge, State School Superintendent“Making Education Work for All Georgians”www.gadoe.org
![Page 36: Presenter: Michael Thaler 21 st CCLC State Program Manager mthaler@doe.k12.ga](https://reader036.fdocuments.in/reader036/viewer/2022062520/568159e9550346895dc7366a/html5/thumbnails/36.jpg)
Dr. John D. Barge, State School Superintendent“Making Education Work for All Georgians”www.gadoe.org
Lessons Learned by Veteran 21st CCLC Program Directors
• I asked some of our most successful and experienced 21st CCLC program directors to come up with a list of tips and pieces of knowledge they wish they knew their first year
• Review of Lessons Learned handout– This is GOLD
36
![Page 37: Presenter: Michael Thaler 21 st CCLC State Program Manager mthaler@doe.k12.ga](https://reader036.fdocuments.in/reader036/viewer/2022062520/568159e9550346895dc7366a/html5/thumbnails/37.jpg)
Dr. John D. Barge, State School Superintendent“Making Education Work for All Georgians”www.gadoe.org
Any Questions?
• The biggest thing to remember is you have a support system of GaDOE staff and other subgrantees to help you through this process.