Copyright © 2011

American Institutes

for Research

All rights reserved.

Oregon 21st CCLC Oregon 21st CCLC Leading IndicatorsLeading IndicatorsBreakout SessionBreakout Session

Neil Naftzger and Deborah Moroney

October 21, 2011

2

AgendaAgenda

• Review purposes and goals of the Leading Indicator system

• Grantee review and feedback on the Leading Indicators

• Outline next steps

3

The Fine PrintThe Fine Print

• Primary purpose of the leading indicator system is to support program improvement

• The leading indicators are not meant to be an accountability system

• Leading indicators are aligned with ODE-defined goals for the 21st CCLC program

• Predicated on (a) research-supported best practices and (b) practices that are theoretically linked to the achievement of desired outcomes

• Leading indicator data needs to be used in a fashion at the grantee and center level which considers local context

• The primary data source used to support the Leading Indicators is the Site Coordinator survey

4

Key QuestionsKey Questions

How do we know if the indicators accurately describe the quality of your program?

? Is this Indicator understandable and interpretable?

? Does this Indicator convey meaningful information?

? Will this Indicator support discussions and conversations with 21st CCLC staff?

5

What We Know About What We Know About Program QualityProgram Quality

Collaboration & Partnerships (inputs) +

Quality Staff w/ Diverse Skills (inputs) +

Intentional Programming (activities) +

Attendance & Engagement (outputs) =

Positive Youth OutcomesPositive Youth Outcomes

6

Collaboration Collaboration (Inputs)(Inputs)

LI: Partner engagement – Partners associated with the center are actively involved in planning, decision making, evaluating, and supporting the operations of the afterschool program.

LI: Partner engagement – Partners associated with the center are actively involved in planning, decision making, evaluating, and supporting the operations of the afterschool program.

LI: Staff at the center will be engaged in intentional efforts to collaborate and communicate frequently about ways to improve program quality.

LI: Steps are taken by the center to establish linkages to the school day and use data on student academic achievement to inform programming

7

Great Staff (Inputs)Great Staff (Inputs)

LI: Staff at the center are provided with training and/or professional development.

LI: Staff at the center complete one or more self-assessment during the programming period.

LI: Staff at the center are periodically evaluated/assessed during the program period.

8

Intentional ActivitiesIntentional Activities+Students+

LI: There is evidence of alignment between (a) program objectives relative to supporting youth development, (b) student needs, and (c) program philosophy/model AND frequency/extent to which key opportunities and supports are provided to youth.

LI: There is evidence of alignment between(a) program objectives relative to the academic development of students, (b) student needs, and (c) program philosophy/model AND activities being provided at the center.

LI: Intentionality in activity and session design among staff responsible for the delivery of activities meant to support student growth and development in mathematics and reading/language arts.

9

Intentional Activities Intentional Activities +Families+

LI: Steps are taken by the center to reach out and communicate with parents and adult family members of participating students.

LI: There is evidence of alignment between (a) program objectives relative to supporting family literacy and related development, (b) family needs, and(c) program philosophy/ model AND activities being provided at the center.

10

Outputs & OutcomesOutputs & OutcomesOutputs• Attendance • Program Hours• Program Offerings

Outcomes• Academic Success• Social and Emotional Development

11

Key QuestionsKey Questions

How do we know if the indicators accurately describe the quality of your program?

+ Is this Indicator understandable and interpretable?

+ Does this Indicator convey meaningful information?

+ Will this Indicator support discussions and conversations with 21st CCLC staff?

12

ImpactImpact

The data we collect from the Leading Indicators and PPICS study ideally supports conversations and implementation of quality programs locally and also helps us as a field understand Out-of-School Time program

IMPACTIMPACT

13

Dates to RememberDates to RememberTask Start Date End Date

PPICS Related Activities

APR Data Open 11/15/2011

Student ID Data 10/18/2011 12/15/2011

Site Coordinator Survey

11/21/2011 12/15/2011

Site Visits

Site Visits 2/15/2012 3/15/2012

14

ContactContact

Neil NaftzgerPrincipal ResearcherP: 640-649-6616E-Mail: nnaftzger@air.org

Deborah MoroneySenior ResearcherP: 312-288-7609E-Mail: dmoroney@air.org