CSP Monitoring and Data Collection · Purposes of CSP Monitoring and Data Collection To increase...
36
WestEd.org WestEd.org CSP Monitoring and Data Collection Project Directors Meeting Presentation
Transcript of CSP Monitoring and Data Collection · Purposes of CSP Monitoring and Data Collection To increase...
WestEd.orgWestEd.org
CSP Monitoring and Data Collection
Project Directors Meeting Presentation
WestEd.org
Objectives of Presentation
• Highlight new monitoring and
data collection content and
processes
• Establish common understanding
of monitoring purpose and
objectives
• Provide opportunity for Q&A
WestEd.org
Agenda of Presentation
• Project Overview
• Definition of Monitoring
• Purposes of CSP Monitoring and
Data Collection
• Process Updates
• Content Updates
• Reflection and Discussion
WestEd.org
Who is WestEd?
is a research, development, and
service agency that works with education and other communities to promote excellence, achieve
equity, and improve learning for children, youth, and adults.
has monitored several
U.S. Department of Education (ED)
discretionary grant programs including the Charter Schools Program, Magnet Schools Assistance
Program, and Voluntary Public School Choice.
WestEd.org
Basic Features of Monitoring and
Data Collection Project
Monitoring
•Select SEA and CMOgrantees annually
•Updated content
Data Collection
•Annually for all grantees
•Limited change in content
Online System
•Data storage
•Reporting dashboard
Evaluation TA
•Contract with external consultants
•Primarily for ED staff
Other
•Admin
•Protocol development
•Advisory group
•PD meetings
WestEd.org
Where Does Monitoring and Data
Collection Fit?
Grantee Oversight
Post-Award
Calls (ED)
Quarterly Calls (ED)
SubawardData
(WestEd)
Monitoring Visits
(WestEd)
Corrective Action
Plans (ED)
Technical Assistance
(ED)
WestEd.org
Purposes of CSP Monitoring and Data
Collection
To increase fiscal and programmatic accountability
To support and improve grantee capacity in carrying out the purpose of the CSP statute through the timely and efficient administration of
Federal funds awarded under the CSP program
To assist grantees with the planning and implementation of high-quality charter schools
To assess impact of CSP funding on the development of high-quality charter schools
WestEd.org
Definition of Monitoring
Monitoring is…a continuousfunction that uses the systemic collection of data on specified indicators to provide management and the main stakeholders of an ongoing development intervention with indications of the extent of progress and achievement of objectives and progress in the use of allocated funds
(OECD, 2002)
A friendly reminder that monitoring is not…
An Evaluation
A Witch-hunt
A Gotcha
WestEd.org
Monitoring Process
Pre-visit Preparations
On-site Observations
Post-visit Synthesis & Analysis
Report Review (from ED, grantee)
Technical Assistance & Corrective Action
(from ED)
• Grantees selected
based on risk and
other factors
• Occurs at least
once within a grant
cycle
• Conducted on-site
WestEd.org
Defining Data Collection
Grantee data collection is designed to… track
Federal funds to charter school developers and
charter schools through the collection of award
data, including awardee contact information,
award amount, project period, and the purpose
of the award (i.e., planning, implementation,
replication/expansion, or dissemination).
WestEd.org
Data Collection Process
Pre-submission Technical Assistance
Submission of Award Data
(ongoing)
Data Review/Cleaning
Award Database Production
Standard and Ad Hoc Analyses
• Annual collection
for all grantees
• Conducted
remotely
WestEd.org
What’s New? Process Updates
• Online monitoring and data
collection system
• Checklist vs. Descriptive Items
and Indicators
• Differentiated report content and
revised format
WestEd.org
Online Monitoring and Data
Collection System
• All data entry (monitoring visits,
data collection templates)
• Reporting and dashboard
features
• Analytics
WestEd.org
Online Data Collection Efforts
• Rolling out with new FY15
grantees
• Existing grantees will continue with
previous approach
• Will include user manual and
other technical assistance
WestEd.org
Data Collection – Key Changes
• Added element on administrative expenses
(proposed)
• Added data validation features
• Continuous access to the data collection
system
• Enables real-time updates of award data
WestEd.org
System Architecture - Inputs
Online Platform
Award Data (all)
SEA Monitoring Indicators
CMOMonitoring Indicators
Other (TBD)
WestEd.org
WestEd.org
WestEd.org
WestEd.org
WestEd.org
Reflection & Discussion
WestEd.org
Indicators and Items
• Streamlined content
• Focused on implementation
issues and TA/CA needs
• Standardized response options
• New and expanded ratings
• Standardized recommendations
WestEd.org
Response Options – for almost every
indicator
Implementing as proposed or necessary
(no additional text necessary).
Implementation issues identified (explain).
Non-substantive changes in proposed
activities(explain).
Promising practice(s) identified (explain).
WestEd.org
Response Options - for Progress
Toward Objectives
No concerns with data quality or performance measure interpretation
Performance measure not applicable at time of monitoring visit
Data Quality/Interpretation Concerns:
Inconsistent units of measure over time
Data not aligned with performance measure
Inconsistent wording of performance measure over time
Incomplete or missing data
Other (specify)
WestEd.org
Previous Ratings and Justifications
3 – Grantee fully meets the indicator. No areas of Concern. Perhaps a
Promising Practice or two.
2 – Grantee partially meets the indicator. At least 1 Area of Concern. Perhaps a
Promising Practice or two.
1 – Grantee does not meet the indicator. Several Areas of Concern. No
Promising Practices
WestEd.org
Expanded Ratings
Does not meet indicator -Requires
corrective action
Policies, practices, or actions that demonstrate systemic or pervasive
non-compliance
of a monitoring indicator.
Partially meets indicator -Requires
corrective actions or substantial technical assistance
Policies, practices, or actions meet
some elements of
the monitoring
indicator, but also
demonstrate significant areas of
limited or non-
compliance.
Largely meets the indicator -
Recommended technical assistance
Policies, practices or actions that demonstrate compliance
with the majority of
conditions of a monitoring
indicator with areas
identified for technical
assistance.
Fully meets the indicator
Policies, practices, or actions that demonstrate
full compliance
with an indicator.
Fully meets the indicator - Best
practices identified
Policies, practices, or actions that demonstrate
full compliance
with an indicator and
where best or promising
practices have been identified.
WestEd.org
Standardized Recommendations
Does not meet the indicator
Requires corrective
action
Partially meets indicator
Requires corrective
action
Requires substantial technical
assistance (TA)
Largely meets the indicator
Recommended technical
assistance (TA)
Fully meets the indicator
None
WestEd.org
Report Types
Interim
For ED
Summary of compliance
Basic implementation
details
Technical
For ED, Grantee
Summary of compliance
Full implementation
details
Summary
For public
Summary of compliance
No implementation
details
Grantee Context
For ED
Policy context only
WestEd.org
Reflection & Discussion
WestEd.org
SEA Indicators – Key Points
• Added indicator on strategy and vision,
including logic model review
• Expanded indicator on quality authorizing
practices
• Added indicator on assisting disadvantaged
students
• Shifted focus on performance measurement
quality
WestEd.org
Section 1: Subgrant Award Process
1.1 Descriptions
& assurances
1.2 Eligible applications
1.3 Definition of charter
school
1.4 Peer review
1.5 Program periods
WestEd.org
Section 2: CSP & Charter School
Quality
2.1 Strategy & vision
2.2 Quality authorizing practices
2.3 Flexibility & autonomy
2.4 Subgranteequality
2.5 Educationally
disadvantaged students
2.6 Subgranteemonitoring
2.7 Dissemination
2.8 Performance measurement
quality
WestEd.org
Section 3: Administrative & Fiscal
Responsibilities
3.1 Federal programs &
funding 3.2 Allocations
3.3 Administration of CSP funds
3.4 Use of grant funds
3.5 LEA deductions
3.6 Transfer of student records
3.7 Recordkeeping
WestEd.org
Reflection & Discussion
WestEd.org
Next Steps
• Schedule and conduct site visits
• Hold grantee review group meetings• March 24 for SEAs
• March 29 for CMOs
• Pilot online monitoring effort
• Pilot online data collection effort with FY15 grantees
• Continue traditional data collection efforts with pre-FY15 grantees
WestEd.org
Contact us!
Sara Allender
• Co-director
John Flaherty, Jr.
• Co-director
Steve Ruffini
• CMO Coordinator
Khadijah Salaam
• SEA Coordinator & Data Collection Coordinator
