Experiences and Lessons Learned Using Publishers’ Online Assessment Systems for OSEP Reporting...

Experiences and Lessons Learned Using Publishers’ Online Assessment Systems for OSEP Reporting

Presenters:Barbara Jackson – Munroe-Meyer Institute,

UNMCJanThelen, Nebraska Dept. of EducationRuth Littlefield, New Hampshire Dept. of

EducationAugust 27th, 2007

2

State Overview

3

What is Results Matter in Nebraska?

A statewide child and family outcomes system Designed to improve programs and supports For all young children birth to five, including

Part C and Part B 619 Served by school districts, the Early

Development Network, and their partners Results Matter applies to all school-based

early childhood programs in Nebraska

Results Matter in NE

4

Results Matter in Nebraska Purpose…

improve experiences, learning, development and lives of young children (birth to age five) and their families

inform program practices demonstrate program effectiveness guide the development of local and state policies

and procedures provide data to demonstrate results

5

Results Matter in Nebraska calls for curriculum and child outcomes assessment that ...

is based on ongoing observation of children engaged in real activities, with people they know, in natural settings

reflects evidence-based practices engages families & primary care providers as active

participants integrates information gathered across settings is individualized to address each child’s unique ways of

learning informs decisions about day-to-day learning opportunities

for children reflects that development and learning are rooted in culture and supported by the family


6

Implementation Process for Results Matter in Nebraska NECTAC state planning framework with state

management team in January 2005 State & Local Infrastructure Personnel Development Implementation/Practices Child/Family/Program Outcomes

Engage State Level Child Measurement Task Force on implementation process

Implemented six pilot sites in 2006 Statewide implementation in January 2007


7

Results Matter in Nebraska

Three child measurement tools….

The High/Scope Child Observation Record (COR) The Creative Curriculum Developmental Continuum Assessment, Evaluation and Programming System (AEPS)

Use of publishers’ online systems for supporting documentation of requiredassessments


8

NH Preschool Outcome Measurement System A statewide system to measure outcomes for

preschoolers with disabilities Designed to improve programs & supports For all preschool children with disabilities

with an IEP (effective November 1, 2006) May be used more broadly, at local

discretion

9

NH Preschool Outcome Measurement System GSEG Cornerstone grant for Part C & 619

Engage stakeholders Align Child Outcomes (Birth through 5) with

Early Learning Guidelines Consider how to address family outcomes Federal reporting AND Improving

outcomes Focus on progress, not comparison

10

Two Lead agencies Part C: 10 Area Agencies with 14 programs Part B 619: 177 School districts with multiple

programs Shared goal: improve results for children with

disabilities and their families Part C: COSF 619: Publishers web-based data systems

Behind the scenes conversion to OSEP reporting requirements

NH Preschool Outcome Measurement System

11

Three child measurement tools… Assessment, Evaluation, and Programming (AEPS®):

http://www.brookespublishing.com/store/books/bricker-aeps/osep.htm

Brigance® Inventory of Early Development II (IED-II): http://www.curriculumassociates.com

The Creative Curriculum® Developmental Continuum Assessment for Ages 3-5: http://www.creativecurriculum.net/

Allowed for flexibility Assessor preference Best fit for the child

NH Preschool Outcome Measurement System

12

State Administration

13

NDE Administrative Structure

254 School Districts & 19 Regional Service Units

State License Subscriptions School Districts Head Start Programs

NDE memos

Authorization forms completed as part of enrollment process

NSSRS: NDE student identification number


14

NDE Online Structure


15

NDE Contracts and Costs

Jan 2006 NDE emails surveys to districts to collect: Information on # of children in school district early

childhood programs Selection of tool(s)

Fall 2006 developed contracts with three Publishers Brookes Publishing – AEPSi.com $11.00 Pearson Learning – Highscope.net $11.00 Teaching Strategies – Creativecurriulum.net $10.95

Fall 2007 Head Start Agencies contract with NDE to be part of the license


16

NH Administrative Structure

177 School districts in 80+ School Administrative Units

NHDOE memos Statewide Preschool Technical Assistance

Network (PTAN) Professional development, TA and networking Immediate communication vehicle

17

NH Administrative Structure

NHSEIS (NH Special Education Information System)

Student Identification numbers Limited authority to view personal data Limited resources to fund the system High level of commitment to quality and

simplicity for districts

18

NH Contracts

Local Districts sign on with publishers No State-Level Umbrella agreements Districts negotiate directly with publishers

NHDOE: online access to reports for OSEP district and state level (# & %) run 3 reports and compile as needed

Districts access to all assessment data Inform instruction Demonstrate effective practices Promote discussion with families and other early

childhood partners

19

NH Cost

Survey Monkey: estimate number of children and users for each tool (for pricing & training)

State provided estimated children and or users to publishers

Paid for by local districts NHDOE worked with publishers to establish

best rate (not as good as umbrella agreement)

20

NH Cost

AEPS One-time state fee

Establish data reporting system & Create state account: $1750

Special rate to Beta Testers: $9/child Rate for other NH users: $11/child Complete AEPS®, Second Edition (Administration

Guide, Test for Birth to Three Years and Three to Six Years, Curriculum for Birth to Three Years, Curriculum for Three to Six Years) $239.00 with 10% discount when subscribe to AEPSi

21

NH Cost

Brigance IED II One Inventory per user $159.00 each One Developmental Record Book per student

$35.00/10-pack or $329.00/100-pack Web-based IED-II Management System

annual subscription $124.95 each user first year ($99 for subsequent years)

State account charged at same rate as local user ($124.95)

22

NH Cost

Creative Curriculum $11.75 per child No state fee No additional cost for tools/materials (unless

want hard copies)

23

NDE Customizing Needs

Funding Source-allows for differential children

Special Education Funding Early Intervention Part C (Birth-3) Early Childhood Sp Ed Part B (3-5) Not Applicable

State Funded EC Funding Source EC Grant Program Even Start Migrant Program Title I Not Applicable

Other Funding Source Head Start Early Head Start Not Applicable

Child Newly Verified (IFSP/IEP) after 8.1.06 Yes /No Added 10 Character ID #


24

NH Customizing Needs

NH Head Start Collaboration office is working with NHDOE to better coordinate outcome measurement across systems

Need to consider Work Sampling (used by some NH Head Start)

Work Sampling: user enters assessment information the COSF data point is generated the user can adjust up or down

25

NH Customizing Needs

Each Publisher offers different benefits and challenges

Each has a representative dedicated to NH Areas for negotiation:

Tracking students who move within the state Online and phone TA for users Ease of federal reporting

Always ask the next question!

26

Components for Implementation

27

NH Training/Professional Development

Assessment Fair May 2006 Publishers paid us: they provided

demonstrations and resource PTAN

local networking phone and online TA Coordinated series of publisher trainings

Preschool Leadership Conference (Sept 07)

28

NH Supports and Resources

Statewide Preschool TA Network (PTAN) http://ptan.seresc.net/

Preschool Outcome Measurement Toolkit NHDOE Special Education Memos Links to ECO

Developing new FAQ

29

NDE Training/Professional Development Statewide Training Series on Assessment – Rolled

out January 2006 Publishers provided first training (January) State trainers provided at three regional sites

(Spring) Results Matter State Conference on Measurement –

Spring 2007 Regional Training Series on Early Childhood

Curriculum – Planned for Fall 2008 Developed collaboratively and supported through the

Early Childhood Training Center


30

NDE Supports and Resources

http://www.nde.state.ne.us/ECH/ECH.html

Click on Results Matter

NDE Administrative Guidelines

NDE Assessment Fidelity Process


31

Why implement a fidelity process?

Maximize the credibility of observational data collected for Results Matter

Reliable and valid data maximizes quality of information Consistency among providers in scoring

observations (intra- and inter-reliability) Documentation to support scoring (validity)

Quality information forms the basis for program planning


32

Who does the plan affect?

All providers responsible for administering and scoring observational assessments for Results Matter Early Childhood Practitioners

Early Childhood Special Education Practitioners

Related Services Providers


33

Levels of Expertise

Fidelity process is based on providers’ experience

Tier 1: Providers new to the assessment

Tier 2: Providers experienced with assessment

Tier 3: Experienced providers who can mentor others


34

Five Components of the Fidelity Process

1. All providers must meet reliability standards by April 1, 2008

2. New provider orientation

3. Continued group monitoring

4. Individual monitoring

5. Ongoing mentoring of providers


35

Sample Plan & District Level Strategies

Tier 1: Process for Providers New to the Assessment Tool Check all that are part of your plan and add additional activities as

needed. Approximate # of providers at this level: ____

Score state reliability tapes and achieve reliability standard

Shadow experienced provider, documenting observations and ratings, checking for reliability

Supervisor reviews samples of supporting documentation


36

Reports Available andHow They Are Used

37

AEPSi

Near entry only reports now available OSEP near entry – near exit report available

as of Sept 2007 State and district level reports Districts access online information about their

children and programs

38

AEPSi – Near Entry Report

39

Creative Curriculum

OSEP Reports Child Progress Reports Development of 9 point scale to assess at-risk

children CreativeCurriculum.net + Special Needs

Accountability Pack (SNAPTM) and OSEPreports.net

District and State level reports Local school districts: all information on their

children/programs

40

Creative Curriculum

41

High/Scope COR

Child Outcome Reports OSEP report – Data Downloaded to

High/Scope Foundation for analysis Report is in progress – available Sept 2007


42

NH Brigance Inventory of Early Development

43

Publisher/State Challenges

Different stages of online development Different levels of experience Creating interface with state data systems

and processes Addressing children with significant delays Responding to OSEP timelines Emerging OSEP guidance

44

Discussion

Experiences and Lessons Learned Using Publishers’ Online Assessment Systems for OSEP Reporting...

Documents

Transcript of Experiences and Lessons Learned Using Publishers’ Online Assessment Systems for OSEP Reporting...