Building Quality Child Assessment Systems for IDEA Infant/Toddler and

Preschool Programs

Where We’ve Been Lately and Where We Might Be Going Next

Mary McLean

8/27/2008

Where We’ve Been Lately

Working on Outcomes Assessment!

IDEA 2004 and Accountability

Is money spent on programs for young children with disabilities

producing good outcomes?

Decisions, decisions….

What to measure ?

Status vs progress ?

One instrument or many ?

Direct assessment or observation-based?

Sample or report on all children?

Part C and Preschool Child Outcomes

% of children who demonstrate improved:

Positive social emotional skills (including positive social relationships)

Acquisition and use of knowledge and skills (including early language/ communication [and early literacy])

Use of appropriate behaviors to meet their needs

Reporting Categories

A. % of children who did not improve functioning

B. % of children who improved functioning but not sufficient to move nearer to functioning comparable to same-aged peers

C. % of children who improved functioning to a level nearer to same-aged peers but did not reach it

D. % of children who improved functioning to reach a level comparable to same-aged peers

E. % of children who maintained functioning at a level comparable to same-aged peers

Timelines

__X__December 2005 – States submit State Performance Plans

__X__Feb 2007 – APR: Child status at entry data

__X__Feb 2008 – APR: First progress data

_____2009 - First report to the public

_____2010 – Targets to be set

Time to Celebrate!!

Where We Are Now

Working on Quality Assessment Practices

As states work through the many challenges involved in developing accountability systems, we may find that the push for accountability has the unintended positive consequence of building better assessment practices.

Hebbeler, Barton & Mallik, 2007

Building a better assessment system

for children, families and programs

Recommended Practices for Assessment Neisworth, J.& Bagnato, S. (2005). DEC recommended

practices: Assessment. In Sandall, Hemmeter, Smith & McLean (Eds) DEC recommended practice: A comprehensive guide for application. Longmont, CO: Sopris West Publishing Co.

NAEYC and NAECS/SDE (2003). Early childhood curriculum, assessment, and program evaluation: Building an effective accountable system in programs for children birth through age 8. http://www.naeyc.org/about/positions/cape.asp

DEC (2007). Promoting positive outcomes for children with disabilities: Recommendations for curriculum, assessment and program evaluation. www.dec-sped.org

Snow, C. & VanHemel, C. (2008). Early childhood assessment: Why, what and how? Washington, DC: National Academies Press.

Characteristics of a Quality Assessment System Useful (utility)

Authentic

Collaborative

Universal

Useful

assessment utility

COSF

1. Assess at entry and

exit. 2. Determine status

ratings at entry and

exit

3. Provide

this information

to the state.

4. The state

collects this data and reports to

OSEP: percentages of

children meeting criteria for each

reporting category

5. The state

determines goals and

improvement activities.

Social Emotional

Acquiring knowledge and skills

Actions to meet needs

The Birth to 6 Child Outcome System

Formative and Summative Assessment

Formative Assessment Assessment for

learning

Ongoing during intervention

Informs instruction

Benefits the children who are being assessed

Summative Assessment

Assessment of learning

Completed as a child is exiting a program

Informs accountability or program evaluation

Benefits programs

Wisconsin Model Early Learning StandardsTeaching Cycle

Ongoing AssessmentGathering information to determine what the child can do and what the child is ready to learn

Planning Deciding what should be done to promote development and what we want children to learn.

ImplementationProviding meaningful, experiential activities that support individual and group goals guided by supportive interaction and relationships

Ongoing Assessment

Helps decide what to teach

Helps decide how to teach

Helps to know when to make changes in teaching

Is “assessment in the service of instruction”

(McAfee & Leong, 2002)

An Assessment System with High Utility

Informs program planning and progress monitoring and accountability

Informs general early childhood planning/monitoring and IFSP/IEP planning/monitoring

COSF

1. Utilize

authentic, on-going

assessment practices.

2. Determine status

ratings at entry and

exit

3. Provide

this information to the state.

4. The state

collects this data and reports to

OSEP: percentages of

children meeting criteria for each

reporting category

5. The state

determines goals and

improvement activities.

Social Emotional

Acquiring knowledge and skills

Actions to meet needs

The Birth to 6 Child Outcome System

Authentic

Authentic assessment

Observation of child behavior over time in typical routines

and activities

Standardized_________________________________Authentic

Direct Assessment__________________Observation-based assessment

“…the science of the strange behavior of children in strange situations with strange adults for the

briefest possible period of time”

Bronfenbrenner (1977)

(standardized assessment)

“The best way to understand the development of children is to observe their behavior in natural settings

while they are interacting with familiar adults over prolonged periods of time.” Bronfenbrenner, 1977

(authentic assessment)

Observation_______Direct Assessment

Procedures for consistency in administration and scoring are built in

Behavior sampled may not be representative of child’s typical behavior

Consistency depends on teacher training and monitoring of implementation

Behavior measured is child’s typical behavior

Increased utility for instruction

(Mathematica, 2007)

Strategies for authentic assessment

Observation and documentation

anecdotal notes

event sampling

activity protocols

portfolio assessment

Embed assessment into ongoing routines and activities

(Raver, S.,2003; Sandall & Schwartz, 2008)

Activity Matrix(Sandall & Schwartz, 2008)

Schedule Shelby Matthew Davion

Arrival Respond to peers Remove and hang up coat Follow 2-step directions

Free Play Engage in activity Join ongoing play Request preferred item

Circle Imitate adult words Imitate adult actions Tell full name

Outside Throw ball with 2 hands Join ongoing play Request preferred item

Snack Use pincer grasp Pour juice

Free Play 2-word utterance Imitate adult actions Sort shapes

Circle/Departure 2-word utterance Identify positional concepts

Transitions Follow 2-step direction

Anecdotal notes

Event Sampling

Schedule Shelby Matthew Davion

Arrival Respond to peers 2/2Remove and hang up coat

with verbal promptFollow 2-step directions 2/2

Free PlayEngage in activity

8 minutesJoin ongoing play 2/2

Request preferred item

3/4

Circle Imitate adult words 5/7 Imitate adult actions 4

Tell full name

with model

OutsideThrow ball with two hands

4/4

Join ongoing play

1/1Request preferred item 1/2

Snack Use pincer grasp 4/9 Pour juice

with physical guidance

Free Play 2-word utterance 4 Imitate adult actions 3 Copy square 2/2

Circle/Departure 2-word utterance 3Identify positional concepts

2/2 under, on top

Transitions Follow 2-step direction 2/2

Activity Protocols

An activity protocol provides a list of skills from a curriculum-referenced assessment that are likely to be observed in a particular activity or routine

Grisham-Brown, Hemmeter & Pretti-Frontczak, 2005

Activity Protocol: Playdough

AREA ITEM CHILD BEHAVIORS

Motor M-1 Holds object with one hand and

manipulates with other

Cognitive C-3 Demonstrates understanding of size concepts

Adaptive A-4 Fastens button on art smock

Portfolio Assessment

Paper

Electronic

Use of Authentic Assessment to Inform Multiple Measures

To inform curriculum-referenced assessments which measure progress toward EC learning targets or curricular goals

To measure progress toward IEP goals or IFSP outcomes

To inform measures of accountability

Collaborative

…with families and teams

Gathering information from families

Family involvement expands the validity of assessment information to home and community environments

Strategies for Gathering Information

Utilize home visits as a strategy to connect with families

Make periodic requests for information from families (describe specific skills to watch for at home.)

Use assessment tools that have family report forms: AEPS, HELP.

Use existing informal communication mechanisms (traveling notebook, daily conversations, e-mail, telephone.)

Gathering information from other service providers

How to Include Information from Other Providers Systematically

Plan collaborative activities for observations

Schedule periodic team meetings or staffings

Share information and request information

Universal

…and individually appropriate

Universality

Design and/or accommodations which enable all children to demonstrate their underlying functional capabilities

Bagnato, Neisworth,& Pretti-Frontczak, (in preparation).

Assessment must be valid for all children including children who are English Language Learners and children with disabilities

Assessment and Learning Targets

Early Learning Standards or Foundations

General Curriculum Commercial

Targeted Curricula(e.g., early literacy, social

emotional)

General Curriculum

Locally Developed

IFSP/IEP

Universal

Individualized

Universal Design

The design of products to be useable by all people to the greatest extent possible (Thompson, Johnstone & Thurlow, 2002)

Center for Applied Special Technology (CAST)

http://www.cast.org/research/udl/index.html

Desired Results access Project

California Department of Education

www.draccess.org

Accommodations/AdaptationsAdaptations are changes in environmental arrangements or differences in

observed behavior that allow children with disabilities to be accurately assessed in the natural environment:

Augmentative or alternative communication system

Alternative mode for written language

Visual support

Assistive equipment or device

Functional positioning

Sensory support

Alternative response mode

www.draccess.org

Where Are We Going Next?

Innovative Practices in Measurement

Innovative Models and Frameworks

Assessment Framework - RTI

Measurement Model - IRT / Rasch

Assessment Framework: RTI

Center for Response to Intervention in Early Childhood

http://www.crtiec.org/

Recognition and Response: An Early Intervening System for Young Children At

Risk for Learning Disabilities

http://www.fpg.unc.edu/~randr/

Where did RTI come from?

Initiatives in the field of learning disabilities

Written into IDEA 2004 – “early intervening”

Part B money can be used to fund programs for children who have not yet been found eligible for special education

RTI Tiered Model

Tier 1: Universal periodic screening and

resulting interventions

Tier 2: Research-based small group

interventions

Tier 3: Individualized interventions and

possible referral for special education

What Might RTI Mean for Early Childhood?

Universal and Periodic Screening (Prevention)

Progress Monitoring

Outcomes Measurement

Screening and Progress Monitoring:

Individual Growth and Development Indicators

(IGDIs)

General Outcomes Measurement (GOM)

Below Average (-1.5 SD) Trajectory

Child’s Observed Trajectory

Normative Trajectory

IGDI’s Communication Trajectory

www.igdi.ku.edu

IGDIs Available

Infant and Toddler

Early Communication

Early Problem Solving

Early Movement

Early Social

www.igdi.ku.edu

Preschool

Picture Naming

Alliteration

Rhyming

www.ggg.umn.edu

Accountability: Crosswalk to OSEP OutcomesAccountability: Crosswalk to OSEP Outcomes

Measurement Model:

Rasch Scaling

Item Response Theory

Rasch/IRT Measurement Model

“Allows the creation of interval-scaled calibrations when raw scores are ordinal”

Translation: a criterion or curriculum-referenced instrument can be calibrated so that it yields scaled scores (equal interval scores)

The child’s ability is described relative to position on a specific developmental path not relative to scores from the norming population.

2% 14% 34% 34% 14% 2%

MeanMedian Mode

Standard Deviation

-1 -1-2 -2

Classic Psychometric Theory: Normal Curve

More of the construct or skill (later developing skill)----

Less of the construct or skill (earlier developing skill)----

Rasch Scaling: Demonstrating higher level skills results in a higher score

So…..

Allows measurement of progress as acquisition of skills – not as position relative to scores from the norming population

Allows creation of an item bank which prevents “narrowing of the curriculum” or teaching to the test

(Meisels, 2002)

Rasch/IRT Instruments

Desired Results Developmental Profile access (DRDP access) www.draccess.org

Ages and Stages Questionnaire (ASQ) www.brookespublishing.com/store/books/bricker-asq/

Assessment, Evaluation and Programming System for Infants and Children (AEPS)

www.brookespublishing.com/store/books/bricker-aeps/

What Do We Need?

What Do We Need?

Research on: direct assessment and observation-based

assessment with young children

new assessment instruments

assessment systems that weave general early childhood and EI/ECSE

procedures for assessing children who are English Language Learners

accommodations for children with disabilities

What Do We Need?

Personnel Development: pre-service preparation that provides a solid

foundation in assessment skills

system for continuous in-service training to address local needs

emphasis on measurement skills in doctoral programs

What Do We Need?

Policy

alignment of early learning guidelines, assessment practices, curricular practices and accountability requirements across agencies and programs serving young children

Thank You!

Mary McLean, Ph.D.

Kellner Professor of Early Childhood Education

Department of Exceptional Education

School of Education

University of Wisconsin-Milwaukee

414-229-2213

mmclean@uwm.edu