Alternate Alternate Assessments on Assessments on

Alternate Alternate Achievement Achievement

StandardsStandardsStudent PopulationStudent PopulationJacqueline F. Kearns, Ed.D.

Elizabeth Towles-Reeves, MS

OBSERVATION

INTERPRETATION

COGNITIONStudent PopulationAcademic contentTheory of Learning

Assessment SystemTest DevelopmentAdministration Scoring

ReportingAlignmentItem Analysis & DIF/BiasMeasurement errorScaling and Equating Standard Setting

VALIDITY EVALUATIONEmpirical evidence

Theory & logic (argument)Consequential features

The Assessment Triangle & Validity EvaluationMarion & Pellegrino (2006)

Cognition Vertex Validity Cognition Vertex Validity QuestionsQuestions

1)1) Is the assessment appropriate for Is the assessment appropriate for the students for whom it was the students for whom it was intended? intended?

2)2) Is the assessment being Is the assessment being administered to the appropriate administered to the appropriate students?students?

Both are important for the validity Both are important for the validity evaluationevaluation

More Different Than More Different Than AlikeAlike

SOURCE: Education Week analysis of data from the U.S. Department of Education, Office of Special Education Programs, Data Analysis System, 2002-03

Issues in Teaching/Assessing

Students in Alternate Assessments Varied levels of symbolic communication

Attention to salient features of stimuli Memory Limited motor response repertoire Generalization Self-Regulation Meta-cognition Skill Synthesis Sensory Deficits Special Health Care Needs

Kleinert, H., Browder, D., & Towles-Reeves, E. (2005). The assessment triangle and students with significant cognitive disabilities: Models of student cognition. National Alternate Assessment Center, Human Development Institute, University of Kentucky, Lexington. (PDF File)

Previous Data Previous Data 165 Students across 7 states165 Students across 7 states Extensive documentation through 111 item Extensive documentation through 111 item

inventoryinventory Findings suggest:Findings suggest:

64% routinely use verbal language64% routinely use verbal language 46% routinely understand pictures used to 46% routinely understand pictures used to

represent objectsrepresent objects 11% don’t understand pictures used to 11% don’t understand pictures used to

represent objects. represent objects. Almond & Bechard (2005) An In Depth Look at Almond & Bechard (2005) An In Depth Look at

students who take alternate assessments: students who take alternate assessments: What do we know. Colorado EAG.What do we know. Colorado EAG.

Learner Characteristics Learner Characteristics Demographic VariablesDemographic Variables

Learner Characteristics (all on a continuum of skills): Expressive Language Receptive Language Vision Hearing Motor Engagement Health Issues/Attendance Reading Mathematics Use of an Augmentative Communication System

(dichotomous variable)

MethodologyMethodology

Four partner states chose to participate States 1, 2, and 3:

gathered data in the administration process for their AA-AAS via scannable document (i.e., bubble-sheet)bubble-sheet)

State 4: gathered data using Zoomerang, an online

survey package. N= 7,075

States & LCI Response States & LCI Response RatesRates

StatStatee

GeograpGeographyhy

DemograpDemographichic

SampleSample

NNResponRespon

sese

RateRate

StatStatee

11

EasternEastern RuralRural

SuburbanSuburban 35953595 75%75%

StatStatee

22

North North EasternEastern

Urban-Urban-

SuburbanSuburban27932793 100%100%

StatStatee

33

EasternEastern UrbanUrban 468468 91%91%

StatStatee

44

WesternWestern RuralRural 219219 47%47%

Alternate Assessment Participation Rates : %

Total populationState 1

.959%

State 2

1.14%

State 3

.766%

State 4

.55%

“Most significant cognitive disabilities”

Expressive LanguageExpressive Language

Expressive Language

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

80.0%

90.0%

State 1 State 2 State 3 State 4

State

Perc

enta

ge

Presymbolic

Emerging Symbolic

Symbolic

Receptive LanguageReceptive LanguageReceptive Language

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

80.0%

90.0%

State 1 State 2 State 3 State 4

State

Perc

enta

ge

Uncertain response

Alerts to input

Requires cues

Follows directions

Use of Augmented Use of Augmented CommunicationCommunication

Number of Students not using ACS

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

80.0%

90.0%

State 1 State 2 State 3 State 4

State

Perc

enta

ge

Presymbolic

Emerging Symbolic

ReadingReading

Reading

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

State 1 State 2 State 3 State 4

State

Perc

enta

ge No awareness

Aware of text

Reads basic sight words

Basic understanding

Critical understanding

MathematicsMathematics

Mathematics

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

State 1 State 2 State 3 State 4

State

Perc

enta

ge

No awareness

Counts by rote to 5

1:1 correspondence

Does computational procedureswith or without a calculator

Applies computational procedures

Who are the Kids? Represent ~1% or less of the total assessed population All disability categories were represented but primarily 3 emerge,

Mental Retardation Multiple Disabilities Autism

Highly varied levels of expressive/receptive language use Most students in the population use symbolic communication Level of symbolic language distribution is similar across grade-bands Only about 50% of the pre and emerging symbolic language users

use ACS Pre-symbolic expressive language users are more likely to have

additional complex characteristics. Most of the population read basic sight words and solve simple math

problems with a calculator. Lack of skill progression in reading across grade bands (elementary,

middle & high) Skill progression apparent in mathematics across grade bands but

still small

Limitations

Only four state participants Small sample size Global items in reading and math Participation rates at 1% or less

Cognition Vertex: Cognition Vertex: Validity Evaluation Essential Validity Evaluation Essential

QuestionsQuestions Who is the population being assessed?Who is the population being assessed? How do we document and monitor the How do we document and monitor the

population?population? What do we know about how they learn (theory of What do we know about how they learn (theory of

learning) academic content?learning) academic content? What do our assessment results tell us about how What do our assessment results tell us about how

the population is learning academic content?the population is learning academic content? Are our data about the population and theory of Are our data about the population and theory of

learning learning consistentconsistent with student performances with student performances on the assessment?on the assessment?

If not, what assumptions are challenged?If not, what assumptions are challenged? What adjustments should be made?What adjustments should be made?

ParticipationParticipation Theory of LearningTheory of Learning Student PerformanceStudent Performance

References Agran, M., Fodor-Davis, Moore, & Martella, (1992). Effects of peer-delivered self-instructional

training on a lunch-making task for students with severe disabilities. Education and Training in Mental Retardation, 27, 230-240.

Billingsley, F., Gallucci, C., Peck, C., Schwartz, I., & Staub, D. (1996).  "But those kids can't even do math:  An alternative conceptualization of outcomes in special education.  Special Education Leadership Review, 3 (1), 43-55.

Brown, L., Nisbet, J., Ford, A., Sweet, M., Shiraga, B., York, J., Loomis, R. (1983). The critical need for non-school instruction in educational programs for severely handicapped students. Journal of the Association of the Severely Handicapped. 8, 71-77.

CAST (2002). Fox, (1989). Stimulus Generalization of skills and persons with profound mental handicaps.

Education and Training in Mental Retardation, 24,219-299. Haring, N. (1988). Generalization for students with severe handicaps: Strategies and solutions.

Seattle, WA: University of Washington Press. Hughes, C. & Agran, M. (1993). Teaching persons with severe disabilities to use self-instruction in

community settings: An analysis of the applications. Journal of the Association for Persons with severe Handicaps, 18, 261-274.

Hughes, C., Hugo, K., & Blatt, J. (1996). Self-instructional intervention for teaching generalized problem-solving with a functional task sequence. American Journal of Mental Retardation, 100 565-579.

Westling, D., & Fox, L. (2004). Teaching Students with Severe Disabilities. Columbus: Pearson (Merrell).

Whitman, T. L. (1990). Self-regulation and mental retardation. American Journal on Mental Retardation, 94, 347-362.

Contact InformationContact Information

Jacqueline Kearns, Ed.D.Jacqueline Kearns, Ed.D. 1 Quality Street, Suite 7221 Quality Street, Suite 722 Lexington, Kentucky 40507Lexington, Kentucky 40507 859-257-7672 X 80243859-257-7672 X 80243 859-323-1838859-323-1838 Jacqueline.kearns@uky.edu

Elizabeth Towles-Reeves, MSElizabeth Towles-Reeves, MS 1 Quality Street, Suite 7221 Quality Street, Suite 722 Lexington, Kentucky 40507Lexington, Kentucky 40507 859-257-7672 X 80255859-257-7672 X 80255 859-323-1838859-323-1838 Liztowles-reeves@uky.edu

www.naacpartners.org