BC PROVINCIAL ASSESSMENTS: DIRECTIONS FOR THE FUTURE

Assessment Group for Provincial Assessments, June 6 2013

Kadriye Ercikan

University of British Columbia

Outline

Purposes of assessments from different perspectives

Sample versus census testing Some principles we can discuss, agree,

disagree on

Purposes of Provincial Assessments To provide system level achievement data

Examine effectiveness of curriculum – is the curriculum appropriate for different age and cultural groups

As a measure of system level achievement, for monitoring learning To identify inequities and help inform policy and resource allocation

To provide classroom and school level data Provide information to teachers Serve as an external measure of achievement that is comparable

across the province To provide individual student level data

Inform teaching and learning, possibly starting earlier than grade 4 Provide information about learning and growth of children in care

and Aboriginal children Provide information to parents about how children are doing

Concerns

Consequences of use of assessment resultsUnfair comparisons of studentsInappropriate interpretations of assessment

results, such as school rankings as indicators of school effectiveness

Narrowing curriculum and negative effects on classroom teaching

Confidentiality when data are released

Random Sample versus Census Testing Random Sample Testing

What it isMotivation

Can it meet all needs of the province Comparison to census testing Continuing to inform key policy

decisions What will it look like?

Random Sample Testing Sampling

the way in which students are selected for the assessment

Sample Selected group of students for the assessment

Population Population refers to all eligible students within a

jurisdiction Representative sample

All students in the population need to have an equal probability of being selected

Large enough number of students to allow for adequate representation of the population’s performance

Census Testing

Test ALL grade 4 and 7 students and make inferences about grade 4 and 7 students in the province

Inference in Census Testing

Population Population

Random Sample Testing

Test a representative sample of grade 4 and 7 students

Make inferences about all grade 4 and 7 students

Inference in Random Sample Testing

Representative sample Population

Motivation for Random Sample Testing Reduce burden on teachers and

students Cost savings Cover a broader range of curriculum Include performance assessment tasks Faster scoring and reporting of scores Reduce chances of inappropriate use of

assessment data

Does Random Sample Testing Fit with FSA’s Purposes? Purposes of FSA

“The main purpose of the assessment is to help the province, school districts, schools and school planning councils evaluate how well students are achieving basic skills, and make plans to improve student achievement.”

Two key purposes Evaluate Inform planning

For both of these purposes the assessment provides information about achievement levels in relation to the provincial curricular and learning goals.

Assessment results based on a representative sample of Grade 4 and 7 students can be generalized to all Grade 4 and 7 students in BC.

Sampling will determine if district level information is obtained

Trade-offs of Random Sample Administration No individual student level scores to

students, teachers and parents No classroom level scores to teachers Schools won’t be able to track their yearly

progress based on FSA If special considerations are not made, no

district level scores Unless special sampling is considered no

scores by aboriginal, ESL, students with special needs groupings

Example: Pan-Canadian Assessment Program (PCAP) A national assessment that measures

the knowledge and skills of 13-year- old students in reading, mathematics, and science.

In 2007, over 30,000 students from across Canada wrote PCAP.

Students from all ten provinces and the Yukon participated.

In BC, around 2,500 students from more than 150 schools participated.

Sampling in PCAP Two-step stratified sampling process

1. Random selection of schools from each jurisdiction, drawn from a complete list of publicly funded schools provided by the jurisdiction

2. Random selection of students, drawn from a list of all eligible participants within each school

In the case where numbers were smaller than the desired size, all schools and/or all students meeting the criteria within the jurisdiction were selected.

This assured an adequate number of participants to allow for reporting on their achievement as if all students within the jurisdiction had participated

Important Sub-group Information Aboriginal, ESL, special needs students

for example Students in special programs These can be achieved by over-

sampling, for example including all aboriginal students in a particular district.

Scenario 1: Province Level Reporting of Scores Only Two-stage random sampling Approximately 150 schools are

randomly selected Within each selected school,

approximately 25 students are randomly selected

OR Within each selected school one

classroom is randomly selected.

Scenario 2: District Level Reporting of Scores Three-stage random sampling Within each district, randomly select a

number of schools proportional to the size of the district.

Within each selected school approximately 25 students are randomly selected

OR Within each selected school one classroom

is randomly selected.

What’s Reported Currently scores are broken down by

Gender, Aboriginal/Non-aboriginal, ESL and Special Needs

Gender does not need special sample considerations

Aboriginal students will require over-sampling of this population

ESL and Special Needs, depending on the purposes of reporting, over sampling may be needed

Future Directions

Principles for the assessments Addressing Concerns

Principles

Assessments need to assess valued outcomes

Assessments need to be aligned with learning, instructional and curricular goals

Assessments need to provide accurate estimates of student knowledge and competencies

Score reports need to be informative and developed in a timely manner

Consequential aspects of assessment results need to be an important consideration

Concerns, Issues to be Addressed Use, misuse, and interpretation of

assessment data Confidentiality Validity investigations Explicit theories of actions Support and resources to address

inequities