Designing Fair and Accessible SACE Assessment Tasks

Adele Pring and Chris LawrenceSACE Improvement Team

Acknowledgement

We acknowledge that we are meeting on the traditional lands of Aboriginal Australians.Some (or several) of the students in your school are more than likely of Aboriginal heritage.Throughout the workshop can we all share effective assessment strategies that are inclusive of Aboriginal learners.

Beginnings

• We hope you will improve your skills in SACE assessment design through this PD.

• Everyone has skills to share so please contribute your expertise and critical questions

What is the Purpose of Assessment?

Jot thoughts for one minute

What is the Purpose of Assessment?

“The fundamental purpose of assessment in education is to establish and understand where learners are in an aspect of their learning at the time of assessment.”

Masters, 2014

Reflection on assessment

Discuss for 3 minutes with a person near you:

The best and worst assessment experience you have had. It may be related to you as a student or as a teacher.

6. Reliable

1. Equitable

2. Transparent

3. Valid

4. Flexible

5. Relevant

Fair and accessible assessment is …

1. Equity: Accessible learning

Many students have attention, vision and hearing problems that may be undiagnosed

Could an audio or visual resource enhance understanding of task requirement e.g. film, diagram, photograph, audio book?

Could a role play activity enhance conceptual understanding more effectively than dense reading material?

Equity: Cultural inclusion

If an Aboriginal student asks “Why didn’t I learn before Year 12 about Charlie Perkins?”See ‘Topics with a focus on Aboriginal knowledge, cultures and perspectives’ on the SACE website www.sace.sa.edu.au 2 minute activity: Consider one topic relevant to your learning area you’d like to find out more about and possibly share with students or find out what they already know.Text your topic to yourself and to google later and write on a sticky note with your name and school.

Equity: Fair time and space for completion

What support does your school provide for task completion for students who have:• inadequate homework environments• Inadequate access to the web, hardware and software• frequent absences• English as an additional language (including speakers of

Aboriginal English)

Equity: Capturing evidence of Capabilities• literacy• numeracy• ICT• critical & creative

thinking• personal and social

capability• ethical understanding• intercultural

understanding

Equity: Empowering students through metacognition

“Assessment as and for learning occurs when students are their own assessors. Students monitor their own learning, ask questions and use a range of strategies to decide what they know and can do, and how to use assessment for new learning.”

(NSW Board of Studies)

Equity: Tips to increase Assessment as and for Learning

• Talk less and allow students to talk more• Get students to sort performance standard descriptors into order of A-E• Check that students understand the task requirements and revise tasks

regularly, using student feedback and that from other teachers• Allow time for students to assess their own work by highlighting where

they have addressed particular assessment criteria at perceived grade levels

• Allow time for students to assess the work of peers• Provide feedback to students at checkpoints prior to deadlines so time

allows for improvements

Equity: Tip for teachers new to SACE subjects

• Read Chief Assessors’ reports on the SACE website and revise assessment tasks accordingly

• Explore all the support materials and advice for your subject on the SACE Board website’s minisites

• Attend ‘Teachers new to the SACE PD’ • Work with students to analyse and annotate examples of student

evidence from previous cohorts or SACE Board examples• Ask others in your school, or call/email the SACE Officer for your

subject (see SACE website) for specific support

2. Transparency

• Is the purpose of the task clear? • Does the task description allow achievement of the

assessment design criteria?• Does the task have an ‘entry level’ question/part that allows

all students the achievement of a ‘C’? (not always applicable)

• Does the task description use descriptors that encourage all students to achieve at their highest level, including an ‘A’?

Transparency: Readability of tasks and sources

• Is the reading level of the assessment task and source material accessible to all?

• What key words and vocabulary in the performance standards of your subject/s do students need to understand?

• Readability is what makes some texts easier to read than others

• Legibility concerns typeface and layout

Transparency: Readability

Question If a student was provided with three painted metal rods, one of which was known to be made from brass, one from magnetised steel and one from non-magnetised steel, describe how, without scratching the paint, the student could identify each of the rods.

Readability and step-by-step instructions

A student has three metal rods• One is made from brass.• One is made from magnetised steel.• One is made from non-magnetised steel.

The student is not allowed to scratch the paint from the rods.

Explain how the student could find out what each rod is made of.

Transparency: What should evidence look like?

You are having guests for dinner and want to try a new recipe.

Would you choose a recipe that had no illustration?

Transparency: What can evidence look like?

• Are the students provided with one or more annotated exemplars of particular genres expected?

• Which exemplars should be shared with students? SACE website materials, your school’s students, other schools’ students?

• As, Bs, and Cs? • Discuss then share effective ways these be provided, e.g.

paper, moodle, word walls.

3. Validity

“… the extent to which any assessment measures what it has been designed to measure.”

(Isaacs, 2013)

3. Validity

Is the assessment task designed to assess what is intended?Is it consistent with the:• learning and assessment plan (LAP)?• relevant content of the subject?• assessment task type description in the subject outline?• stated assessment design criteria?Are the stated criteria relevant and achievable?

Validity: Amending tasks

Should changes be made and recorded in the Addendum because:• a different task would better suit needs of the whole cohort• a different task would benefit the needs of one or more

students while still allowing them to meet the same criteria Also keep in mind the SACE ‘Special Provisions’ processes.

4. Flexibility

Within a subject:• Do your tasks collectively build students’ skills in

different modes (eg oral, slides, photo story, written argument)

• Do students have choices in mode for particular tasks that can equally show valued evidence of learning?

• How can you support students to learn a valuable range of modes?

Flexibility: Modes of evidence of learning

• Multimedia presentation• Portfolio• Round-table presentation• Written argument or analysis or report• Performance• Oral (live or pre-recorded)• Product (with oral or written explanation/annotations)• Interactive interview (with minimal teacher support as this may result

in a D in some subjects)

Flexibility: Skills and applications tasks (SATs)

Other than tests, what else could SATs be?

Flexibility: Tests and preparation for exams

Students do tests to prepare for exams in some subjects but they do not need to count in the 70% school assessment.Tests can be undertaken in pairs, small groups, and as individuals (as appropriate) to build skills in:• decoding multiple choice questions• learning from texts (open book)• collaborative problem-solving• writing short answers• preparing for exam pressureTeachers can reflect on how best to support students learning needs

Flexibility: Stage 2 Biology SAT ideas

• a data interpretation exercise• a multimedia product• an oral presentation • a practical demonstration• an extended response• a written assignment• multiple-choice questions• short-answer questions• a structured interview• a response to text(s).

5. Relevance

Is the task worthwhile and meaningful?

Relevance: Does the task allow/extend each student to build on what they already know, do and understand?

Source of cartoon unknown

RelevanceCan students provide input into tasks and suggest improvements?

RelevanceCould your student information management system provide indicators for students:• interests• passions• pathway• strengths• needs

Relevance and choices in texts

Murray Middleton, winner of the 2015 Vogel’s literary Award

“I hated reading growing up and finished second last in my year for literature in Year 12. But I found that once I was allowed to start choosing what to read, it became a lot more interesting.”

The Australian 21 April 2015, p3

Relevance: Career Pathway learning options• A colleague said he could see no point to maths until he started his

apprenticeship.• Do students have opportunities to learn the specific maths and

English required for their identified career pathways?• Community Studies can allow aspiring electricians to study the

specific maths they need without extra topics required in higher level maths subjects (they still need to also meet the numeracy requirement through eg Essential Mathematics).

• Discuss for 2 minutes how career pathways could inform your subject/s and clarify how you could find more information.

6. Reliability

6. ReliabilityReliable assessment design allows the student opportunity to:• understand the purpose and what the task requires• have learned the relevant content and mode

required• undertake the task in a supportive context (safe,

accessible sources/resources, minimal distraction)• be assessed consistently on the stated evidence

required (not by distractors)

Reliability

Is a task reliable if:• one or more students have had extended absences?• it allows students to achieve a C but not an A?• the reading level is beyond them?• there has not been adequate teaching and learning time for

some of the students?

Reliability

Are tasks designed collectively to provide reliable evidence of what students know, understand and can do?Do your tasks:• Have too many criteria (or too few)?• Expect too much writing or expected time for students to present?• Have a mismatch between the task description and criteria?• Provide templates or questions that support struggling students but

don’t allow others to provide creative and critical ‘A’ level evidence?

What the Research tells us

“… the most powerful single approach to improving student achievement is focusing on the use of minute-to-minute and day-to-day assessment to adjust teaching to meet student needs.”

Wiliam, 2005

6. Reliable

1. Equitable

2. Transparent

3. Valid

4. Flexible

5. Relevant

Fair and accessible assessment is …

Reviewing a Society and Culture Task

How could the assessment task (a real one from a school) be improved?

• Refer to the checklist ‘Considerations in structuring SACE assessment tasks’

• Have a go by yourself for 5 minutes, annotating possible areas for improvement

• In pairs, record your advice to the teacher to improve the task

This powerpoint, the task and a checklist can be downloaded at http://dlb.sa.edu.au/ctmoodle/ > ‘SACE Improvement Resources’ under the heading

‘Designing fair and accessible SACE assessment tasks’

Assessing an example of evidence

To what extent could the evidence provided on the next slide meet the current task requirements?• Have a go by yourself for 2-3 minutes then discuss with

colleagues to get to a shared understanding.• Do this with other student work evidence in future with

colleagues in the same or different learning area.

“When you see this photo, some of you may assume this photo was taken somewhere else in the Middle East… This is the uniqueness of Australia … People freely practice their culture without any pressure. And with my Timorese background where conflict and lack of resources limit my way to explore profoundly other people’s history and culture, sometimes I tell myself, buddy you’re lucky to be here, because you’ll learn various things beyond your course.”

http://www.meldmagazine.com.au/2011/05/melbourne-photo-story-competition-winners-announced/ downloaded 22-4-15

To what extent could this evidence meet the criteria?

Reviewing another Assessment Task

• Annotate the changes you would make• Exchange your annotated task sheet with a partner and review the

changes• Provide your partner with feedback about the improvements made

and any other suggestions you would make

Your thoughts….?

• What feedback do you have to inform a review of this professional learning activity?

• Please email adele.pring@sa.gov.au

Resources and bibliography

Topics with a focus on Aboriginal knowledge, cultures and perspectives (download from SACE website www.sace.sa.edu.au then search the title)

Isaacs, T., Zara, C., Herbert, G., Coombs, S.J., & Smith, C. (2013). Key concepts in education assessment. SAGEMasters, G.N. (2014). Assessment: getting to the essence. Assessment and Reporting Projects.Wiliam, D. (undated c. 2005). Construct-referenced assessment of authentic tasks: alternatives to norms and criteria.