Integrating LLN and the ACSF

into our Training & Assessment

To be effective as trainers/assessors we need to have an understanding of the

significance of language, literacy and numeracy (LLN) in learning and assessment

processes.

Numeracy

LLN Reading Writing

Oral communication

The LLN demands of the workplace for which

the learner is being prepared

The LLN demands we impose through

our training and assessment

materials and resources

The LLN skills of the learner

Three intersecting sets of LLN demands

on the learner

What do we need to know about LLN?

The session will also consider how aspects of the very complex Australian Core Skills Framework can be used by the general trainer/assessor to analyse, determine

and cater for each of these demands.

How do we establish the level of these demands?

A complex framework, designed primarily for “LLN specialists”

BUT, also a framework that can help the “ordinary”

trainer/assessor

ACSF Overview

The ACSF describes each of the five core skills [learning, reading, writing, oral communication, numeracy] across three interactive dimensions.

5 Core Skills

Five levels of performance ranging from 1 (low level performance) to 5 (high level performance) The five levels of performance represent milestones along a continuum of performance.

ACSF PERFORMANCE INDICATORS

Core Skill Indicator

Number Description

Learning

.01 Active awareness of self as a learner, planning and management of

learning

.02 Acquisition and application of practical strategies that facilitate

learning

Reading .03 Audience, purpose and meaning-making

.04 Reading strategies

Writing .05 Audience, purpose and meaning-making

.06 The mechanics of writing

Oral

Communication

.07 Speaking

.08 Listening

Numeracy

.09 Identifying mathematical information and meaning in activities and

texts

.10 Using and applying mathematical knowledge and problem solving

processes

.11 Communicating and representing mathematics

Indicator numbers

are consistent across core skill levels

e.g. Reading

1.03 / 1.04 2.03 / 2.04 3.03 / 3.04 4.03 / 4.04 5.03 / 5.04

Core skills are colour coded

ACSF Overview

Applying the ACSF

What are important reading tasks carried out by an assessor?

FOR EXAMPLE Reading and interpreting competency standards Reading attendance sheets Reading and interpreting assessment tools Reading and analysing documents supplied as evidence Reading and analysing evidence presented in different media Reading learner records sheets

ACSF Reading Levels

• The first Reading Indicator addresses audience, purpose and meaning-making. • The second Reading Indicator addresses reading strategies. • Specific Indicator statements describe exit performance at each level.

Reading attendance sheets 2.03 Reading learner records sheets 2.03

Reading strategies

Audience, purpose and

meaning making

Reading and interpreting competency standards 3.03 Reading and analysing documents supplied as evidence 3.03 Reading and interpreting assessment tools 3.03 Reading and analysing evidence presented in different media 3.04

Applying the ACSF

What are important writing tasks carried out by an assessor?

FOR EXAMPLE Completing learner records Filling out previously prepared checklists Writing judgements about evidence provided Using various communication strategies to record outcomes (e.g. text, charts, electronic)

The first Writing Indicator addresses audience, purpose and meaning-making. The second Writing Indicator addresses the mechanics of writing. Specific Indicator statements describe exit performance at each level.

ACSF Writing Levels

Completing learner records 1.05

Filling out previously prepared checklists 2.05/2.06

Writing judgements about evidence provided 4.05 Using various communication strategies to record outcomes (e.g. text, charts, electronic) 4.06

Establishing LLN Skill Levels

oral communication skills,

and numeracy skills,

required in your job.

We could do a similar mapping of the learning skills,

Note that we will probably end up with a “spiky profile”.

“Spiky” Profiles

This concept of “spiky profiles” draws our attention to the lack of correspondence

between AQF levels and ACSF levels.

ACSF AQF

More Profiles

We could do an analysis of the LLN demands of the training and assessment materials and processes we provide as part of our training context.

Hopefully we would find some match between these “spiky profiles” and the “spiky profile” of the LLN demands of the actual job.

Having established the LLN demands of the work environment and of the training and assessment context, we now need some understanding of the LLN skills of our learner – another “spiky profile”.

Learner LLN Skills

In particular, we need to be aware of mismatches between the learner “spiky profile” and the workplace and training contexts “spiky profiles”.

As “non-specialists” in LLN we are not able to carry out a

fine-grained analysis of each learner. However, we can use LLN assessment tools that will

help us make useful judgements.

Identifying Learner Skills

These tools need to better focused than generalised

reading, writing and maths tests.

Such tests don’t tell us if a learner will be able to deal with

the LLN demands of the workplace and training contexts.

Go back to the list of reading tasks we identified as being required by an assessor.

Let us assume that one of these was reading a report to gain some required information.

How could we establish if someone else would have the necessary reading skill to complete this task?

Developing a Better LLN Test

So we could build up a test of reading tasks directly related to the workplace and training contexts.

We could do the same thing with writing tasks, oral communication tasks, and

numeracy tasks.

We would then have a set of assessment tasks that would allow us to make

judgements of the extent to which our learners could cope with these tasks.

The outcomes of these tests would identify where the LLN

skills of learners are adequate

and where there are gaps in these skills.

Developing a Better LLN Test

So What?

This information might help us make adjustments

to our delivery and assessment processes so that

we could include the development of some of the

“gap” skills, or refer the learner to more

specialised help.

It might also help us identify where our own training and assessment materials and/or

processes do not match the LLN demands of the workplace/unit of competency.

Validated LLN Assessment Tools

Some of these will exist in your industry area. If not, existing tools may be able to be contextualised to address the LLN demands of your industry area.

If these LLN assessment tools are to have real value it is important that they are validated (with our peers and/or with specialists and/or with industry.)

Fortunately, a considerable amount of work has already been done in developing sets of such validated tools.

Some of these will work in your industry area. If not, existing tools may be able

to be contextualised to address the LLN demands

of your industry area.

Validated LLN Tools - Precision

http://www.precisionconsultancy.com.au/acs_framework

Validated LLN tools - VET-FEE HELP

'The testing tool must be an approved testing tool. The approved testing tool, available since 1 January 2016 and specified in the VET Guidelines is the Core Skills Profile for Adults (CSPA).’ Department of Education and Training, Update on VET FEE−HELP Reforms, 8 December 2015

Assessment of LLN testing instruments and processes for VET FEE-HELP providers Report (Stage 2)

• The provider must arrange to have their LLN assessment tool independently assessed. • An independent assessor who undertakes the audits of any LLN testing instruments

should be an independent body or approved independent and specialist personnel, such as an auditor with VET experience for details.

• The independent assessment must directly respond to the specifications and requirements for approval.

• Completed assessment must be sent to the Secretary, DET, care of the Branch Manager, VET FEE-HELP Branch for approval. This could be in electronic or paper form.

Very detailed requirements are included in pages 31 - 43

Validated Tools - VET-FEE HELP

Criterion 1: Validity • The assessment is based on quality assessment development processes • The learner undertakes the assessment independently (Also in Criterion 2) • The tasks and questions are validly mapped against the different ACSF levels (Also in Criteria 2, 3) • Information is provided about the performance or scores required to decide that a learner has

independently proved their competence at ACSF Exit Level (x) Criterion 2: Reliability

• Administration guidelines include details about who can mark the assessment if it is to be manually marked and a specified set of marking information and scoring rubrics are provided

• The marking and scoring guides or rubrics can be applied consistently and reliably across different assessors/markers

• Administration guidelines include clear details of the conditions under which the assessment should take place (Also in Criteria 3 and 4, plus reasonable adjustment)

Criterion 3: Fairness • The assessment tasks were tested and trialled with different groups of learners and checked for

performance and feedback gathered from different learners Criterion 4: Well-constructed

• The LLN assessment tool is well constructed with quality presentation and quality and clear sets of instructions for both the learners being assessed and for the administrators of the assessment

This list of criteria has been adapted from the ACER list for developing tools for VET FEE-HELP purposes. It is provided as a guide for developing less formal tools.

NOTE: VET FEE-HELP tools MUST meet ALL of the original requirements.

Validating your own LLN Assessment Tools

1

2

3

We have discussed LLN and

ACSF

We have identified the LLN demands of workplaces and

training materials and training processes.

We have looked at ways of identifying learner LLN levels,

using validated tools.

We have put all of this together

Dave Tout

Numeracy: A Core and Crucial Skill in VET