PHARE 2003-BG 2003/2004 937 05.03
Magda Kirsch, Yves Beernaert & Luk Indesteege
1
Methodological Guidelines for Assessment of Profiles by Vocation
October 2006
PHARE 2003-BG 2003/2004 937 05.03
Magda Kirsch, Yves Beernaert & Luk Indesteege
2
Objectives
• Methodological assistance
• For assessement of workers in the workplace
• Based on ‘learning outcoms’ from the national Bulgarian SER’s
• Linked to European VET policy
PHARE 2003-BG 2003/2004 937 05.03
Magda Kirsch, Yves Beernaert & Luk Indesteege
3
Definition of assessment
Assessment means :
the process of collecting evidence and making judgements on whether competency has been achieved to confirm that an individual can perform to the standard expected in the workplace as expressed in the relevant endorsed industry or enterprise competency standards or the learning outcomes of an accredited course.
PHARE 2003-BG 2003/2004 937 05.03
Magda Kirsch, Yves Beernaert & Luk Indesteege
4
Competences
A competence is an integrated set of knowledge, skills and attitudes that can be employed efficiently by the candidate
• Knowledge Knowing facts,understand
• Skills Being able, apply
• Attitudes Wanting, behave
PHARE 2003-BG 2003/2004 937 05.03
Magda Kirsch, Yves Beernaert & Luk Indesteege
5
Process of Assessment
When discussing the process of assessment of vocational competences we want to focus on – Why?– What? – How?– Who? – When?
PHARE 2003-BG 2003/2004 937 05.03
Magda Kirsch, Yves Beernaert & Luk Indesteege
6
Why assessment ?
• To check whether the workers meet the professional standards
• To certify workers achievements in their profession
• To diagnose the learning needs• To have feedback for future instruction• To select for the next stage in work• To encourage workers• To evaluate AP(E)L
PHARE 2003-BG 2003/2004 937 05.03
Magda Kirsch, Yves Beernaert & Luk Indesteege
7
What to Assess?
Classroom based assessment changes:
• Learning outcomes• Workplace based• Independent of
learning pathway
PHARE 2003-BG 2003/2004 937 05.03
Magda Kirsch, Yves Beernaert & Luk Indesteege
8
WHAT do we assess?
We assess the core competences described in the SER.A competence is an integrated set of knowledge, skills and attitudes that can be employed efficiently by the candidate
• Knowledge Knowing facts,understand• Skills Being able, apply• Attitudes Wanting, behave
PHARE 2003-BG 2003/2004 937 05.03
Magda Kirsch, Yves Beernaert & Luk Indesteege
9
WHAT do we assess?
• We assess the learning outcomes described in the SER.
• Learning outcomes are the basis for an assessment program that focuses on what workers have to do to be successful in their job.
• Assessment identifies what action the worker should be able to perform. It also denotes the conditions under which the workers should demonstrate mastery.
PHARE 2003-BG 2003/2004 937 05.03
Magda Kirsch, Yves Beernaert & Luk Indesteege
10
Example: Refrigeration technician in food industry Unit 1: Electrical maintenance
Core competences
Learning outcomes: To achieve this unit the learner should be able to:
1.1. Read electrical diagrams
To indicate the parts of electrical circuits on a diagram and in reality
To achieve electrical measurements based on diagrams
1.2. Operate switch-boards
To distinguish between main circuits and steering circuits To measure if a circuit is working correctly
1.3 Operate electrical drives
To connect a 3-phase AC motor correctly To explain the thermal and magnetic safety circuits on a drive
1.4 Work safely at electrical installations
To apply the safety signals correctly To switch in a safe way according the procedures To explain the different ground connection systems
PHARE 2003-BG 2003/2004 937 05.03
Magda Kirsch, Yves Beernaert & Luk Indesteege
11
WHAT do we assess?We can also assess the level of masteryLevel 1 - vocations involving routine activities performed
under routine conditions
Level 2 - a range of complex activities performed in changing conditions
Level 3 - a range of complex activities performed in changing conditions, as well as responsibilities for the performance of others
Level 4 - a wide range of complex activities performed in changing conditions, as well as managerial responsibilities for the performance of others and for allocation of resources.
PHARE 2003-BG 2003/2004 937 05.03
Magda Kirsch, Yves Beernaert & Luk Indesteege
12
HOW do we assess?
Principles of assessment
– based on understanding of workers’ learning + workplace– accommodate for individual differences (flexible)– clearly explained to workers (e.g. criteria)– valid and have a reliable and fair process – allow for timely feedback– allow learners and trainers to reflect on their learning
PHARE 2003-BG 2003/2004 937 05.03
Magda Kirsch, Yves Beernaert & Luk Indesteege
13
HOW do we assess?Key elements of assessment:• Assess in a congruent way; how the competence is implemented in the
professional situation: Plan, Do, Check, Act ;• Assess authenticity of products developed, of assignments or activities
carried out in real professional situations;• Assess on the basis of competence evidence demonstrated by the
worker and which can be compared with the SER;• Assess in an integrated way: theory and practice have to be assessed
together.• Criterion-related assessment: assessment is based on criteria taken
from the SER;• Assessment independent from the learning pathway: based solely on
the demonstration of the competences by the worker • Assessing in a specific professional context by a professional assessor
PHARE 2003-BG 2003/2004 937 05.03
Magda Kirsch, Yves Beernaert & Luk Indesteege
14
HOW do we assess?Assessment tools should be:
– Valid, really linked to the prfile
– Reliable, repeatable
– Practical, usable
PHARE 2003-BG 2003/2004 937 05.03
Magda Kirsch, Yves Beernaert & Luk Indesteege
15
Valid assessment?To be valid, assessment tools should:
– Be appropriate to the SER– match the type of skills listed in the LO– assess a broad range of the LO– have a weighting that reflects the relative
importance of the range of skills in the specific LO. – have a good balance between theory and practice
PHARE 2003-BG 2003/2004 937 05.03
Magda Kirsch, Yves Beernaert & Luk Indesteege
16
Reliable and practical assessment?
Reliable Consistant results with different candidates Consistant results with different assessors
PracticalTakes account of available resources and
equipment Uses an acceptable amount of time
PHARE 2003-BG 2003/2004 937 05.03
Magda Kirsch, Yves Beernaert & Luk Indesteege
17
Assessment tools 1. Observation in real or live work
situation
2. Simulation3. A brief case (scenario) focusing on
action and activity in a context 4. An extended case or scenario 5. Assignments for presentations 6. 360° feedback 7. Portfolio
PHARE 2003-BG 2003/2004 937 05.03
Magda Kirsch, Yves Beernaert & Luk Indesteege
18
360° feedback
Assessee
CustomerLevel+1
Peer
level-1suplier.Level-1
Peer
Level+1
PHARE 2003-BG 2003/2004 937 05.03
Magda Kirsch, Yves Beernaert & Luk Indesteege
19
Assessment tools Assessment tools :
– Checklists– Questions and Answers– Multiple choice questions– True/false questions– Observation in real or live work situation – Simulations– Cases– Scenarios– ……
PHARE 2003-BG 2003/2004 937 05.03
Magda Kirsch, Yves Beernaert & Luk Indesteege
20
Important remarks on HOW to assess
• Assessment tools or techniques should be linked to the learning outcomes
• Attitudes and personal competences are difficult to assess because of subjectivity Therefore use a measuring scale
• Assess those personal competences that are relevant
PHARE 2003-BG 2003/2004 937 05.03
Magda Kirsch, Yves Beernaert & Luk Indesteege
21
Linking LO to assessment criteria
Learning outcomes
Methods, tools
Features Issues Criteria
Practical skills• making and producing
things• manipula-ting materials
and objects
-Observation
-Micro tasks (smaller components assessed separately and built up over time)
- Checklists
- Practical tests
- Progress sheets
- Scenarios
Direct and indirect evidence
- Assessment increasingly involves workplace assessors and joint -assessment decisions.
- Criteria need to be discussed among assessors.
-Range and scope of practical skills are specified
- Simplicity or complexity of materials
- Working to time scales and deadlines
PHARE 2003-BG 2003/2004 937 05.03
Magda Kirsch, Yves Beernaert & Luk Indesteege
22
SAM: Scale for Attitude Measurement example 1.1 initiative
Insufficient Low Good Very good
Notices no work and
takes no initiative.
Notices only work
when the manager is
around and takes
initiative in that case.
Notices werk, and
takes initiative only
after some trigger.
Notices work and takes
initiative
spontaneously.
Has only attention for
assignments that she/ he
likes.
Executes sometimes
assignments that
he/ she doesn’t like.
Executes all
assignments with
quality and devotion
Executes all
assignments
spontaneously and with
devotion, even those
he/ she doesn’t like.
Waits for others to take
action and does nothing
him/ herself.
Takes only action
when told to or when
the manager is around
Takes action
spontaneously after
assignment of tasks
that are part of the
job description.
Takes action
spontaneously after
assignment of tasks,
even if they are no part
of the job description.
Notices no problems,
mentions no problems.
Notices problems, but
don’t react.
Notices problems and
mentions them.
Notices problems and
looks spontaneously for
solutions if possible.
PHARE 2003-BG 2003/2004 937 05.03
Magda Kirsch, Yves Beernaert & Luk Indesteege
23
How to assess with scenarios
1. Group the competences into families of situations with professional significance
2. Select a family of situations which can be integrated into a scenario and its components
3. Choose an integrative situation in relation with the selected family of situations
4. Decide on the criteria and the indicators5. Decide on the weighing to be used6. Make references to all the competences involved
PHARE 2003-BG 2003/2004 937 05.03
Magda Kirsch, Yves Beernaert & Luk Indesteege
24
WHEN do we assess?
Assessing is a continuous process
For strategic reasons we distinguish:– Diagnostic assessment: before entry of procedure– Recognition : AP(E)L– Formative assessment; continuous, more
process than event;– Summative assessment; at the end of period of
assessment
PHARE 2003-BG 2003/2004 937 05.03
Magda Kirsch, Yves Beernaert & Luk Indesteege
25
Accreditation of PRIOR …..
• Accreditation of prior learning (APL)• Accreditation of prior certificated
learning (APCL)• Accreditation of prior experiential
learning (APEL)• Accreditation of prior certificated and/or
experiential learning (AP E/C L)• Accreditation of prior learning and
achievement (APL&A)
PHARE 2003-BG 2003/2004 937 05.03
Magda Kirsch, Yves Beernaert & Luk Indesteege
26
Importance of AP(E)L in assessment
• Recognition of experience
• Allowing people without formal qualifications to acquire qualifications
• Enhancing the chances of low-skilled or workers
• Allowing career changes
Respect European principles for validation of non-formal and informal learning
PHARE 2003-BG 2003/2004 937 05.03
Magda Kirsch, Yves Beernaert & Luk Indesteege
27
Common European Principlesfor the identification and validation
of non-formal and informal learning
Elements of the identification and validation of non-formal and informal learning
• Individual entitlements
• Obligations of stakeholders
• Confidence and trust
• Credibility and legitimacy
PHARE 2003-BG 2003/2004 937 05.03
Magda Kirsch, Yves Beernaert & Luk Indesteege
28
Who is involved in assessment
of competences in the workplace?
• the worker himself (self-assessment)• his peers (peer-assessment) • mentors in the company• accredited assessors : experienced practitioners
who are present during practical tests or to assess portfolios of workers
The assessment procedure is controlled by a verifier from the ministry of labour, the ministry of education or the leading industrial body
PHARE 2003-BG 2003/2004 937 05.03
Magda Kirsch, Yves Beernaert & Luk Indesteege
29
Example of an assessment gridAssessed competence : Name of the candidate :
Performance criteria
Method of assessment
Details of evidence
Successful or not
Assessors Date
1.1
1.2
1.3
1.4
PHARE 2003-BG 2003/2004 937 05.03
Magda Kirsch, Yves Beernaert & Luk Indesteege
30
Top Related