UK Masters Dissertation DCAF DeVries P2
-
Upload
hans-de-vries -
Category
Documents
-
view
82 -
download
0
Transcript of UK Masters Dissertation DCAF DeVries P2
Masters Dissertation
Digital Competence-based Assessment Frame A conceptual design
Author: Johannes de Vries
Date: 31 August 2006
Master of Science in E-Learning, Multimedia & Consultancy
School of Education, Sheffield Hallam University (UK)
HAN University for professional Education (NL)
Digital Competence-based Assessment Frame
Dissertation for Master of Science - ii - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
MASTERS DISSERTATION Master of Science in E-Learning, Multimedia & Consultancy School of Education, Sheffield Hallam University (UK) HAN University for professional Education (NL) Author : Johannes de Vries Profession : Lecturer Communication & Information services School of Communication, Information technology and Information management, Location Deventer Task : Chairman of the Workgroup Implementation Core instruments for the project Personal Learning Process Saxion Universities of applied Science
Supervisor : Wim van Beek
Local Co-ordinator : Steef Woldinga
ABSTRACT This master’s dissertation aims to design a concept of a Digital Competence-based
Assessment Frame (DCAF) in Higher Education. In setting this Frame, attention will be paid
to the following subjects:
• Competence-based education
• Student-centred/demand-centred learning
• Assessment
• Project-based education
• Computer Supported Collaborative Learning
The EA Model (Educational Analysis Model) will be instrumental both in describing
competence-based learning and in constructing the assessment frame.
This EA Model came into being during the preliminary investigation of this study. When the
outlines of the object of study were drawn, three aspects gave shape to EA cube as a model,
namely, level of approach, individuals involved and processes within the educational
environment
The concept of the DCAF as developed in this dissertation aims to define the functional
requirements for the DCAF and its implementation. This study has been done at the Saxion
University of Professional Education, and within the Deventer School of CII in particular.
KEY WORDS:
Competence-based Learning, Assessment, competence-based assessment, student-centred
learning, assessment portfolio, digital assessment
Dissertation for Master of Science - iii - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
Dissertation for Master of Science - iv - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
Manuel Castells
Furthermore, Internet-based learning is not only a matter of technological proficiency: it changes the kind of education that is required both to work on the Internet and to develop
learning ability in an Internet-based economy and society. The critical matter is to shift from learning to learn-to-learn, at most information is on-line, and what really required is the skill
to decide what to look for, how to retrieve it, how to process it, and how to use it for the specific task that prompted the search for information.
In other words,
the new learning is oriented toward the development of the educational capacity to transform information into knowledge and knowledge into action.
(USA-2001)
Dissertation for Master of Science - v - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
Dissertation for Master of Science - vi - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
ACKNOWLEDGEMENTS A word of thanks
Many have consciously or unconsciously contributed to the realisation of my Master’s Dissertation.
I fear I forget to mention some people in the list below. I sincerely apologise for that. JO,
thank you for your patience and support, and also Agaath and Jeroen.
My supervisor Wim van Beek has been so patient as to encourage me to write this dissertation. He made me believe I could do it. I want to express my thanks for that, Wim. Several people were kind enough to let me interview them: Carla Doumen, Fokkelien von Meyenfeldt, Gerard Straetmans, Harm Abel Kunnen, Itje Wijshake-Bult, Marion Keiren, Nico Reizevoort, Peter Schuszler, Siete Akker, Ton de Bruyn, Wijnand Aalderink, Wouter Schoonman. I thank you all for your efforts. My fellow students helped me to give shape to this dissertation during many discussions; they contributed to the completion of the EA Model. My thanks to all of them, and to my travelling companion Rein Vrielink in particular; our debates in the car before and after the local meetings never ended. The School of CII supported this dissertation in many ways. It provided me with the financial means, supplied me with a research environment, and gave me the opportunity to develop the RAS concept. Restricting myself to mentioning Jan te Hennepe only, I would like to thank the management for their directions in the background. I would also like to express my gratitude towards Ton de Bruijn for his straight and sometimes provoking support. The students Eduard Haaijer and Kristian Kuiper trusted me to develop the RAS together with them; I thank them for their efforts. A further thanks to my office mates Margriet de Vos and Geert Balsma, my colleagues and fellow-students who participated in many tests and investigations. Many members of the Saxion seminars (DLWO2 & PLW & DPF & WIK) have contributed unwittingly to the realisation of this dissertation. I mention: Alexander Kremers, Co Braspenning, Mirjam Aartsen, Henny Groot Zwaaftink, Heino Logtenberg, Jonny van Egmond, Machteld de Roos, Marijke Kling, Melanie van der Wee, Monica Buijink, Paul Hilferink, Piet Hendriks. I want to thank them too. Finally, I want to thank translator Stephan Severin for his contributions. Apart from taking care of the translation, you have given me editorial feedback as well. Hans de Vries
Agaath de Vries; Design of the cover.
Dissertation for Master of Science - vii - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
Dissertation for Master of Science - viii - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
SUMMARY
THE OBJECT OF STUDY This study is a first attempt to define the functional requirements to be met by a Digital
Assessment Frame. This study was done from a perspective of developments in Higher
Education towards competence-based learning and assessment of independently learning
students. The DCA Frame was developed and realised specifically within the courses
Information Management (BI) and Information Service- and Management (IDM) of the
Saxion School of Communication, Information Technology and Information Management in
Enschede, Deventer and Apeldoorn.
The courses BI and IDM introduced competence-based learning in 2003. This introduction
ignited an educational reform process. The instructive nature of lessons and practicals is
disregarded more and more, whereas coaching of student groups around a professional
semester theme prevails.
This study aims to develop a concept for a DCA Frame, which visualises testing and
assessment in order to guarantee the personal learning process of a student within a changing
learning environment.
METHOD
The following methods and techniques have been used to describe the concept of a DCAF
sytem:
• Secondary literature was consulted;
• A case study was done of a previous project involving Route Assessment System
within the Deventer School of CII;
• Several persons considered being experts in the professional field were interviewed.
The results were processed in the EA Model, the Educational Analysis Model (De Vries,
2005), which represents the learning environment in a cube diagram. The EA Model is
tripartite:
• Approach levels: Macro – Meso – Micro Levels
• Persons involved: Student – Lecturer – Supportive Personnel Participants
• Processes: Coaching – Learning – Assessment Processes
The EA model serves as a framework in which the requirements for a DCA frame take shape.
Dissertation for Master of Science - ix - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
FOCUS
Setting up the requirements for a DCAF involved an explorative/qualitative research method
and can therefore be seen as an Extended Curriculum Development project. Three research
questions were formulated that constituted the focus of this study:
• On micro level, how can a student-centred assessment process be digitally realised;
and how should the roles of lecturers and students come into the conceptual model;
and finally, how should a course implement this process?
• On meso level, how is the assessment process to be supported and facilitated in order
to register an actively and collaboratively learning student’s development in
informational records and systems?
• On macro level, how should a guarantee as towards the quality of student-centred
learning be defined within the frames of a legislative policy?
EA MODEL
The EA Model is the framework describing the very aspects of the learning environment
which define the context of the DCA Frame. The EA model, then, deals with several
participants, such as the students, the lecturers and supportive personnel. Further, three
processes can be distinguished, namely, coaching, transfer of educational content, and testing
and assessment. According to a sociological approach, three levels are to be discerned in an
educational organisation and its environment:
- on the shop-floor, interaction between participants takes place on micro level
- on meso level, it is of major importance for matters to be organised adequately
throughout the course
- a civil organisation usually certifies a qualification on macro level.
See for EA Model; diagram 1: Educational Analysis Model – EA Model (De Vries, 2005)
CONTEXT
Considering the reforms in Higher Education and their social origins, (we see that more
flexible and broadly competent professionals should be supplied by educational organisations;
On a European level this can partly be realised by introducing the BaMa structure – the
Anglo-Saxon educational model - for all member states of the EU. In this way, the
disappearing borders within the EU as well as the debate about the connection between
Dissertation for Master of Science - x - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
education and job market are anticipated. From a bird’s eye view we can distinguish several
parallel paradigm shifts:
From an industrial to an information society
From old reproductive learning to new constructive learning
From lecturer/offer centred to student/demand-centred
From instructive transfer to knowledge acquisition
From testing subjects separately to an integral assessment focused on profession
From Face-to-Face (F2F) to Computer Mediated Communication (CMC)
From closed digital applications to an open and flexible learning environment
( Dochy & Bouwens, 1992, Kayzel, 2004, Hoeksema, 2002)
The reforms in the educational field are called “The New Learning”, and can be specified as
“the four Cs listed below:”
• Competencies: the student works independently and tries to acquire his professional
domain competencies;
• Constructivism: it’s the student himself who constructs and expands his knowledge,
capacities and professional patterns of behaviour;
• Collaborative: students collaborate on professional tasks and products in an authentic
and equal learning environment;
• Coaching: the lecturer’s roles come out well; he coaches a team of collaborative
students and supervises each student’s personal learning process at the same time.
(Jochems, Merriënboer, Koper en Bastiaens, 2005)
A COMPETENCE-BASED CURRICULUM
The framework of the EA Model defined the elements relevant to competence-based Higher
Education.
LEVELS
The learning environment in Higher education was described using three approach levels:
macro, meso and micro. The broader context of political and social developments on macro
level was a first point of attention. Mentioned was the European unanimous policy laid down
in Bologna, which decided for the introduction of a new Bachelor/Master model equal for all
member states. Ever since the Bologna meeting educational reforms have started in all
Dissertation for Master of Science - xi - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
European countries involved. In terms of Dutch education policy, this implies all courses have
had their quality evaluated and accredited by civil institutions
On meso level we see that Saxion University decided to set up a competence-based
curriculum along the lines of their innovated education policy laid down in 2000. In the new
curriculum the concept of actively and independently learning students is pivotal. The newly
developed model of the Personal Learning Process (Du.: PLW) serves to facilitate the
students’ demand-centred learning. On micro level, the Deventer School of CII attempts to
realise the PLW in a thematic, project-centred learning environment, where students can
acquire their competencies by working on tasks similar to the ones they will encounter in
future professional situations.
PARTICIPANTS Competence-based learning brings along several changes for those involved. As a demanding
client, the student is expected to give shape to his personal learning needs. In this, a lecturer,
who has transformed from instructor into coach, will guide him. ICT supporting tools will be
used progressively more; implying, a lecturer’s didactical role will change accordingly. In
order to make these processes possible, the roles of supportive personnel change as well.
Previously, support functioned as a complementary means for the lecturer in his role as
instructor, and educational tools and accommodations were taken care of by non-educational
assistants. Nowadays, it is the student who needs to be facilitated. This means the student
must be provided for sufficiently, so that he will be able to set up his education the way he
wants it. The latest ICT tools and applications, then, will have to be focused on the
demanding student.
PROCESSES As mentioned in the section Context, processes in competence-based learning take a different
course from the old way. Lecturer’s roles will change and gain additional aspects. Apart from
being a subject teacher, a lecturer will now have to assume several other roles, for instance,
coach of a student’s personal learning project and E-moderator.
A learning environment must meet the requirements of the students’ personal learning
processes more constructively; this means utilising more differentiated educational methods.
More often than not, students will collaborate in varying groups. Therefore, ICT supplies are
necessary to enhance digitally supported collaborative learning. If the student has the
opportunity to choose the competencies to be acquired in the course of his personal learning
Dissertation for Master of Science - xii - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
process, assessment will have to be geared to his choices. As a consequence, various forms of
evaluation will be used for assessing his competencies.
THE DCAF CONCEPT
The Digital Competence-based Assessment Frame can be seen as a structure which translates
the processes explained in the EA Model - coaching, Education and assessment – into a
concept.
See for scheme, diagram 19: Concept DCAF-system
The DCAF concept has been represented as a system similar to Koper’s domain model for e-
learning. Koper states it is urgent to have a clear perception of where and how the model is
used in order to understand e-learning. For education, the model should be a system
supporting the ‘primary educational functions’ (Koper, 2005). This DCAF systems aims to
visualise the functional requirements for the concept.
The student sets up his Personal Development Plan at the start of his course. He decides
which competencies he wants to develop per semester. It is these competencies the student
will be evaluated on, which is confirmed by the assessment frame.
The student himself decides what kind of learning environment will be best suited for him to
develop the competencies he selected. He may opt for working on professional tasks/products
in an educational workshop (Du. OWP), or choose for an external company The student is
focused on gathering evidence which indicates he acquired the competencies defined in his
Personal Development Plan. Formative instruments to be used may include, for instance:
diagnostic knowledge tests, 360-degrees-feedback questions regarding actions, and
conditional testing of performance tasks/products. RAS, the Route Assessment System, may
serve as an instrument to gather the formative feedback moments. Eventually, all kinds of
formatively tested files can be collected in a digital portfolio.
As soon as the student thinks he is able to meet the requirements of his selected competencies,
he can request an assessment. Nature of his competency and/or professional product will
determine the kind of assessment to be applied, such as there are, an overall test, a simulation,
or an oral assessment of his portfolio. Reliability of the latter may be improved by means of
the Protocol Portfolio Scoring.
Dissertation for Master of Science - xiii - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
CONCLUSIONS
The digital consequences of the DCAF Frame have been specified as functional requirements.
Other points of interest emerging from the study can be mentioned:
The EA model may be used as a model describing a learning environment. It served as a
thread within this dissertation. It would be advisable to extend investigations of the EA model
and explain interrelations between the cube diagrams in terms of transfer processes.
Realisation of the DCAF system into a concept positively defined the successive processes of
Coaching – Education – Assessment as a logical sequence.
With a view to the latest developments in e-learning and collaborative learning, the functional
requirements as described in the DCAF Frame might be extended to the Sakai open learning
source environment.
The suggestions above may generate new projects, which might further develop this
explorative DCAF system.
ACKNOWLEDGEMENTS This master’s dissertation could not have been accomplished without the opportunities offered
by my immediate educational environment. Support and assistance came from students,
lecturers as well as management of the courses BI & IDM in Deventer.
By actively participating in educational reforms at the Deventer School of CII I was able to
assimilate my observations and notes, and present them as new knowledge in this dissertation
Johannes de Vries, August 2006.
Dissertation for Master of Science - xiv - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
Dissertation for Master of Science - xv - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
INDEX
Abstract...........................................................................................................................iii Acknowledgement – A word of thanks.............................................................. vii Summary......................................................................................................................... ix Index .............................................................................................................................xvi 1. Introduction............................................................................................................1 2. Design ....................................................................................................................3 2.1. Aims ......................................................................................................................3 2.1.1. Sectors ...................................................................................................................3 2.1.2. Frame .....................................................................................................................4 2.2. Sketch of the situation ...........................................................................................5 2.2.1. Saxion Universities ...............................................................................................6 2.2.2. School of CII Deventer .........................................................................................7 3. Methodology ..........................................................................................................9 3.1. The EA Model .......................................................................................................9 3.1.1. Participants........................................................................................................... 11 3.1.2. Processes ............................................................................................................. 12 3.1.3. Levels ...................................................................................................................13 3.2. Method .................................................................................................................13 3.2.1. Secondary literature .............................................................................................14 3.2.2. Case study ............................................................................................................14 3.2.3. Interviews key figures..........................................................................................15 4. Levels ...................................................................................................................17 4.1. Macro ................................................................................................................... 18 4.1.1. Politics and education ..........................................................................................19 4.1.2. Competencies as intermediaries between education and job market ...................20 4.1.3. Educational policies and higher education...........................................................21 4.2. Meso.....................................................................................................................21 4.2.1. Saxion and self-regulated learning.......................................................................22 4.2.2. Saxion & Assessment...........................................................................................24 4.3. Micro....................................................................................................................25 4.3.1. The educational model of the courses BI & IDM ................................................25 4.3.2. Project-based learning..........................................................................................26 5. Participants...........................................................................................................27 5.1. Student ................................................................................................................. 27 5.1.1. Digital expectations..............................................................................................31 5.1.2. The digital student / The Net-Generation.............................................................31 5.2. Lecturer ................................................................................................................32 5.2.1. The new roles .......................................................................................................33 5.3. Support.................................................................................................................35 5.3.1. Processes seen from support’s point of view .......................................................36 6. Process .................................................................................................................41 6.1. Coaching ..............................................................................................................41 6.1.1. PLW Instruments/Tools ......................................................................................43 6.1.1.1. Digital Portfolio e-Portfolio .................................................................................44
Dissertation for Master of Science - xvi - Hans de Vries E-Learning, Multimedia and Consultancy School of CII
6.1.2. E-Coaching ..........................................................................................................46
Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
6.2. Education .............................................................................................................49 6.2.1. Competencies .......................................................................................................50 6.2.2. Constructivism .....................................................................................................52 6.2.3. Collaborative........................................................................................................54 6.2.4. ELO / DLO / VLE................................................................................................56 6.3. Assessment...........................................................................................................59 6.3.1. Assessment and competence-based education .....................................................59 6.3.2. Assessment in Higher Education ........................................................................60 6.3.3. Case study CII Deventer ......................................................................................62 7. Digital Competence-based Assessment Frame ....................................................65 7.1. Foundation of the DCAF......................................................................................65 7.2. The DCAF-system ...............................................................................................68 7.2.1. Describing the DCAF concept .............................................................................69 7.2.2. The DCAF and the learning cycle........................................................................76 7.3. DCAF, the PPS and the 4C/ID.............................................................................77 7.3.1. PPS and the DCAF...............................................................................................79 7.3.2. PPS and 4C/ID in combination with the DCAF...................................................81 8. Conclusions and recommendations......................................................................85 8.1. Considering the EA Model ...............................................................................85 8.2. Possibilities for the DCAF ................................................................................86
Definitions............................................................................................................88 Diagram / Tables..................................................................................................90 References............................................................................................................91 APPENDIX
I. Key figures research a. Interview protocol (Dutch) b. Item Analysis (Dutch)
II. Case study of the RAS in the school of CII Deventer (Dutch) III. Conceptual description of the DCAF (Dutch)
Dissertation for Master of Science - xvii - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
Dissertation for Master of Science - xviii - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
1. INTRODUCTION
In recent years, Higher Education in the Netherlands has been the scene of action for many
reforms concerning the issue of competence-based learning and working. Towards the end of
the nineties, debates about the introduction of competence-based education were instigated by
business. This was signalled and described by Everwijn: “In fact, business urges education to
prepare students for a rapidly changing society which is progressively getting more complex.”
He continues, “competence-based learning aims for students to develop capacities and
knowledge indispensable for functioning in a dynamic-complex society” (Everwijn, 1999)
In order to determine if a student has sufficiently mastered the competencies of any
professional profile, a course will have to evaluate him. If an evaluation indicates a student
meets the requirements, he will be able to enter the job market as a fully qualified
professional. In their study, “Competencies: from Complications to Compromise”, The
Education Council defined it as follows:
“Success of competence-based learning in Higher Education will mainly depend on
the ability of educational institutes to bring evaluation in line with this new
approach”. (Onderwijsraad, 2002)
This study attempts to establish a competence-based evaluation for Higher Education.
In the world of education, the English word “Assessment” has become a common term,
showing distinctively these evaluations involve more than a mere testing of learned subject
material. It is essential to determine if a student knows how to act professionally in a relevant
context. The frame developed in this study departs from the perspective of a competence-
based learning environment; and as education cannot ignore the influence of ICT
advancements on society, the functional requirements for a digital realisation of the
Competence-based Assessment Frame (CAF) will be formulated as well. Subsequently, the
concept of Digital Competence-based Assessment Frame (DCAF) can be described.
Construction of the DCAF Frame implied getting a clear perception of competence-based
learning in advance; this in order to decide how testing and evaluation – assessment – may be
realised to the full. The concept of the Educational Analysis Model (EA Model) serves to
describe the relevant processes. At the same time, the question whether the model is
applicable at all is carefully taken into account.
The Saxion School of Communication, Information Technology and Information
Management (CII) formed the background for this study.
Dissertation for Master of Science - 1 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
In chapter 2, set up and situation of the DCAF are described; goals and research questions for
this study are formulated. The situation at Saxion University is described and the courses BI
and IDM are explained into further detail.
Chapter 3 deals with methodology, the chosen investigational approach and working methods
employed. The frames of the EA model, namely, the three approaches Levels, Participants
and Processes will be described.
In chapters 4,5 and 6 the three ways of approach, c.q. Frames of the EA model are developed;
point of departure is the situation at Saxion Universities and the courses at The Deventer
School of CII. The micro, meso and macro levels of the broad educational field are described
in chapter 4. A survey of those involved in the educational field, such as students, lecturers
and supportive personnel and their respective functions, is presented in chapter 5.
In chapter 6 the three distinctive processes – Coaching, Education and Assessment – are
investigated. Chapter 7 presents the DCAF concept and defines its functional requirements.
Finally, conclusions and results lead to advice: chapter 8 suggests how to proceed from the
issues mentioned in this dissertation.
The descriptions have been based on the context of Saxion Universities and the courses
Information Management and Information Service- and Management. However, this does not
imply descriptions and findings are in accordance with educational policy at Saxion
Universities; they are theoretic elaborations to construct the DCAF
Note: “He” in this dissertation includes “She”. For stylistic reasons, I decided for the
masculine form of the singular pronoun.
Dissertation for Master of Science - 2 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
2. DESIGN
Any student who starts a study aims to acquire the certified qualifications that come with the
intended educational courses. For taking his bachelor’s degree, a student will get several
forms of feedback to digest, and he will be frequently tested and assessed. All this helps to
determine if the student meets the criteria laid down for the professional profile of the
education. There are several forms of tests and assessments; however, which are the ones
most suited to determine a student’s capacities? In Competence Based Learning, Assessment
is used to determine and evaluate (assess) the student’s acquired competencies.
This study intends to formulate an Assessment Frame for Competence Based Learning;
moreover, it proposes how a digital interpretation of the Assessment frame can be realised.
2. 1. AIMS
This study aims to produce a research report determining functional conditions for a Digital
Competence Based Assessment frame. The aims of this study are as follows:
• Indicating the functional conditions which are to be created for the realisation of a
draft model for a Digital Competence Based Assessment Frame in a learning
environment controlled by students;
• Indicating how this Frame supports students who actively learn together in an
effective and efficient way;
• Defining the functions of students and teachers in this environment.
The Aims clearly state that the learning environment is student-controlled and competence
based. Starting point is the educational concept of collaborative learning.
2. 1. 1. SECTORS
Starting point for specifying the aims for this study are those aspects that matter in an
assessment process. In a student-controlled learning environment, it is basically the student
who directs the course learning takes. The teachers are there to answer students’ questions
and can function as coaches, or as professional teachers, or as assessors. Further, the
educational organisation should facilitate both learning environment and opportunities for
testing and assessing. Next, it should be possible to determine how the implications of an
assessment process can be realised on several levels, such as there are on Macro Level, the
governmental decision-making regarding assessment methods within the legally defined
Dissertation for Master of Science - 3 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
frames in educational policy. On Meso Level, the processing of the legislative decisions
within the educational organisation of a college/school. Which area of attention is selected to
translate these into an innovative and organisationally well-functioning policy? On Micro
Level, how is this policy to be implemented within an institute’s educations, and how is it
actually put into practice.
Taking the above into account, an attempt will be made to fill in the functional requirements
of the Assessment Frame. Sectors can be defined more specifically, thus reaching matters
with respect to content, c.q. questions per sector regarding a definition of the requirements a
digital assessment process should meet. The approach mentioned above is the basis for my
forming three questions per level, each of which can be divided further into functions and
processes.
• How is a student-controlled assessment process to be realised digitally, and how
should the functions of teachers and students fit into a conceptual model; how can an
education actualise this process at micro level;
• How is the assessment process to be supported and facilitated at meso level in order to
register the advancement of an actively and collaboratively learning student in
informational records and systems;
• On macro level, how should a guarantee as towards the quality of student-centred
learning be defined within the frames of a legislative policy.
The functional requirements for the Digital Competence-based Assessment Frame (DCAF)
will be defined by means of secondary literature, interviews with persons involved, and the
interpretation of case studies with the educations at School CII in Deventer.
2. 1. 2. FRAME
For the sake of clarity I have decided for the term “frame”. This term can be explained in
several ways. For my Masters Dissertation I proceed from the meaning, “framework”,
“scheme” or “cadre”. These words imply filling in a part of demarcated reality on a
conceptual level: the Competence-Based Assessment, focusing on digital realisation.
I found the term “model” to have a wider meaning. It implies the total form, or a filled-in,
finished constitution of a part of reality. As an example I mention the still to be accomplished
EA Model (Educational Analysis Model), which can be seen as a representative analysis
model of educational reality. The EA Model serves a descriptive model in this dissertation
Dissertation for Master of Science - 4 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
On a higher level of abstraction we encounter the term “paradigm”, which has a linguistic and
philosophical explanation. Philosophically, the word “paradigm” implies a coherent system.
In this context, the so-called “paradigm shift” taking place in education from teacher- to
student-centred education is often referred to.
2. 2. SKETCH OF THE SITUATION
In the 90-ties it was concluded that Higher Education should fit in with the development on
the job market and other tendencies in society. Dochy and Nickmans, for instance, write the
following: “From the 80-ies and onwards, the global shifts of mass production within
economy have given rise to a refocusing on quality; from the 90-ties, more attention has been
paid to the learning of skills.” (Dochy and Nickmans, 2005). Basically, the question of
vocational education fitting in with the job market is a recurrent subject of discussion: society
changes and technological developments rapidly succeed one another. The role of work
depends on the technology used within companies; meaning, students will have to adapt to the
latest technological applications in order to be able to start on the job market as qualitatively
useful employees. In order to realise the better connection, the job market demands for
education to meet two essential requirements, namely:
1. Considering the flexibility of the future employee;
2. Considering the training of wider capacities that fit in with professional areas and their
progressively technological developments (Dochy and Nickmans, 2005)
Higher Education attempts to meet these requests by filling in the premises for competence-
based learning. It aims at turning out a starting employee who is more flexible, can cover
several disciplines, and has sufficient applied knowledge, skills and even patterns of
behaviour for future professional areas.
New educational perceptions have an essential function in defining these questions. The
diagram below shows the shift of focus in education:
Dissertation for Master of Science - 5 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
Traditional: ‘The Old way of Learning” New : ‘The New Way of Learning’ Education is knowledge and ability Objective knowledge Theory first, practice after Transfer, transmission Transfer of knowledge Subject centred, disintegrated Supply-controlled Lectures curriculum Passive students Learning individually Teacher-centred Directing and teaching Abstract, generalising
Education is knowledge and capability Subjective interpretation Practical skills integrated Independence, constructive Acquisition of knowledge Integrated, actual Demand-controlled Study programmes Active students Collaborative learning Student-centred Coaching and assessing In context, actual
Table 1: Changed insights in education (Dochy & Bouwens, 1992; Kayzel, 2004)
In educational literature mention is made of the ‘paradigm switch’ from a supply/teacher-
controlled learning environment to a demand/student centred one.
2. 2. 1. SAXION UNIVERSITIES
In 2000, the changed insights in learning took shape in a new educational policy for the
Saxion Schools too. The cornerstones of this vision can be identified as follows:
• Education aims for students to acquire competence, focused on a level of Higher
Education (HBO) professional practice
• Education provides for a learning environment for independently and actively
learning students, with an emphasis on acquiring competence
• The educations of the Saxion School offer opportunities for students to set up their
training course individually along the lines of the personal learning process
(Saxion Educational Policy, 2000).
The generally recognised paradigm switch is distinguishable in the actualisation of this vision.
All colleges were assigned to set up their educations in accordance with the vision as
described above. Fourteen colleges with a total amounting to fifty-five Bachelor’s and six
Master’s educations are involved.
Dissertation for Master of Science - 6 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
2. 2. 2. SCHOOL OF CII DEVENTER
The Saxion School of Communication, Information Technology and Information
Management (CII) has two educations in Deventer, to be taken either full-time (as day
courses) or part-time (as evening courses). The two educations involved are Information
Management and Information Service- and Management. As from the school year 2003-2004
both Deventer educations have changed their curriculum into competence-based learning, and
attempted to realise the New Learning within the frame of the Saxion Educational Vision.
The educational concept as employed by the educations mentioned above is based on
‘Adaptive Learning’. The three cornerstones of Saxion education - competence-based
learning, independently and actively learning students, and the personal learning process -
have been fully realised. With ‘Adaptive Learning’, (Blok, 2002) an educational learning
situation is created that purposefully aims to take the differences between students into
account, and tunes them in to delivering several professional products.
As for another important aspect, it was decided both educations should employ Project-Based
Learning. This method shows a gradual change of the teacher’s function from directing to
coaching, whereas students are allowed more freedom to develop the progress of their
personal learning course with respect to their preferred competence acquisition. For every
thematic term, the student sets up his Personal Development Plan (PDP), which points out the
competencies he wants to acquire as well as the thematic term he will participate in.
(Kleinpaste, 2001)
Progressive insight has indicated the necessity for the Academy CII Deventer to define
matters more accurately per term. It might be said that the educations BI and IDM are
continuously being developed. Due to these ongoing developments, foundations of
competence-based learning are getting more and more solid.
Dissertation for Master of Science - 7 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
Dissertation for Master of Science - 8 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
3. METHODOLOGY
This study has been done in order to come to a Digital Competence-Based Assessment Frame
(DCAF). The aims formulated in the previous chapter as well as the sectors defined serve as
my point of departure; my object of study.
• First, this study will deal with competence-based learning in general, and the
realisation of a competence-based learning curriculum within the courses BI and IDM
of the School of CII at the Deventer Saxion University in particular.
• Second, the main focus of this study is on the process of competency assessment.
• A learning environment modelled on the concept of project-based learning constitutes
a third subject of study.
• The fourth and concluding aspect of this study aims to formulate the digital
requirements for a competence-based assessment frame (DCAF).
The Educational Analysis Model (EA Model) as developed during this study will serve as a
framework to fill in the assessment frame. By using this EA Model, the model as such is
tested to see whether it might be instrumental in analysing a learning environment. During the
preliminary stages of my investigation I searched for a theoretical framework to capture my
subject of study in a clear and unambiguous way.
Subsequent discussions with my supervisor and student peer group resulted in my choice of
the EA Model as an analysing guideline for my Masters Dissertation. I have developed this
model further.
The study can be seen as a pilot investigation which, by using a qualitative research method,
seeks to give content to a Digital Competence-based Assessment Frame. A thorough
investigation of secondary literature will be made for this purpose. In addition, the DCAF will
be realised in an exploratory way by means of open interviews with experts in the field.
The results of the case study - c.q. the project “Route Assessment System” - will be used
supportively.
3. 1. THE EA MODEL
A student starts a course in order to acquire a certified qualification acknowledged by society.
With it, he hopes to gain a position in that society. Essential factors describing this progress
are looked at from a student-centred approach of the learning process. Arranging these factors
has resulted in a ‘potential’ analysis model, defined as the Educational Analysis Model (EA
Dissertation for Master of Science - 9 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
Model). In this research report the EA Model is used as a guideline for setting up the
functional requirements for a Digital Competence-Based Assessment Frame (DCAF).
Construction of, and the theory behind the EA Model will be described first in order for it to
be fully understood.
We depart from a student-centred approach. A student’s first business when choosing a course
is the organisation of the school/university/course. When the student starts his course, he is
confronted with a teacher. At first, the student will see his teacher as a person who executes
education; he will not immediately understand a teacher may assume other roles in
competence-based learning. The student basically means to attend courses, and after some
time he is tested in order to determine if a he has reached the required level. During courses,
the teacher, who sees to it he will acquire the necessary qualifications, coaches the student.
More often than not, the teacher is the first one to determine if the required level for any
subject has been reached. Environmental learning conditions in this situation can be described
as belonging to the safe and closed micro environment of the group.
The sum of several tests will be civilly effective by means of a certificate, which attests the
student did indeed follow a specified educational curriculum and met the required end terms.
On macro level, a certificate has a particular social value; it is this value that will help the
student to gain a position in society.
The educational course was described previously. Three aspects are to clearly to be
distinguished:
• There are three participants, namely, students, teachers and supportive personnel of
the educational organisation;
• There are three processes - coaching, transfer of educational content, and assessment;
• Depending on the size of the educational organisation, three levels may be defined:
- In the executive field interaction takes place on micro level;
- During courses it is essential matters are organised correctly; this belongs to
meso level;
- On macro level, there is a civil organisation that can lawfully determine the
validity of the acquired certificate
Dissertation for Master of Science - 10 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
Diagram 1: Educational Analysis Model – EA Model @ De Vries, 2005
In order to gain an understanding of these aspects, factors determining each aspect will be
defined more specifically in the paragraphs below.
3. 1. 1. PARTICIPANTS
In relation to the direct learning process, three participants can be distinguished within any
educational organisation.
• Students
• Lecturer
• Supportive personnel
The student wishes to acquire the desired qualifications by availing himself of education.
The teacher/lecturer is the immediate executive of education and as such, takes on several
roles. He is the expert and the coach.
Supportive personnel function as assistants in education; they facilitate the learning process.
As a rule, a simple model is used to show the dependence of teacher and student on an
educational organisation within a supply-centred learning environment:
Dissertation for Master of Science - 11 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
Diagram 2: Education triangle
In this model, education is directed/controlled by the educational organisation providing for
the course. Those managing the course fill in framework for the learning process, and tasks
assigned. The student depends on the educational supply of the course, which is executed by
the teacher. The curriculum is definite for the entire learning course.
3. 1. 2. PROCESSES
The three processes a student will have to deal with in a learning environment have been
defined as follows:
• Coaching
• Education(ional transfer)
• Assessment
A first contact between a student and a representative of the school focuses on the coaching
process of the future course. Coaching aims to stimulate a student’s personal and professional
development. For this, the course, c.q. the school provides for a learning environment that
offers a student the required knowledge, capacities and behavioural attitudes belonging to a
profession. If he passes the tests - in other words, meets the defined assessment parts required
for the certificate, the student will gain a socially authorised qualification.
An assessment seeks to test and evaluate whether a student meets the quality requirements
specified in a competence to be tested.
Dissertation for Master of Science - 12 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
3. 1. 3. LEVEL
From a sociological point of view, three aggregation levels c.q. organisationally grouped
levels can be defined in a learning environment:
• Micro
• Meso
• Macro
The actual learning process within a class or group takes place on Micro level. Elements in
this process are the contacts between students, the contacts with teachers and supportive
personnel.
The actual organisation accommodating a particular course belongs to Meso Level. As for
Higher Education, we may think of departments, institutes or academies.
One step above that, a School of Higher Education or University often serves as an umbrella
organisation. This is the Macro level; authorities, interest groups and development institutions
belong to it. These organisations and institutions mainly focus on policy-making and the
development and investigation of new educational contents.
In this Master’s dissertation, the EA Model is used as an analysis model aiming to arrange the
several aspects of the investigation, and to define the functionality of Digital Competence-
Based Assessment Frame. The analysis of the learning environment departs from the
student’s demanding position
3. 2. METHOD
In order to set up and define an Assessment Frame in a Competence-based Learning
Environment, three methodological approaches of the subject have been applied.
• The study of relevant secondary literature such as books, articles and additional
information from several websites serve as a theoretical foundation;
• During the 2004/2005 curriculum, case study experiments essential to the underlying
facts were done at the Deventer School of CII;
Dissertation for Master of Science - 13 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII
• Further, people within the Saxion University as well as some external key figures from
two other Universities were interviewed.
Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
3. 2. 1. SECONDARY LITERATURE
In an attempt to firmly base the research subject, “Competence-based Assessment”, three
studies, researches and books were used relevant for both our subject and areas of common
interest. Aims and subject specifications defined what to look for in secondary literature:
- Competence-based Learning within Higher Education
- The New Learning, or the Theory of (Social) Constructivism;
- The assessment, ranging from simple tests and evaluations to authentic assessments
and the use of portfolios;
- Digital, or electronic, virtual learning environments;
- Collaborative learning in a digital environment (CSCL)
These aspects were carefully selected according to their theoretical foundation and relevance
to current developments. The results are presented in this Master’s dissertation.
3. 2. 2. CASESTUDY
During the course Master of Science in E-Learning, Multimedia and Consultancy, educational
reform of the Deventer School of CII was a regular subject of investigation. In the
Communication- and Change Management Module CCC, a preliminary investigation was
done by the organisation CII Deventer in order to initiate a reform course involving the
introduction of a Digital Portfolio (DPF) that might be instrumental as a means of assessment.
The research unit RM (Research Methodology) did a pilot inquiry among groups of lecturers
and students, asking them how they would feel about a digital portfolio. During the 2004/05
curriculum, a project was set up as a part of the module Project Study in order to test the
Route Assessment System with the help of a group of students and lecturers. These parts
together may be interpreted as a case study of the implications of educational reform brought
about by the introduction of Competence-based Learning within the courses BI & IDM. This
specific, reformed educational practice was realised on a location, which knew controllable
conditions and a stable team of lecturers. Changes were reported in several ways; the
conditions mentioned above define this project as a case study. (Cohen a.o., 2000; Schreuders,
Peters, 2002). The case study particularly focused on the student/teacher’s place and function
in determining the acquired competencies, and the role of digital support therein.
Dissertation for Master of Science - 14 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII
Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
3. 2. 3. INTERVIEWS KEY FIGURES
Several semi-structured open interviews were held in order to relate the results of the case
study to common practice in Higher Education. These interviews intended to hear from key
figures at Saxion University how they viewed developments as regards competence-based
learning. To get a better picture, some external persons from the Universities Han and
Windesheim were interviewed as well. The interviewees have been assigned to the EA
Model; the numbers in the cubes correspond to their tasks and functions.
Diagram 3: Outline interviewees
The interviews aims to give a defining and explanatory representation of the elements
involved in ‘Competence-based Assessment”.
An interview guide was constituted for the research proposition, c.q. working method to be
used for the master study (Cohen a.o., 200; Saunders a.o., 2002). Next to an introduction
revealing the main purpose of the interview, and some general questions about the person’s
function and tasks, the following operationalised items were taken up in the interview guide:
- The interview guide was built around the Maco / Meso / Micro levels;
- On Macro level questions were asked involving an interpretation of competence-based
learning from a perspective of larger organisational institutions;
- On Meso level, questions were formulated as towards realisation competence-based
learning; assessments and the New Learning in a demand-centred environment;
- On micro level, special attention was paid to the position of assessment and the digital
aspects
(See appendix 1.a – Interview guide)
Opinions of the interviewees per subject were transcribed and collected into written reports.
After this, every subject was described in a summary, which has been used for this study.
Characteristic statements serve as a support for specific conclusions within (the scope of) this
master study.
Dissertation for Master of Science - 15 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
Dissertation for Master of Science - 16 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
4. LEVELS
In the three-dimensional EA Model, “Level” stands for the analysis of the learning
environment seen from a perspective of interests and influences in the educational field. This
approach means to analyse political and social decisions as towards policy-making on macro
level. Political decisions and experiences, then, influence meso level, with which we mean the
set-up of the educational field. Daily practice, organization and realization of education are
executed and described on the micro level of actual teaching. (De Jager, 2004)
Diagram 4: Levels of EA Model
W. Jochems does not explicitly refer to the three layers as macro, meso and micro; however,
the changes implied in his theories about educational reform can certainly be interpreted along
the lines of this model:
• Macro is the organisation level. The institution generally defines the contours of
education as seen from an organisational and infrastructural perspective
• Meso is the curriculum level. Course directors use it as a framework by means of
which the various sectors of the courses are structured, lecturers directed and
educational tools employed
• Micro is the level of learning in class. It aims to provide lecturers with rough
guidelines to realise the learning conditions in education (Jochems, 1999)
In accordance with the research hypotheses, the levels can be defined as follows:
• Macro: the social context, authorities and umbrella organisations;
• Meso: the organisation within a School of Higher Education;
• Micro: the structure of the actual course
The three levels will be dealt with more thoroughly in this chapter.
Dissertation for Master of Science - 17 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
4. 1. MACRO
The sociological approach of a macro environment concerns itself with a major political and
social context (De Jager, 2004). On the higher generic level of a macro environment, analysis
targets social and political developments and their influence on a national as well as on a
European level. Political decision-making cannot be separated from developments in society.
The current state of affairs in the world reveals a major role for the following developments:
• Changes generated by a society availing itself of information technology and society
storing more and more information);
• Availability of various means of communication and their possibilities;
• The 24-hour economy;
• Being online continuously; at present, it is possible for people to get information from
all over the world via the internet without their being dependent on time and location.
Summarised, what we see here is a globalisation of society (Castells, 2001). The aspects listed
above will be integrated into daily practice. For education, this implies these elements will be
part of a curriculum as resources to be utilised; learning and everyday use will urge students
to internalise them. Society is rapidly changing from an industrial to an informational one, as
can be seen in the column below:
Industrial Society Information Society Offer centred Managing/Efficiency/directing Extra training focused on function Engines, procedures Long-term repetition of the same processes and tasks
Demand centred Stimulating/adaptive/facilitating Development of productivity through knowledge People, knowledge Rapid changes presenting new problems New knowledge about the latest ICT developments is required, as well as about new products and services
Table 2: Features of innovation in society, Hoeksema (2002) with own addition
This transforming society, in many respects changing at a more complicated and faster rate
than ever, urges education to provide for professionals who are broadly orientated, have more
capacities, and are therefore better adapted to the requirements of the job market c.q. society.
Central here is the concept of Man as a source of knowledge, as described in the HRM
theories. This concept implies society is progressively seen as a knowledge society; the aspect
Dissertation for Master of Science - 18 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
of ‘lifelong learning’ - or the ‘educational capacity’ to transform information into knowledge
- will be essential for future developments. (Castells, 2001)
4. 1. 1. POLITICS AND EDUCATION
The changes currently taking place in Higher Education are prompted by the introduction of
the Bachelor and Master system. This is consistent with European policy. Within the concept
of a European Union, which involves free traffic of people, services and goods between
member states, a need was expressed for an equal evaluation of certificates. In order to
achieve an equal treatment of the various certificates and learning courses within Higher
Education and Universities, several agreements were accepted by the European council of
Ministers of Education. In this way, the European countries want to equalise the curricula in
Higher Education in terms of level and quality.
In June 1999 the European Council of Ministers of Education and Science signed the Bologna
convention. This convention ordains to organise Higher Education in all EU member states
along the lines of an equal educational system. This so-called ‘BaMa’ model is derived from
the Anglo-Saxon model of Bachelor (undergraduate) and Master (graduate) system (Bologna,
1999). In Dutch and Flemish Higher Education and academies the courses are made ‘BaMa-
proof’. Another result of the Bologna convention implies the introduction of an unequivocal
European method to credit study points; this with a view to levelling out differences in study
pressure. With the introduction of the European Credit Transfer System, the EU intends to
regulate an equal crediting of study (pressure) points. One ECTS point constitutes a value of
28 hours of study pressure for the student; all EU institutions will implement this method for
Higher and Academic Education. The Dutch-Flemish Accreditation Organisation (NVAO)
should accredit all courses and studies within Higher Education working with the BaMa
structure.
In March 2000, Lisbon, the European council of Prime Ministers laid down the so-called
Lisbon Strategy. This strategy’s main object aims for the European Community to gain an
eminent role as towards knowledge and innovation. This requires extra efforts from the
member states in matters of education. One wishes to achieve a guarantee with respect to
quality and educational innovation. The average level of education of all member states
should be improved. In addition, designs are being developed to secure ‘lifelong learning’. It
goes without saying that the developments above affect higher education in the Netherlands.
Dissertation for Master of Science - 19 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
4. 1. 2. COMPETENCIES AS INTERMEDIARIES BETWEEN EDUCATION AND JOB MARKET
The past decennia show some characteristic features of social developments which can be
distinguished in education as well. In the 60-ties and 70-ties the production process grew to be
an industrialised mass production.
Technological developments continued in the 80-ies and 90-ties, and automation of data
processes emerged with the introduction of microcomputers. The administrative business
processes were automated/computerised; the 90-ties gave birth to the concept of information
society. Ever since, business models of organisations have become more flexible and adapted
to changing environments Education attempts to be in touch with these developments by
training the entirety of professional competencies, rather than assessing each subject and
expertise separately (Baartman a.o., 2004). This change has also affected contacts with
employees in a labour organisation.
This changing approach of an organisation is expressed in the theory of Human Resource
Management. This theory tends to no longer see an employee as a source of expenses, but as a
resource of the organisation. The employee is expected to assume the various roles within a
business organisation in a flexible and competent way (Dochy, 2005). As a consequence,
competence has become an important issue in the debate about the inadequate connections
between education and the job market. We refer to Maastricht 2004, when the collected EU
ministers of Education, presided by the Netherlands, laid down the following:
“The modernisation of their (i.e. the EU member states) systems of professional
education and schooling in order to turn Europe into the most competitive economy,
and the supply of qualifications and competencies necessary for all Europeans – be
they young, old, unemployed, or socially at a disadvantage – to be able to fully
integrate into a developing knowledge society, thus contributing to more and better
jobs”.
With a view to European collaboration, an unequivocal assessment of qualifications,
competencies, certificates and courses is highly advisable. In this process of transformation
one aims to acknowledge competencies acquired elsewhere/abroad. This is advisable with
respect to an adequate realisation of the EVC process. Relevant here is the assessment of
competencies acquired in the practice of a professional function in business.
Dissertation for Master of Science - 20 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
COURSE/FIELD COMPETENCIES
In order to adequately tune in education with the job market, an agreement was made on a
national level. Involved were two umbrella organisations, namely, the council of Higher
Education and the VSNU, and two organisations representing the work field – VNO-NCW
and MKB-Nederland. This agreement means to lay down the rules for a national work field
consultation with respect to formulating the course/field competencies of any particular
branch. In this way, the course/field competencies can be defined nationwide, and related to
the Bachelor’s degree in question (Knottnerus, 2004). The course/field competencies will be
used as a quality assessment for the accreditation of a Bachelor’s course.
4. 1. 3. EDUCATIONAL POLICIES AND HIGHER EDUCATION
During the consultation with the Ministry of OCW and the Council for Higher Education (i.e.
the umbrella organisation of all Schools of Higher Education in the Netherlands), agreements
were made as towards achievement. Some of the agreements defined are relevant for the
subject of this Master dissertation and will be referred to briefly.
In the coming years, Higher Education will see to it that more students will graduate as
bachelors; their variegated background will ensure their professional functionality in a
complicated working environment. This constitutes focus/central point two and three The first
focus/central point means to improve the quality guarantee of particular aspects within the
higher educational courses. Improvements are specified as follows:
a. A system securing internal quality;
b. Study pressure;
c. The quality of assessment.
Setting up the DCA Frame may possibly contribute to a realisation of the third focus point.
(point c)
4. 2. MESO
The meso approach of the educational field focuses on structures, relations and/or groups (De
Jager, 2004), such as there are: the educational organisations, departments, academies and/or
institutes. It is on meso level that educational organisations make decisions involving political
and social education policies; and it is here that political decrees are turned into actual policy
plans and documents for the educational organisation in question. These policy plans are put
into effect within a department, academy or institute, and establish the lines along which Dissertation for Master of Science - 21 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
courses develop their educational practice. The educational frameworks – the curriculum,
forms of education and education philosophy - are set up for the School of Higher Education
in its entirety.
Choice of policy has an important effect on the underlying levels of the educational
organisation (academies and courses); how can a specific policy concerning competence-
based learning be realised within all sectors of a School for Higher Education. Questions may
arise as to how much these underlying levels are expected to comply with the frameworks
imposed, and if academies and courses have their own say in the new developments. Saxion
University serves as a first research area for this master’s dissertation. In this area, the
research is focus on the innovations in self-regulating learning and assessment is carried out
within the personal learning process. (PLP)
4. 2. 1. SAXION AND SELF-REGULATED LEARNING
The personal learning process is essential to Saxion’s education philosophy as regards their
realisation of demand control and support of the actively learning student.
The PLP tools provide students with the means to give shape to their own course of learning.
With respect to this, Saxion came to develop a cyclical model centring around the system of
coaching. Learning of students can be seen as a recurring movement. Every period in
education shows the same learning cycle in four phases:
• In the orientation phase the student investigates the discrepancy between the
competencies he seeks to acquire, and those which he already has. This results in
learning goals;
• In the planning phase the student decides how to realise his learning goals. He decides
how he will employ professional situations and Saxion’s learning environment in
order to acquire the competencies;
• In the executive phase he undertakes the activities planned;
• In the evaluation phase he calculates if the discrepancy between desired and already
internalised competencies has diminished indeed.
(Project PLP, 2006)
This cycle takes place in a learning environment in which the personal learning process of the
student is central. Other important elements in education are competencies, learning
environment and assessment.
Dissertation for Master of Science - 22 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
Diagram 5: Cyclical education model These elements can be described as follows:
• Competencies: the course competencies are derived from the professional
competencies. These have been formulated nationwide by a specific branch or any
relevant professional group. These competencies make out a student’s decisions
regarding his personal learning course, and define the type of evaluation in an
assessment.
• Learning environment: per semester, a student can decide how he is going to acquire
his preferred competencies in the learning environment. He can take the regular
programme, or perform an external task as a trainee; as a dual student, perform tasks
resembling the preferred competencies, or work on specific professional products in a
education workshop (Du.: OWP). It is essential the student can develop his
competencies in any of these environments.
• Assessment: As soon as a student thinks he has gained sufficient professional
experience to have his competencies evaluated, he may request an assessment.
Dissertation for Master of Science - 23 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
ELEMENT OF DISCUSSION
In interviews had several times a discussion about the approach and the way the instruments
for Personal Learning Process should be implemented. The focus or view is top-down
(student/course) or bottom-up (Saxion central organisation). Based on the flexibility for the
active learner (micro) in relation to the framework of the organisational (macro) level. This
discussion goes about freedom and the learners own route and the bounders of the
organisation. See in the diagram the two views.
Diagram 6: Place of the instruments (PLW-PLP)
4. 2. 2. SAXION AND ASSESSMENT
Educational reform involving competence-based learning has urged Saxion’s management
board to appoint two instructors, this with a view to the changing roles of lecturers. It is their
task to professionalize lecturers in the field of competence-based assessment.
The first instructor, dr. W. Schoonman will mainly concern himself with tests of conduct and
their function in competence assessment. In a test of conduct, the student is required to show
his professional skills and actions in an environment related to the work field.
Learning assignment for instructor and the knowledge group aims to find a way to perform
generic tests of conducts and their related tests of conducts within a professional context.
During an interview, mr. Schoonman summarized his task as follows:
“The learning assignment comprises processing and proposing of a new way of
assessment in competence-based education. We mean to improve the quality of
lecturers with respect to conduct testing in competence-based assessment”
The second instructor, dr. G.J.J.M. Straetmans will concern himself with the measurable
aspects of competence assessment related to professional tasks and products. His learning Dissertation for Master of Science - 24 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
assignment indicates that explicit attention should be paid to improving lecturers’ expertise in
the field of tools and procedures for competence assessment. In his interview, mr. Straetmans
defined his assignment as follows:
“The learning assignment aims to improve educational assessment skills of
lecturers at the Saxion University. We mean for lecturers to adequately represent
educational standards. Competence-based learning involves the ability to assess in
a professional context.”
In this way, Saxion University hopes to support curriculum reforms on two fronts; the
personal learning process for students, and improvement of assessment expertise for lecturers.
4. 3. MICRO
The bottom layer of the world of education deals with daily practice in the educational field,
namely, directing, interpreting and performing education. It is the interaction between student
and lecturer in the way of learning and teaching of a course, subject or lesson which matters
here. The Micro approach investigates the relation/interaction between individuals and small
groups (De Jager, 2004).
The EA model defines this level of approach as the micro environment of an educational
organisation. For this dissertation, we have decided for a case study of the courses BI & IDM
within the educational organisation of the Deventer School of CII.
4. 3. 1. THE EDUCATIONAL MODEL OF THE COURSES BI & IDM
The Deventer school of CII realised new, competence-based learning by means of the model
of adaptive education. Adaptive education seeks to meet a student’s basic needs. The student
is the key figure in any educational process; in other words, education is meant to serve him.
This implies a student will be actively involved in setting up his own course-route; he is the
one to decide how his learning is going to happen. Naturally, this will all take place within the
bounds of the School of CII and the domain/course competences.
The lecturer’s role, then, may be defined as supportive of the personal learning process of
individual students. A lecturer will assist students to develop their own educational course. As
such, a lecturer no longer transfers knowledge, but spends part of his time coaching individual
and/or small groups of students. (Kleinpaste, 2001; Blok, 2004)
Dissertation for Master of Science - 25 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
Diagram 6: Role from lecturer-guided to self-regulating learning
Most students will have to learn how to cope with this alternate way of educational service. In
order to counter this, the courses BI & IDM decided to gradually hand over control to the
students themselves. The model of project-based learning was taken as an educational model
for this process.
4. 3. 2. PROJECT-BASED LEARNING
The initial drafts of the new education curriculum for the courses BI & IDM used the term
“competence centres”. As this term may seem a bit confusing – it is common in business as
well – the current phrase reads as “thematic project learning”. In addition, the definition
“Educational Workshop” is used, provided there is a thematic relation with the practicability
of a task as it was commissioned by the external instance. Within thematic project learning we
work on cases, which are nearly identical with authentic external tasks.
The concept of thematic project learning has the following features:
• Each semester, several themes are offered. These vary in content and level,
• Each theme has a case related to professional practice and its corresponding
competencies,
• Each theme is represented by a team of lectures who function as project manager,
coach, advisor or commissioner, and who are responsible for accomplishing the theme
to the full.
Advancement of the theme is related to professional practice; it tries to reflect the whole range
of actual working situations as much as possible. Student teams will have to attend all phases
of the project, running from preparation – performing – evaluation and presentation of the
final product. Different methods may be used per theme, and complexity of tasks will vary.
Purpose is to strife for separate themes that are not sequentially interdependent. (Geerligs en
Van der Veen, H11, 2001) Dissertation for Master of Science - 26 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
5. PARTICIPANTS
Three groups emerge from the educational field as participants in the learning process. The
groups involved have been identified as Students, Lecturers and Supportive Personnel; they
have become part of the three-dimensional EA Model.
Diagram 8: Participants of EA Model
First, there are the students (Student) who see education as a means to develop themselves
and qualify as a starting professional for the job market. Teachers (Lecturer) - as pedagogues,
subject teachers and mentors - want to help the students to achieve this goal by teaching them
the profession of their choice, and coach them. If this learning process is to be successful,
both students and teachers will need support from the organisation. Administrative,
organisational and staff personnel make up the various services and provisions in an
educational organisation. They will have to support and facilitate the educational field with
the necessary means.
As previously recognized, a conscious educational reform is taking place in Higher
Education; we may speak of a shift from offer-(teacher)-centred to demand-(student)-centred
learning. In addition, the technological developments involving ICT have a major influence
on learning processes. It is these two aspects that determine the roles of the participants in a
learning environment.
5. 1. STUDENT
The student is the key figure in the educational field; he wants to gain a position on the job
market by acquiring a professional qualification. Within the structures of the “Old Learning”,
subjects were offered which were related to professions. In the “New Learning”, however, a
Dissertation for Master of Science - 27 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
student is expected to develop his personal learning process. This new approach is new to
many students, and they will have to adapt to this modern way of ‘learning how to learn’
Two interviewees said the following:
“It is important to raise the students, to teach them this way of learning” (R01)
“The student’s position will change; it is anticipated a student is going to assume a
more active role and will evolve as a person who signs up for courses personally,
and who plans in a goal-oriented way”(R02)
Several interviewees point to the importance of an adequate instruction as towards demand-
centred learning (PLW -Personal Learning Process- PLP), and of maintaining a good
communication with students. The Saxion PLW System takes both instruction and
communication into account. For this, competence-based learning makes use of an
introductory interview between career coach and student, comprising eight steps based on the
cyclical education model:
Orientation:
1. The learning question: the student makes an analysis of acquired competencies and
competencies to be acquired. These should belong to the domain of his course profile;
2. He translates the learning question to learning goals, formulating them as directives for
his learning process;
3. He makes an estimation of influences, which may either stimulate or obstruct
achievement of his learning goals;
Planning:
4. Learning goals and estimation are taken up into the POP (Personal Learning Process).
Student and career coach discuss his findings, and the latter will advise the student how
to realise his Personal Learning Process. Subsequently, the student makes his choices;
5. The student plans all activities intended;
Executing:
6. The student executes the activities as planned. He will regularly evaluate his
progressions with fellow-students and coaches;
Evaluation:
7. When the student starts to deliver professional products, he can show these as the results
of his efforts in an assessment portfolio. He digests the results and takes tests;
8. The examiners/assessors judge the acquired competencies and credit them with study
points. (Project PLW, 2006)
Dissertation for Master of Science - 28 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
One interviewee commented that the proceedings as described above do not actually represent
anything new; in his opinion, it would all be quite similar to learning according to the
principles of Montessori education:
• Student: learn to deal with freedom, and learn to be personally responsible for one’s
learning process
• Lecturer: realise and facilitate ‘assistance in learning’(R10)
Next to Maria Montessori’s education theory and the adaptive education model at primary
and secondary schools, adult education has a distinctive history of using partial certificates.
What to think of the concept of open learning developed at Open universities? For many years
now, learning processes within these institutions have been taking place along the lines
described above. Higher Education may consult other institutions in the matter of various
learning concepts and adapt them to their specific situation.
Reflection is important for independent and efficient learning. It enables the student to clearly
perceive the course his personal learning process is going to take in practice, and it enables
him to communicate with his coaches and assessors. By means of reflection, a student learns
how to function more professionally in the work field; it keeps him motivated and enables
him to adequately direct his professional behaviour.(HBO-V, 2005) In our rapidly changing
society, many students will frequently switch jobs; having acquired reflective skills will
enhance their perception of their professional capacities, and how to adapt them to a new
work environment. Reflection, then, makes sense if a student is able to - and indeed dares –
to evaluate his professional actions and adjust them if necessary. Skills such as these are
inextricably bound up with independent, personally managed learning and give shape to his
own demand-centred learning process.
The Korthagen model may support a student in developing his reflective skills. Korthagen
defines reflection as, “Somebody is reflecting when he tries to structure or restructure an
experience, problem, or existing knowledge and perceptions”(Korthagen, 2001)
Dissertation for Master of Science - 29 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
Diagram 9: Spiral model for reflection (Korthagen)
1. Actions/experience> 2.Reflecting> 3.Awareness of essential aspects> 4. Chosing and developing alternatives> 5.Ttrying out>
Phase 1/5 of the previous cycle will become 5: > Phase 1: Actions/experience/try out
Which were my aims?
What to pay attention to?
What did I want to try out?
Phase 2: reflecting
What actually happened?
What did I want/what did I do?
What did I have in mind?
What did I feel?
What do I think my customers wanted, did, thought, felt?
Phase 3: Awareness of essential aspects
How are the answers to the previous questions interrelated?
How does this particular working environment influence affairs?
What implication does this have for me?
So, how can the problem be defined, or what positive discovery can be made?
Phase 4: Alternatives
What alternatives can I find (i.e. solutions or methods to use for my discovery/ what are
the advantages and disadvantages these alternatives bring along?
Which will be my next options?
Dissertation for Master of Science - 30 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
Working it out according to this specific model cut down to a person’s individual experiences
may help a student develop his own learning process. Reflection, then, forms a link between
acting and thinking.
5. 1. 1. DIGITAL EXPECTATIONS
Several respondents at different times referred to digital possibilities in order to facilitate
freedom of the actively and independently learning student. A digital portfolio may support a
student to collect his personally developed learning documents such as POP, reflections, plans
of action, professional products and formative assessments.
Another example of digital developments is the communicational aspect: apart from e-mail,
the phenomenon of e-learning platforms has come into being. On these platforms, a student
can dispose of a wide range of opportunities for interaction between him and his fellow-
students, his lecturers and his course. At this moment, Saxion University offers facilities for
e-mail communication, the intranet and admission to various websites. These are used for
providing students with information about courses. Further, the Lotus Learning Space
programme serves as an e-learning environment at Saxion University; and Quickplace is
available to improve collaborative learning. At this moment, there is no unequivocal Saxion
standard of e-learning, although several steps were taken to establish one. In March 2006, the
DLW2 was set up and presented to the management board; it proposed to realise a Digital
Learn- and Work Environment by means of the E-Learning Framework (ELF, 2004). The
proposal involved the development of a standard for future use, which defines the
functionality of various aspects of e-learning. Interviewees from other Schools of Higher
Education explained their courses work with the Blackboard e-learning programme and offer
ready-made digital frames for each student’s personal e-Porfolio.
5. 1. 2. DE DIGITAL STUDENT / THE NET-GENERATION
We see that many young students have grown up with various media. Students are fully
capable of using computers and their applications. They all know chat programmes,
participate in communication on the net and own other means of mobile communication.
Students expect the ‘New Learning’ to have the latest technological developments imbedded
in its structure. Schools do well to reckon with these expectations when setting up web-based
communication.
Dissertation for Master of Science - 31 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
Diane Oblinger states we can learn from the trends to enhance our education. She mentions
the following features identifying the ‘Net-Generation:
• Fast reactions;
• A visual predisposition;
• Ability to perform several activities simultaneously;
• Interactive.
Education should be focused on essential matters; it wants to do things that are relevant, and
which match current technological developments.
Students of the Net-Generation are involved in a lot of interaction. Therefore, opportunities
for teamwork and peer-to-peer contacts are all-important. Students do not learn in class
anymore, but in their own groups (informal learning) – which makes it necessary courses
offer them a structure directing their learning actions towards the educational goals. This
structure aims to set up a learning environment, which will gradually activate students; it
accommodates ‘authentic’ processes for them to work on collaboratively. Interaction, then,
should be aimed at having students solve problems together, and it should contain elements
such as ‘gaming’, debating and discussing (Oblinger, 2005). An adaptive and coaching role of
education with respect to independent learning is of major importance for an e-learning
environment.
A full realisation of the suggestions above would meet the wishes of some of the respondents
– but only if each student has his own digital learning/working environment, to be set up
according to his personal needs. Considering new ICT developments, blogs or weblogs may
partly constitute the e-Portfolio. A students’ blog may provide the student with an additional
means to digitally record his personal learning process and lay down his reflections on it in a
diary.
5. 2. LECTURER
First, ‘Old Style” education should be considered; in which course and lecturer determine
what teaching activities will be offered.
Transfer of knowledge is pivotal in the old learning environment. Groups of students attend
lectures/lessons, and the subject material is evaluated by means of knowledge tests, which
together constitute the final examination. A lecturer has a dual role:
a. He is a subject teacher, passing on knowledge;
b. He evaluates the knowledge transferred by testing the student. Dissertation for Master of Science - 32 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
Van Gelder’s cognitive education theories define in advance which cognitive end goals are to
be achieved. Subject material, pedagogical means, tutorials etc. are geared to these end goals;
tests eventually determine if the student has acquired sufficient knowledge. (Geerligs & van
der Veen, 2001)
How is the lecturer’s role going to take shape in this so-called “New Learning”, which is
founded on ‘constructive’ and ‘competence-based’ learning? Jochems and Schlusmans no
longer see a lecturer as a person who ‘translates and transfers knowledge’, but as a coach,
helpdesk and guardian of the learning process (Jochems and Schlusmans, 1999). This does
not mean knowledge is no longer an issue in the new competency-based learning
environment; however, it is no longer the only element in it. Basic knowledge remains an
essential requirement; main issue now is how this knowledge should be acquired. Apart from
that, education will have to create professional, tasks which resemble authentic tasks in the
work field as much as possible. Development of skills and professional attitude, then, should
be part of a competency-based learning environment
5. 2. 1. THE NEW ROLES
The Digital Assessment Portfolio Project and the related case study done at the Deventer
School of CII helped to identify the various new roles of lecturers in a competency-based
learning environment. Lecturers are expected to assume the following roles:
• career coach (Du. SLB-er) for individual students;
• coach/guide of a project team;
• subject teacher, taking care of supportive education such as theory lessons, workshops,
practicals; giving feedback on requests for formative tests
• giving a representative performance of the part of, for instance, commissioner or client
in a case;
• assessor of a conclusive interview which sums up the domain competencies required.
(De Vries, 2005)
It is advisable to keep these roles separate as much as possible. During his inaugural speech
Simons (2002) referred to the aspect of digital didactics in which lecturers can be seen as
having three distinguishing roles:
• Developer: designing, developing and providing digital tools of education for a course,
or for an entire curriculum.
Dissertation for Master of Science - 33 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
• Coach: teaching students how to learn from competence-based education, and how to
learn from a digital environment; coaching a team of students and coaching a student’s
personal learning process.
• Assessor: designing, creating and organizing opportunities for digital assessment,
developing digital tests, defining test standards by means of which students are
evaluated.
(Simons, 2002)
Change and expansion of a lecturer’s role will largely imply a shift of attention towards the
various aspects of coaching a student’s personal learning process. Looking specifically at the
ICT means involved, we might say that e-Moderating/Coaching is mentioned more and more.
Salmon indicates that the lecturer in his role as e-Moderator should basically understand the
difference between Face-to-Face communication (F2F) and Computer Mediated
Communication (CMC). As opposed to the former, CMC doesn’t have the additional aspect
of the physical senses; however, substitutions such as emoticon icons do emerge in CMC
coaching. Communicative and social-psychological aspects of e-learning as well as its
reliability will motivate students to use ICT tools. (Salmon, 2004)
Finally, Straetmans distinguishes new roles in assessment; according to him, lecturers:
• Assess; they spend part of their time evaluating students’ achievements;
• Construct; they design and make assessment tools;
• Innovate; they guarantee quality, are members of the examination board and are
involved in assessment policy.
(Straetmans, 2006)
The research done clearly confirms some of the above mentioned elements from secondary
literature and case study. All respondents brought forward that it is essential lecturers’ roles
should change in a competency-based learning environment.One of the respondents said,
“A lecturer’s role will ‘turn over’, i.e. be geared to students’ demands”. (R04).
According to another interviewee, the lecturers’ new roles can be divided into three parts:
“Assessor/Teacher/Career Coach (R08)”
This division into main groups is endorsed by the view based on the respondent, and is similar
to the processes Coaching – Education – Assessment. The lecturer is expected to assume a
different role during each process. As an additional element in the lecturers’ roles, Straetmans
mentioned the novel aspect of the lecturer as a designer and developer of digital applications;
he should select those which he considers most suitable.
Dissertation for Master of Science - 34 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
Many respondents stress the importance of another change in of the lecturer’s role: apart from
his task as coach, a lecturer should supply professional subject knowledge as well. He gives
students feedback on their advancements by making an adequate assessment of their
competencies, thus improving quality. In order to meet these peremptory changes in the
lecturer’s role, a school for Higher Education / Academy / course will have to train their
educational personnel. In doing so, they are sure to raise their standards.
Conclusion: lecturers will face three new aspects as regards their functions in competence-
based education:
a. Coaching and support prevail in his interaction with the student. This is quite new to
many lecturers
b. Digital means of education will be innovated and new ways of knowledge transfer
designed. The ‘New Learning’ and its use of ICT tools involves other didactics and
learning methods
c. Competence-based education requires an integral assessment of all elements, i.e.
knowledge, capacities, professional attitude and flexibility in varying environments. A
lecturer should be able to perform this assessment.
d. Training of lecturers in order to improve their expertise is essential for education to be
able to raise standards in competence-based learning.
A lecturer may specialize in any of the roles defined by De Vries, Simons or Straetmans.
5. 3. SUPPORT
Inherent to the paradigm shift in education are the changing roles of the supportive,
organisational services. Until recently, these services have been made to answer basic
requirements for recurrent lessons and patterns. This will change thoroughly. Schedules aren’t
standard anymore; classes are not to be seen as clusters to allocate class rooms to; regular
class rooms do not exist, and classes, grades and courses blend to produce mixed groups of
students. Provisions and means should be applied flexibly in the New Learning.
These reforms urged the Saxion organisation to start various projects.
Short summary of the projects:
• AO; visualizing administrative processes within the services and courses and gear
them for each other in order to meet student-centred requirements of education
• Pandia; a comprehensive registration system for settling administration regarding
students’ data, educational supplies and personnel
Dissertation for Master of Science - 35 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
• PLW; The personal learning process becomes implemented policy and will be
instruments to both students and lecturers
• DLWO; the Digital Learn- and Work Environment. This environment was described
in its efficacy as a personal portal for student and teacher. It is here where students
gives shape to the very learning route he desires to take.
• IA; most projects have significant implications for the ICT infrastructure. In order to
visualise and stream these implications, the project Architectonics of Information
presented a sketch of future developments. These will annually be carried out in
phases for a course year.
Employers belonging to ‘Support’ (as defined in the EA model) will have to adapt their
services to the changing roles of the lecturers; and they will have to supply the required
provisions for the demanding, actively learning students.
Support should recognise the generally acknowledged division into micro- and meso level,
that is, the layers of actual learning and organisation of the educational institute respectively.
Several interviewees among Saxion’s supportive personnel consider the course as an
independent unit, and each course should be allowed to set up and realise its own educational
concept. Respondents from other Schools for Higher Education share this view.
A central organisation aiming at ‘standardizing’ should offer ample opportunity for courses to
deal with the flexibility of demanding students. Students’ data, therefore, should be
transferable.’
5. 3. 1. PROCESSES SEEN FROM SUPPORT’S POINT OF VIEW
Supportive personnel at Saxion’s serve as point of departure for our definition of the frame
‘Support’ in the EA Model.
COACHING
Saxion’s Management Board has adapted national decision-making in educational policy
(=macro) to its own educational philosophy (=meso)
The PLW project has set up the Coaching process along the lines of this philosophy; it has
developed workable core instruments for the academies and courses to implement on micro
level - that is, in the course-related learning environment of students.
Dissertation for Master of Science - 36 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
EDUCATION
The project DLWO defined the ICT tools to be used by courses according to the IA – level
planning. In addition, the process ‘Education’ may be realised by means of the educational
development model 4C/ID (Four-Component Instructional Design). This model attempts to
stream competence-based learning involving professional tasks and products. It is advisable to
check to what extent the 4C/ID model can be applied to competence-based learning. This
should be explained as follows: The educational model of the BI & IDM courses tries to
establish a case, which is authentic. Working on professional tasks should resemble real life
situations as closely as possible. Student project teams should try and give these tasks a
semblance to professional products. Taking part in the activities involved in making these
products; assembling his own products and gathering formative feedback - all of this
constitutes evidence of a student’s competencies as recorded in his portfolio. This in
accordance with the 4C/ID approach, which aims for students to learn when working on
integral, useful tasks that copy real life situations. (Janssen-Noordman, 2002).
As the 4C/ID concept shows similarities to the DCAF, it will be briefly referred to, and
explained on its four main points.
Diagram 10: 4C/ID model
Dissertation for Master of Science - 37 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
The four main points of the 4C/ID model can be defined as follows:
1. Learning tasks; these are real, authentic and useful tasks.
2. Supportive information; the sustaining theory, lectures, (conceptual, structural &
causal) models students need to perform a particular task.
3. JIT – Just-in-time Information; the information required to perform standard
professional actions+ information answering the actual task involved.
4. Practical tasks practice; instructive exercises to automate routine aspects of learning
tasks; acquisition of professional skills.
(Merriënboer a.o., 2005)
Straetmans used the model to connect the Protocol Portfolio Scoring system to the 4C/ID
model. This may open up new prospects for the assessment frame (Straetmans, 2004).
However, the 4C/ID model implies setting up a new system; this may affect the flexibility of
demanding, actively learning students negatively.
ASSESSMENT
Testing and evaluation in competence-based learning still needs reinforcement. The Saxion
organisation has decided to improve lecturers’ qualities in this matter by appointing two
instructors, whose task it is to gear assessment to competence-based learning.
Supportive personnel’s involvements in assessment comprise the following aspects: design,
development, management and exploitation. In their book, “Flexibilisering van Toetsing”,
Draaijer a.o. refer to the assessment cycle model constructed by Jaspers and Schade (2002).
This model presents assessment management as a cycle of 8 phases, and was intended
primarily for knowledge tests.
The eight phases are:
1. Designing
2. constructing items
3. compiling tests
4. testing
5. scanning
6. analysing
7. sanctioning
8. evaluating
The model means to structure the assessment process.
Dissertation for Master of Science - 38 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
Once the criteria for a competence are known, a test can be designed which incorporates the
items to be measured. After testing, students’ answers to the questions should be scanned,
analysed or evaluated. If students meet the criteria, rating may be sanctioned and registered
for both students and course. Finally, the validity of a test should be checked; does it measure
what it intended to measure? If necessary, the test can be improved per item. It is the lecturer
and the course who develop (design, construct, compile) a test. Actual testing, and possibly,
scanning of answers (multiple choice) may be facilitated by ‘Support’. Evaluating, scanning
and analysing is primarily the lecturer’s responsibility. Actual testing and assessment by
lecturers take place under well-defined conditions (micro level). The board of examiners of a
course does sanctioning and evaluating. Certification at the highest level – the ‘bachelor’s
degree’ is founded on the required competencies of the professional domain (macro level).
Rating; the sum of course competencies belongs to the tasks of the School of Higher
Education (meso level), c.q. the board of examiners. This board laid down rating procedures
in the examination regulation of any course.
Dissertation for Master of Science - 39 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
Dissertation for Master of Science - 40 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
6. PROCESS
The three-dimensional EA Model observed three main processes in an educational
environment.
Coaching is of eminent importance in the new paradigm of demand/student-centred
education. Students need a person to talk to; it is here where the career coach comes in. A
student may consult his career coach throughout the course of his learning process, but
remains personally responsible for the way in which he attempts to become a fully able
professional. Educational institutions offer students a supportive environment in which they
can achieve their goals. Within a favourable learning environment, a student may develop into
a professional. The EA model defines this process as Education. Society expects education to
turn out competent workers. Therefore, students’ competencies have to be evaluated and
tested; this happens through an ongoing process of Assessment.
Diagram 11: Processes of EA Model
Demand-centred education, then, requires for processes to necessarily succeed each other as
follows: 1.Coaching 2. Education 3. Assessment.
These processes might be arranged differently in a traditional, offer-centred learning
environment. Education would probably come first, followed by Assessment; and questions
may arise as to the position of Coaching. Concluded, the EA Model may be seen as a dynamic
model. The order of the processes largely depends on the educational concept it is used for.
6. 1. COACHING
The frequently quoted ‘paradigm shift’ in education mentions the fact that lecturers’ role will
gradually change from teaching to coaching. In the EA model this has been identified as an
independent process, named ‘coaching’. The demanding, actively and independently learning
student is coached during his endeavours to become a professional. The term “coaching” is
Dissertation for Master of Science - 41 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
derived from the verb ‘to coach’; the dictionary refers to the example of coaching a sports
team (Van Dalen, 2004). In the world of sport the term is a general one, denoting the tasks of
a person who helps sportsmen/athletes to accomplish his goal. The sportsman/athlete, of
course, bears the ultimate responsibility for his achievement. In the educational work field the
terms ‘mentor’ and ‘tutor’ exist; these refer to persons who coach students. As a rule, the
mentor is a lecturer who is responsible for a group of students (class). He is the one to be
addressed if students want to discuss organisational matters, whereas the Anglo-Saxon term
‘tutor’ stands for a person teaching an individual student. Boshuizen and Kirschner (2005)
define the terms ‘mentor’ and ‘tutor’ more sharply, adapting them to the role of the lecturer as
coach. A tutor, then, helps a student to apply his knowledge and capacities in various
situations ranging from simple to complex. A mentor helps students to develop themselves; he
helps them understand this process of development and the influence culture has on it; and he
teaches them how to recognise their advancements. The mentor’s function, then, is similar to
the role of career coach in the Saxion PLW model (Boshuizen and Kirschner, 2005).
During the PLW Project at Saxion’s (Project PLW, 2006), participants construed the model of
the learning cycle as follows:
Diagram 12: Learning cycle and lecturers’ roles (Saxion PLW Project)
Dissertation for Master of Science - 42 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
The role of assessor is beyond the scope of this paragraph; we will therefore leave it aside and
restrict ourselves to tutor and career coach. The former helps the student to orientate on, and
plan his learning course; the latter coaches students during their personal learning project by
helping them to perform the professional tasks relevant for their competence acquisition.
Returning to the parallel with the world of sport, we might say that the coach is a sportsman’s
counsellor, who advises him in every which way possible. This notion is fundamental for the
process of coaching (in the EA model). The career coach functions as a student’s personal
guide and mainly deals with the latter’s learning course toward competent professionalism;
moreover, he pays attention to a student’s individuality as well.
The tutor is the subject teacher, who guides the student during his competence acquisition; he
challenges him to improve and broaden his professional tasks and products. At the Deventer
School of CII, students are placed in thematic project groups. A lecturer assists them and
functions as a theme coach. The student should be able to give a convincing performance as a
professional, and the theme coach may give him formative feedback in this matter. Further, a
theme coach will encourage collaborative leaning in student groups. He intends to guide
students towards making a task-oriented product, and coaches a process of co-operation.
The career coach aims to stimulate and facilitate a student’s personal development.
Summarized, the following elements of coaching are essential:
• The personal development of a student;
• Subject-oriented development to professionalism;
• Focus on the process of students in professional roles
• Task-oriented to work collaboratively and individually on professional tasks and
products
This is what the coaching process entails. The lecturer decides which part of coaching is most
in line with his personal qualities.
6. 1. 1. PLW INSTRUMENTS/TOOLS (PLP)
In this paragraph, the development of the Personal Learning Process (Du:PLW / Uk:PLP) at
Saxion University will serve as an illustration of the coaching process.
Interviewees were aware of Saxion’s strategic choice for the PLW system. Several of them
confirmed that the PLW tools are well-suited for students to decide independently on his
options for his learning route. However, they emphasized students should learn how to use
these tools; course and teachers are expected to encourage students to do so. Dissertation for Master of Science - 43 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
‘Students will have to learn “how to learn” in an innovated educational environment
(competence-based), and how to make use of the instruments developed for it”.
(R05).
More often than not, it is only when they have become seniors that students find out how to
work with the instruments; only then will they experience the advantages of competence-
based learning. By developing the PLW tools, Saxion University wishes to create
circumstances in which the coaching process comes into its own.
The core instruments are listed below:
1. Intake Assessment
2. Career coaching
3. intake interview and orientation programme PLW/PLP
4. POP Personal Learning Process
5. Study contract/Individual Study Programme (ISP)
6. Portfolio
(Project PLW, 2006)
Intake assessment makes it possible to determine elsewhere and previously acquired
competencies of beginning students (Du.: EVC). During the intake interview, career coach
and student can go over the ISP with each other. In most cases, a student will have to learn
how to deal with competence-based learning; in order to facilitate ‘learning how to learn’, the
instrument PLW orientation has been developed. Another essential tool is the POP, the
Personal Learning Process. The student records his identity in it, answering questions such as
“who am I, what do I want, how would I define my style of learning, etc; and next, sets up
his plans for his learning course. The career coach can tell him if his plans are realistic, and if
they fit into the profile of a domain/course competence. Next, an individual study programme
can be compiled, resulting in a study contract for a particular period. The school will provide
the student with a digital portfolio in which he can record and manage his POP. If required, a
student may present his POP to the career coach right from his portfolio; he can use this
portfolio throughout his entire learning course.
6. 1. 1. 1. DIGITAL PORTFOLIO / E-PORTFOLIO
In educational literature, the so-called e-Portfolio has progressively become the focus of
attention. The ‘e’ in this term is short for ‘electronic’ - which is, in this case, a web-enabled
application. The Digital Portfolio knows various forms, such as there are, a standalone Dissertation for Master of Science - 44 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
application on a student’s PC combined with a cd-rom; or a ‘members only’ system, which
may be accessible through the school’s intranet. This master’s dissertation uses both terms for
web-enabled applications for students to access via the Internet. In this way, the applications
are independent of time and space.
Higher Education sees an (e-) Portfolio as ‘means for the student to make visible his personal
development’ (Tartwijk a.o., 2003). The world of art and architecture has known the use of a
Portfolio since long; it is a large file in which examples of previously performed tasks are
collected; when shown, these examples may lead new commissions.
Free after Tartwijk, education tends to see a portfolio as ‘a selection of evidence visualizing
the way in which a student has worked on his professional tasks and products. This selection
improves a student’s personal understanding of his competence level and facilitates his efforts
to raise it. (Tartwijk a.o., 2002)
The DPF is an essential core instrument for the student to set up and realise his independent,
individual learning course.
Diagram 13: Learning cycle and portfolio (Saxion PLW Project)
This diagram visualises the three functions of a DPF. Three other functions have regularly
been discussed at Saxion University (as Bouman and Lisdonk quoted from professional
literature in 2005):
Dissertation for Master of Science - 45 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
1. Development portfolio; this PLW function is instrumental for the student to set up his
Personal Learning Process and Individual Study Plan (POP + ISP), to record the course
of his development, and to reflect on his professional growth.
2. Assessment Portfolio; in order to be assessed, the student gathers all materials which
prove he meets the required criteria of the relevant competencies. As an example, we
mention formative feedback on his products and task performance given by lecturers,
students and professionals. All related reports, documents and products constitute
archive material for the portfolio. It is up to the student to make a proper selection which
proves he meets the criteria
3. Presentation portfolio; the student has an opportunity to show his accomplishments by
presenting products he deems successful in a so-called showcase. This may entail a CV,
for instance, which he uses for work placement acquisition, a graduation job, or even
applications.
Another aspect emerging from the interviews is the difference between ‘open’ and ‘closed’
portfolios. Van Tartwijk explains this difference as follows: does a student have standard
forms which oblige him to show particular content, or is he free to build his portfolio as he
thinks fit? Further, who is responsible for access to his portfolio: should that be the course,
and can a student merely indicate who may look into his materials. (Tartwijk a.o., 2003) Right
of ownership of the portfolio is at stake here.
The University Arnhem/Nijmegen (HAN) uses the system Roxen, which is based on Content
Management System (CMS) where the access is controlled by the educational organisation.
The same goes for the Concord system employed at the Windesheim University. Saxion,
however, has opted to leave right of ownership in the hands of students themselves; they can
decide who they will allow to examine their portfolio. According to Tartwijk’s differentiation,
Saxion’s DPF would be defined as an open portfolio, whereas the systems Concord and
Roxen are closed portfolios.
6. 1. 2. E-COACHING
Digitalisation of communication and interaction in society progresses gradually. As
mentioned before, we appear to be living in an information society. Digital skills of students
increase rapidly, and education inevitably has to adapt to this fact. Therefore, the lecturer in
his role as coach certainly needs digital support. Experiences and theories concerning support
have been gathered by Smits a.o., and denominated as “E3 coach”, in which “3” stands for Dissertation for Master of Science - 46 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
three qualities: Effective – Enthusiastic and Electronic (Smits, 2006). The term e-Coaching as
used by Smit fits in with the EA Model process of “Coaching”, and covers all forms of
guidance in an electronic, web-enabled environment. Smit defines these forms as follows and
I arrange them in a way similar to the successive processes in the EA Model:
• Coaching; guarding of learning processes and advancements of individual
students or groups of students;
• Education; providing instructions during learning processes and supporting
students when they work on study assignments and professional tasks;
• Assessment; giving formative feedback and reflection on the tasks executed.
(Smits a.o., 2006)
Smits confirms the previously described differentiation in a lecturer’s role. According to him,
a lecturer will assume several roles in his interaction with students; these differ per phase in a
learning process. Further, Smits observes full justice will be done to e-Coaching when it takes
place in a learning environment where digitally supported, collaborative learning is prevalent.
This notion will be examined in greater depth in the chapter describing the EA Model process
“Education” .
Smits a.o. mention four perspectives of coaching:
Diagram 14: e-Coaching from four perspectives (Smits e. a. 2006)
Dissertation for Master of Science - 47 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
The four perspects of e-Coaching:
1. Building motivation according to Salmon’s theory; this takes place in four stages:
a. Access and motivation
b. Online socialisation
c. Information exchange
d. Development
In order to have students develop themselves as regards content, the e-coach should
pay attention to the procedural aspect of online communication. He should increase
motivation and skills of a student group.
2. Giving form to Collison’s communication interventions: communication should be
facilitated, and students supported. The e-coach facilitates the communication process
in two ways:
a. He teaches students basic communicative skills such as rules of proper conduct
with respect to a functional and effective online communication;
b. At a higher level, he instructs students about discussion techniques in an electronic
environment aimed at developing and broadening a subject
3. The pedagogical, didactic style as seen from a lecturer’s own perspective, see Paulsen.
Paulsen’s model expects a lecturer/coach to choose from three functional roles:
a. Organisational: he plans time and content of online meetings
b. Social: the lecturer provides for a pleasant and stimulating learning environment
by giving feedback and input during processes
c. Intellectual; which may be the most important role. The lecturer asks crucial
questions and answers questions from students, thus encouraging them to get down
to work together.
4. According to Hootstein’s so-called “four-pairs-of-shoes” model, an e-coach can take
on several roles in an online learning environment (2 zinnen samengetrokken). He can
be:
a. An instructor; he gives support by passing on suggestions and strategies,
linking subject matter and students’ previous knowledge (constructivism);
b. A social manager; he encourages students to work collaboratively, stimulates
interaction and builds a sense of community. In short, he creates circumstances
which are advantageous for learning
Dissertation for Master of Science - 48 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
c. A Programme manager; as an organiser, he supplies students with required
information and resources; he guards procedures and protects students from an
information overload.
d. A technical assistant; he makes sure students know how to work with systems
Hootstein states these roles meet the demands of coaching in a self-regulating, realistic
learning environment – or, in competence-based learning. (by Smits a.o., 2006)
Concluding, we may say that three aspects of Computer Mediated Communication are
essential to creating a digital learning -, work, - or assessment environment, namely:
• Digital skills of those involved;
• Social and motivation-centred patterns of communication;
• Moderating subject content
Finally, lecturers and students have to depent on the reliablety of the ICT network infra-
structure that had to be manages the applications and keeps them in good repair by the
support.
6. 2. EDUCATION
In Higher Education, “New Learning” is the subject of many discussions. It is often attributed
to the paradigm shift in education:
• from old, reproductive learning to new, constructive learning
• from lecturer/offer-centred to student/demand-centred education
• from instructive transfer to knowledge building developments
Many articles, websites, documents and policy reports seem to catch New Learning in the so-
called four C’s:
• Competencies: students aims to acquire their professional domain competencies in an
independent way;
• Constructivism: students construct and develop his own knowledge, capacities and
professional attitudes;
• Collaboration: students work together when performing professional tasks or making
products. They do so in a learning environment which is equal for them all, and which
resemble authentic work situations;
• Coaching: in which several aspects of a lecturer’s role as career coach and guide of a
collaborative team of students come out well.
(Jochems, Merrïenboer, Koper and Bastiaens, 2005) Dissertation for Master of Science - 49 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
The first three C’s will be dealt with in the sections below; the fourth C was described in the
previous section.
6. 2. 1. COMPETENCIES
During the many conferences and study days on the subject of competence-based learning, the
first topic under discussion is usually the meaning of the term “competence”.
We may begin by using a dictionary to clarify the word, and find under competency:
expertise, capability, being qualified to act or judge. Basically, “competence” expresses a
licence for people to do something in the professional area for which they have been found
suitable.
To explain the term further, we may have a look at professions and functions.
We see that that people’s functions in the work field are defined as follows: personnel is
expected to perform tasks under various conditions in several situations. Complexity of the
tasks in question may vary with the level of a function.
Next, I want to present the results from interviews regarding the subject of competence.
Many respondents mention capacities and professional attitudes as noticeable features of
competence. Those involved in the educational field perceive these features as an ability to act
expertly in a professional environment, and indicate it is all-important to apply them to
education. Work proficiency, then, should come into its own when students perform
professional tasks in a learning environment. One of the respondents clearly illustrates how
competencies should be perceived from an educational point of view:
1. Competencies are a complex of knowledge, capacities and attitudes fully integrated
into a course;
2. These competencies may be realised differently per level of complexity;
3. Once acquired, competencies give evidence of a student’s professional aptitude;
4. In several work fields and –situations. (R05)
One respondent indicates the complexity of the professional tasks to be performed increases
when students reach their final curricula; and another one defines competence-based learning
as “the core tasks of a profession” (R12) centring around a wide-ranging capacity to perform
tasks in various situations and environments.
A Dutch study (Van Merrïenboer, Van der Klink and Hendriks: “From Complication to
Compromise”, a report for the Council for Education) describes competencies as having six
Dissertation for Master of Science - 50 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
dimensions. These six dimensions can be subdivided into two groups of three, one group
mentioning the required competencies, the other group referring to the relevant ones.
Those dimensions are as follows:
Required:
1. Specification: the competencies should be specific for a profession, although general
competencies do come in;
2. Integration: competencies form a cluster of interrelated features such as knowledge,
capacities, and professional attitude;
3. durability; competencies are products of their time.
Relevant:
4. Competencies should focus on actions: that is, they should have a relation with a
particular task or situation;
5. Competencies can be learned: a student may gradually learn and develop his
competencies until these reach the required level;
6. Interdependence: competencies may be interrelated.
(Council for Education, 2002; Straetmans, 2004)
Each of the above mentioned dimensions may be applied in several ways, depending on the
educational level it is used for. In Higher Education, for instance, integration of
course/domain competencies will be a main concern, whereas this is of significantly less
importance in Vocational Education. As is to be expected, specification of competencies
prevails in Vocational Education. (Council for Education, 2002)
Researchers regularly make mention of the dimensions “integration” and “action focus”.
Jochems says:
“A competence is the ability to appropriately handle knowledge, perceptions, capacities and
attitudes when performing a task”.
Dochy and Nickmans, having compared the various definitions of competencies, suggest:
“A competence is a personal proficiency, which profiles as successful behaviour within a
particular context. It is a variable, which to some degree, may be developed. A competency
represents an integrated unity of knowledge, capacities, and attitudes; personal features and
situation-bound aspects of a profession may influence its development to a certain extent.”
(Dochy, 2005). Dochy mentions all of the above dimensions in his study.
Dissertation for Master of Science - 51 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
A final conclusion of my study may define the necessary features of course competencies
focusing on a professional domain as follows:
• Competencies should involve knowledge, capacities and attitudes;
• Tasks specific for tasks and actions in a professional environment should be integrated
with generic/general methodical and communicative behaviour;
• Required are levels and criteria defining learnability, development and growth;
• Complexity of the competency tasks should vary according to the level of the
professional situation;
• Perceptions as towards personal reflection should come into competency-based
education.
The above attempts to explain how the various dimensions can be realised, and made suitable
for assessment.
6. 2. 2. CONSTRUCTIVISM
The concept ‘constructivist learning’ has been referred to regularly in the course of this
investigation into an educational reform towards the ‘New Learning”. Ideas about
constructivist learning are similar to our definition of demand-centred education;
constructivism, namely, departs from a student’s independent advancement and assimilation
of information. According to Boekaerts and Simons, learning takes shape when students ‘are
dealing with subject matter in an active and constructive way’. (Boekaerts & Simons, 1993).
Steps involved in a constructivist learning model/theory comprise orientation as a starting
point, followed by awareness, and subsequently, an active conversion of acquired knowledge
into practical solutions. The student himself takes care of advancing his knowledge;
constructivism defines this as an accumulative process (Sanden, 2001). Constructivism
emphasizes the importance of both learning environment and collaborative learning; this
corresponds with the adaptive learning situation as currently employed at the Deventer School
of CII. The lecturer’s role in independent learning should be seen as an encouraging one: he
coaches the students; and it is the educational organisation itself which supplies them with the
necessary learning conditions.
Boekaert’s and Simon’s theory can be summarized as below:
• Students search for information which may help them to build up their knowledge. Their
learning environment should challenge them to accomplish this. Students should be self-
motivated and personally direct his information acquisition Dissertation for Master of Science - 52 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
• It is the students themselves who find out what the information gathered means
• Digesting information develops through experience; this results in knowledge which
gives meaning to information
• New information in itself does not constitute new knowledge; only when students reflect
on their experiences, information will crystallize into a meaningful tool.
• When trying to convert information into knowledge, a student will need the support of
others in his team. The lecturer functions as a coach guarding content and quality of his
learning process.
• Feedback and reflection from team mates and lecturer will stimulate the student to turn
his new information into knowledge.
(Social) constructivism, then, sees students being engaged in interaction with fellow-students
and lecturers as essential; this should prevent their possibly misconstruing knowledge.
Many sources refer to the Shuel’s developmental stages:
“Learning is an active, constructive, cumulative and goal-directed process…” (Shuel, 1988)
Referring to Simons and Lodewijks, Bouman and Van Lisdonk define the following features
of social constructivism:
1. A student actively processes new information in order to learn in a meaningful way;
2. A student attempts to comprehend new information and compare it to other information.
He should then construct it to knowledge – implying, he intends to get a thorough,
personal understanding of basic information;
3. A student places knowledge cumulatively, i.e. he evaluates if it bears any relation to his
previous knowledge and in how far the latter can be used. In this way, new knowledge
will be generated;
4. A student is goal-directed in his attempts to give his newly gained knowledge the right
place in his personal learning process.
5. Learning is a reflective and diagnostic process. The student monitors his learning,
assesses himself and others in order to detect gaps in his knowledge, and to check if his
learning still pursues his end goals
6. Learning is a context-dependent, social process. It takes place during interaction with
others (fellow-students, lecturers, work placement coaches, etc) in a cultural
(professional) and historical context
(Bouman & Lisdonk, 2005)
Dissertation for Master of Science - 53 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
In this way the student will realize his goals by gaining competence.
Points of special attention in constructivist learning should be taken into account; first, a
danger may be involved that various realities exist among students as towards interpretation
of facts. Further, it demands from students to function on a rather abstract level of cognitive,
independent learning. They are expected to personally search and gather information, and
enlarge it in practice.
The learning model/theory of (social) constructivism should be seen as a component of a
pedagogical approach which means to teach students how to learn independently.
6. 2. 3. COLLABORTIVE
The concept of collaborative, or co-operative learning aims to stimulate a group of students to
work on and learn professional tasks together in a project group. The courses BI and IDM at
the Deventer School of CII arranges for students to work in groups on an authentic case, or
even on a real-life professional task in a educational workshop (Du.: OWP) per semester. The
constitution of the group varies per semester, and with it, the roles of the students. This
depends on students’ individual PDP and ISP, in which they indicated what competencies
they prefer to acquire for that period. With respect to technological developments, it is
important to consider how ICT means can support the proceedings. The courses BI and IDM
have at their disposal the facilities supplied by Saxion’s organisation; the process ‘Education’
saw the introduction of its own website, on which materials for the semester themes are
presented. The programme Quickplace enables students to work together in groups, and the
Digital Portfolio is meant to have them collect their own materials. These are independent
applications that may possibly get definite shape when realised within a structure of
Computer Supported Collaborative learning (CSCL).
In order to make clear what CSCL is about, we will briefly refer to its essential points.
Wilfred Rubens explains CSCL is a comprehensive term; meaning, it can be realised along
the lines of the various educational concepts fashionable in learning environments (Rubens,
2003). Explaining this notion into further detail, Paul Kirschner a.o. mention the following
aspects of CSCL:
• the educational context of collaborative learning;
• the social context of a group;
• the technological context of PC support
(Kirschner, 2004; Bouman & van Lisdonk, 2005) Dissertation for Master of Science - 54 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
Essential here is collaborative learning supported by ICT instruments. Anna Elske Veldhuis-
Diermanse describes CSCL as follows:
“Computer Supported Collaborative Learning aims to stimulate and support learners to
expand their knowledge together. In this context, learning should be seen as a dynamic
process of knowledge construction. A CSCL environment offers opportunities to
negotiate about knowledge. In CSCL, then, interaction between learners in essential”.
(Veldhuis-Diermanse, 2002)
This description is coherent with the concept of social-constructivist learning as explained in
the process ‘Education’ - Constructivism (6.2.2) of this master’s dissertation.
Next, the SALDO literature reseach project of the Digital University (DU) in the Netherlands
will be looked into as a possible form of CSCL within Higher Education. After extensive
survey of professional literature, The SALDO project was able to identify five requirements a
CSCL environment should meet:
Diagram 15: SALDO Themes for collabortive learning
There are three central themes belonging to the main process ‘Education’:
1. Assignments/products; students perform tasks in groups
2. Group processes; in order to effect positive collaboration, it will be urgent to pay
attention to processes evolving in the group and to take care of collaboration between
the participants.
3. Tasks/roles; these should be divided among the students of a team, including the role of
coordinator/chairman.
Dissertation for Master of Science - 55 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
The remaining main processes in this model are:
4. Coaching
5. Assessment;
(SALDO DU Project, 2004)
The lecturer is expected to give shape to the process ‘coaching’; in some cases he will be
involved in ‘assessment’s well.
The project developed a matrix for a logical succession of process phases to be:
Designing > preparing > executing > guarding quality > delivering assignments.
Veldhuis-Diermanse’s investigation of CSCL applications pointed to factors, which might
guarantee success in competence-based learning. We mention them here:
• Experience and motivation of students;
• A well-organised course/theme/semester;
• Frequent arrangement for meetings between lecturers and students;
• Complicated assignments provoking debate and requiring negotiations about
knowledge;
• A user-friendly CSCL environment;
• Ample time to get used to the ICT programme;
• Sufficient opportunity to work on the tasks, to study contributions of participants and
react;
• Structuring tasks in advance;
• Brainstorming and subsequent discussion of tasks
• Evaluating students’ contributions in a CSCL context, and possibly, assessing a final
product.
It is important to realise staff and organisation should have a major role in accomplishing
these factors. Deploying multi-disciplinary lecturers will bring about success as well. In this
respect, we refer to the various roles of the lecturer in competence-based learning.
6. 2. 4. ELO / DLO / VLE
Digital education can be defined as a compilation of three abbreviations: ELO, the Electronic
Learning Environment; DLO, the Digital Learning Environment; VLE, the Virtual Learning
Environment. Together, these terms constitute the web-enabled environment for electronic
learning. Other terms in use are e-Learning and Blended Learning, the former pointing to a Dissertation for Master of Science - 56 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
mainly independent electronic learning environment, the latter to an environment connected
to a Face-to-Face course.
This master study focuses on a Blended Learning Environment, as can be concluded from a
respondent’s (student’s) reaction:
“Coaching is an important aspect, you shouldn’t be entirely dependent on a digital
learning environment. In other words, ‘Blended Learning’ is the way to go; however,
for some issues Face-to-Face moments remain necessary..” [R03]
Main point of the various educational theorist quoted in this dissertation is the influence of an
information society on education. Diana Oblinger described the phenomenon of the digital
student from the ‘Net-generation’. Simons and Smits investigated the changing roles of a
lecturer as a developer and coach in a digital environment. Merrienboer indicated digital
didactics may be developed employing the 4C/ID Model; four basic components may
constitute a competence-based, constructivist, electronic learning environment. The SALDO
DU project defined the requirements for a CSCL environment; as Veldhuis-Diermanse
mentioned in her study, it is essential to pay attention to the factors determining success.
The Saxion philosophy acknowledges the fact that a competence-based learning environment
– in which an independently and actively learning student sets up his own learning course –
needs an electronic learning environment departing from the demands of a self-regulating
student.
In one of the interviews, a respondent mentioned how an electronic learning environment
should meet the requirements.
Student Docent - an ISP which contains the student’s
preferences for his learning course - a student fills in forms in which he
mentions his preferences - saving evidence, organisational aspects,
privacy, authenticity, security - e-mail communication and Computer
Supported Collaborative Learning (CSCL)
- offering multimedial, asynchronic information resources
- taking care of supportive education - being responsible for allocating
information/links/resources - dialogue-centred communication
Table3: points of attention for an electronic learning environment based on an interview with a respondent [R04] (2005)
Dissertation for Master of Science - 57 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
Concerning right of ownership and freedom to use the e-environment in his own personal
way: the student wants to give shape to his learning environment according to his personal
learning course. The lecturer wants to be able to do the same with respect to his roles and his
involvement in themes. Support, finally, wants to know about both students’ and lecturers’
decisions in order to take full responsibility for the facilities to be provided in a learning
environment. In the diagram below, this is represented as follows:
Figuur 16: Model owner and responsibility
A student can dispose of his own Digital Portfolio; if he has any, he may attach his personal
blog to it. He fills in his POP and ISP, and if this has been approved of by the career coach, he
will sign up for the semester theme and sets up his RSS-feed in order to keep informed about
relevant items concerning his learning course. Next, he asks his career coach to inspect his
POP and ISP, and requests for feedback from his lecturer; and finally, he requests for an
assessment as soon as he has gathered sufficient materials.
The rings in the diagram represent a participant’s mutually shared right of ownership; they
indicate how he should employ the systems and all others involved. If a significant overlap in
ownership shows, the organisation will be involved
The situations described above should be taken as essential for designing/developing/realising
an electronic learning environment. Probably, no existing environment meets these
requirements. The Sakai development, then, may be improve matters because many interested
parties help to shape Sakai. Sakai is an international, educational open source community
attempting to develop a general e-learning platform for Higher Education (Sakai, 2005-06).
Taking international standards in account, it opens up many possibilities for Higher
Education.
Dissertation for Master of Science - 58 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
6. 3. ASSESSMENT
The concept of ‘assessment’ is essential in competence-based education. Main point here is
whether a student can prove his competent. A first definition of assessment originates from
the theory of Human Resource Management (HRM), where assessment is employed as a
means to support application procedures. (Hendriks, Schoonman, 2006). Meeting selection
criteria is not the first concern of education; personal development, feedback and evaluation
are its predominant aims (Dochy, 2005).
The early 90-ties saw the formation of an Assessment Centre (AC) in adult education; it
sought to determine what knowledge and experience is already present at the start of a
learning course. AC means to ‘picture a student’s current state of knowledge and competence’
(Dochy/Bouwens, 1992).
Higher Education implemented this notion in their development of Assessment Development
Centres (ADC). This method of assessment recognizes a student’s own capacities and
knowledge at a given moment in time, and indicates what course his personal learning process
might take; it will measure a particular competence within a conditioned environment. A
respondent from Windesheim School of Higher Education pointed out that the ADC is used at
a student’s entry: “at his introduction, a student is tested by means of an Assessment Centre.
He is required to answer questions like, Who am I; what do I want to learn. The assessment
centre at Windesheim, then, gives a clear outline of the learning tasks to be performed, and
the way in which students are assessed” (R09)
Bruijns a.o. defined ADC as “an intensive instrument of assessment, by means of which
complex behaviour in situation-bound environments can be observed and evaluated” (Bruijns
a.o., 1999). This quote serves as a point of departure for a more detailed definition of the term
“Assessment” in competence-based learning.
6. 3. 1. ASSESSMENT AND COMPETENCE-BASED EDUCATION
This paragraph aims to get a clear perception of meaning, place and function of the term
‘Assessment’. The word is a collective noun for various forms of competency-based testing
Dochy explained assessment as “measuring competencies, knowledge and capacities in
authentic situations (Dochy a.o., 2002); Birenbaum mentions assessment as an integrated
unity of learning, instructing and evaluating in an assessment environment (Birenbaum, 2003)
Dissertation for Master of Science - 59 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
Segers, Dochy and Cassalar mention six shifts of Assessment reforms:
Old Learning New Learning Shift
Authenticity Not related to context applying
Professional context, capacities
Number of measurements Single Multiple Level of understanding Low High Dimensions of intelligence Few many/multidisciplinary Relation to the learning process
Isolated tests Integrated assessment
Responsibility Lecturer Student Table 4: Characteristics for Assessment (Segers e.o. , 2003) Competency-based learning closely follows the educational paradigm shift and will be
implemented as an integrated unity. We see a change from single tests to a broad, context-
bound and integrated method of assessing.
The definitions and developments mentioned above recognize various essential elements of
assessment in a competence-based learning environment. In retrospect, then, the following
elements are all-important in Assessment:
• Assessment in competence-based learning is a integrated, multidisciplinary unity;
• Evident capacities, and professional attitude should meet the competence criteria
• A final professional product should give proof of the competence criteria;
• Tasks and activities are to be performed in an authentic professional situation, or in a
situation closely resembling reality
• Complexity of the tasks relate to multidimensional levels of competency
The case study of the Digital Assessment Porfolio (DAP) as employed at the Deventer School
of CII helped to elaborate on the quoted features (Vries, 2005).
Birenbaum (2000) defines the process stages in a learning environment as Instruction –
Learning – Assessment (ILA). This ILA model may be succeeded by the EA Model, which
mentions the stages Coaching – Education – Assessment (CEA). In the new CEA model,
Instruction and Learning are part of the broader concept of Learning. This concept involves
both the supportive roles of the lecturer as and the function of a digital learning environment.
6. 3. 2. ASSESSMENT IN HIGHER EDUCATION
The Saxion department of Education and Quality made an outline of various testing methods.
Applicability of each method indicats by checking the boxes of the columns Knowledge,
Capacities and Attitude (Tests, Saxion 2004) Dissertation for Master of Science - 60 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
Suitable Testing methods related to knowledge, capacities and attitude
Cap
acities
Knowled
ge
Attitu
de
Capacity test ⌧ Knowledge test ⌧ Case/Overall test ⌧ ⌧ Reflection ⌧ ⌧
Test of conduct / perfomance ⌧ ⌧
peer-assessment and 360 degrees feedback ⌧ ⌧
work test ⌧ ⌧ ⌧
Simulation ⌧ ⌧ ⌧
Professional product ⌧ ⌧ ⌧
Portfolio assessment ⌧ ⌧ ⌧
Integral assessment (Du: IBM) ⌧ ⌧ ⌧
Table 5: Suitable testing methods
Saxion tried to establish an Integral Assessment in which a portfolio serves as evidence of the
student’s competencies during an assessment interview.
Integral assessments are moments at which quality and quantity of the evidence is judged. An
assessment portfolio belongs to the evidence material, together with the obligatory
competence criteria formulated by the course.
A portfolio assessment should be a file containing evidence of a coherent set of competencies
or professional tasks. It is evaluated in accordance with the quality criteria (content, level,
quantity) defined by the domain/course (Tests, Saxion 2004)
Pronouncements of interviewed key figures point to the following features of competence-
based assessment:
• The new educational paradigm of self-regulating learning requires freer methods of
assessment. This means students should be allowed to decide when he wants to be
tested. In addition, it is advisable to develop several methods testing the same
competencies. All methods should unequivocally describe the criteria of the related
competencies. Direct implication for a learning environment is a certain degree of
freedom of ‘time’ and ‘method’ of testing.
Dissertation for Master of Science - 61 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
• Various kinds of competencies know various testing methods. We refer to the general
HBO competencies, c.q. the Dublin indicators, in which tests of conduct concerning
communicative skills have a distinctive role. These methods require a correct
implementation of formative and summative testing, as well as adequate feedback
moments and reflection. In addition, diagnostic tests - or, the conditional knowledge
tests - should be paid attention to.
• Preferably, authentic tests should be held in an actual professional environment. This
would really show a student’s capacities. Therefore, it is recommended to have
professional skills tested integrally in an ADC.
• The portfolio, in which students collected various professional products, may be another
important instrument. Students can have their portfolio evaluated summatively by
independent assessors during an assessment interview. PPS, the Protocol Portfolio
Scoring, may be a fit method for assessors.
• Respondents indicated al least three integral elements are essential for a good
assessment, namely, knowledge, capacities and attitudes. Some of them emphasized
competence assessment is a random indication, to be taken as a moment’s reflection
during a process of continuous growth, development, and broadening of competencies.
6. 3. 3. CASE STUDY CII DEVENTER
Per semester, students of the courses BI & IDM decide which project they want to participate
in order to acquire their desired competencies. The competencies show five levels of
increasing complexity; the criteria for each are clearly defined. The student attempts to meet
these criteria by carefully planning his activities and collecting his materials. He belongs to a
project group, and works on an authentic case together with other students. He personally
decides how to accomplish his tasks, and when he wants to have his achievements evaluated
formatively. The Route Assessment System (RAS) is a supportive application enhancing
online communication between student and lecturer. The student collect his materials in his
portfolio; if he wants to have a formative evaluation, the RAS will serve him to present the
materials collected so far to his lecturer. Likewise, the lecturer will pass his formative
evaluation via the RAS. When both are recorded in this way, the collected formative moments
will constitute the final summative assessment; this as soon as the student has assimilated the
feedback into new professional products, reflections and plans of action. When he has carried
out the various competencies (three at least), the student will then set up his ‘defence’ – the Dissertation for Master of Science - 62 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
chosen metaphor within the courses BI & IDM. In it, the student explains how he acquired the
competencies and refers to the evidence in his portfolio. Next, he can request an assessment
via the RAS. Students indicated they would welcome some kind of agenda built into the RAS.
Having done his request, an assessment by two assessors will take place. In the assessment
interview, the RAS will help to determine if the student asked for sufficient formative
feedback per competency level; and the assessment portfolio will check if he assimilated it to
guarantee further advancement.
Diagram 17: Stages of assessment (BI & IDM)
The courses BI & IDM at Saxion University employ the following testing methods:
Formative moments:
• Tests of conduct and capacities when professional tasks concerning general
competencies are executed; video footage may present a student’s advancement and
should be accompanied by preparatory documents;
• 360 degrees feedback forms (co- and peer testing method) by fellow-students, subject
teachers, and if possible, external professionals. These should be worked out in -
• Reflective reports resulting in plans for new actions and improvement;
• Diagnostic knowledge tests showing advancement in competence acquisition;
• A personal process log, and –
Dissertation for Master of Science - 63 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
• A planning scheme and task allotment in the project group;
• Requesting formative feedback during the process of professional product development;
• (Hands-on testing method)
Assessment Summation: • A defence referring to the materials that may prove the level of competency achieved;
• A portfolio containing materials (documents, reports, products, formative responses,
reflections, professional products, tests of conduct and capacities, etc.)
The testing methods mentioned above are the foundation of the conceptual, functional
description of the Digital Competence-based Assessment Frame.
Dissertation for Master of Science - 64 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
7. DIGITAL COMPETENCE-BASED ASSESSMENT FRAME
The ‘Assessment Frame’, which originated from the EA Model, consists of nine
interconnected cubes. These cubes will be described in relation to the process of assessment
in competence-based learning. Results of these descriptions may be seen as the functional
requirements for constructing a concept design of the Digital Competence-based Assessment
frame (DCAF).
Like any other process in the EA Model, the process assessment takes participants and
approach level into account.
Diagram 18: Assessment Frame - EA Model
In the previous chapters, competence-based learning was described in terms of the EA Model.
Participants, Levels and Processes were defined theoretically and practically per frame.
Construction of the concept of the DCA Frame, then, has been firmly based on the findings in
the preceding chapters.
7. 1. FOUNDATION OF THE DCAF
Looking at the basic components which constitute the DCAF, we follow the processes from a
student’s perspective this approach will explain the inner workings of the frame.
In professional literature, this approach is called ‘scenario of use’: several representatives of a
particular target group are required to describe their actions (Preece, 2002). Here,
observations and findings of the RAS case study at the Deventer School of CII are illustrative
of this approach.
The student, then, is the protagonist in this paragraph; he wishes to gain a qualification in
order to enter the work field well armed.
Dissertation for Master of Science - 65 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
The student knows what profession he strives for, and chose a course accordingly. The
course, departing from a policy of the actively and independently learning student, provides
him with the necessary means to give shape to his learning route. In order to acquire the
bachelor’s degree he opted for, the student sets up his personal learning process per semester.
In doing so, he indicates which competencies he will work at, and how he will use the
learning environment to his advantage The Personal Learning Process (Du.: POP) established
as such will show a student’s activities, and serve as a basis to assess the student’s
competence acquisition per semester.
Subsequently, the course will have to judge whether a student’s POP meets the domain
competence criteria as laid down for the bachelor’s degree he seeks to gain. Persons
approving of his POP make up the board of examiners. At the Deventer School of CII, these
persons are the career coach and a semester’s theme coach. If a student’s POP deviates
significantly from the bachelor’s course in question, the board of examiners will have to
approve of it. This approval of a student’s POP is essential for an adequate functioning of the
DCA frame. Once he has decided on the competencies to be acquired in a particular period
and taken them up in his POP, his plans to perform activities advancing his attempts to
become a capable professional will take shape.
Activities taken up in a student’s POP may comprise, for instance: working on specific
professional products within the context of a thematic project-centred team; working on
specific (external) products in a educational workshop; or working on tasks at an external
company in the work field. It is all-important that features of the planned activities, such as
actions and products involved, are laid down clearly in his POP; this will give evidence of a
student’s progress in competence acquisition. In other words, if a student’s POP gives an
accurate record of his learning route, it will enable him to independently manage his quest for
competence acquisition per period.
It is the student himself, then, who should check if he meets the criteria of his preferred
competencies by keeping his POP up to date. Does the POP point to a sufficient foundation of
knowledge; does it show he has made progress in gaining the required capacities, and are
there indications he may develop the right professional attitude? These questions urge a
student to take and make sure there are sufficient formative moments within a case, a
workshop or work placement. If he wants to answer those questions, he will have to do,
among other things: diagnostic knowledge tests, take up feedback queries about the products
he made and apply his findings to new products, insert 360 degrees feedback moments, and
Dissertation for Master of Science - 66 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
keep a log which will enable him to reflect on matters and find the means to improve his plans
of action. The assembled results will contribute to a summative assessment of the
competencies to be required.
Summative assessments depend on the competencies in question and are performed according
to the assessment methods employed by the course. This may take place in a Assessment
Development Centre (ADC) or any other form/environment by which/in which professional
attitudes are tested. In this respect, Saxion University mentions Integral Assessment Moments
(Du. IBM) in its evaluation policy; these may involve case assessments, or an interview
evaluating the evidence collected in a Portfolio.
Concluding, we can identify three essential process stages in the DCA Frame, namely:
a. At the start of the educational process, the student lays down the competencies he
is going to work on in a particular period. He takes them up in his POP
b. The course of his personal learning process itself is another stage; the student
develops his competencies in the learning environment of his choice. He performs
tasks, makes professional products, and gathers formative feedback
c. Finally, a summative assessment stage takes place, which determines whether a
student meets the criteria for the required competencies.
This tripartite division is more or less similar to the three decision stages Straetmans defined
as essentially belonging to the role of Assessor. In his lectorale speech Straetmans states the
lecturer assesses the student at three stages during his personal learning process, namely, at
intake, progress and certification (Straetmans, 2006). When compared to the tripartition,
“Coaching – Education – Assessment” in the EA Model, the models show similarities as well
as differences. This observation makes differentiation between the Assessment Frame and
Straetmans’ stages urgent; therefore, the terms A-coaching, A-education and A-assessment
will be used for the DCAF.
The “A-terms” describe the following:
• A-Coaching involves those parts which are essential for the assessment process,
such as POP and study programme; these set up the competencies for a particular
period. Intake and placement come into it.
• A-education, the stage during which formative tests are gathered, is similar to
Straetman’s process of Progress. Draaijer includes orientation and practice;
acquiring and applying knowledge.
Dissertation for Master of Science - 67 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
• A-Assessment determines if certain competencies have been acquired through a
process of measuring and evaluation, and if this can be converted to
‘certification’, c.q. accreditation. (Draaijer a.o., 2004)
The tripartition in the EA structure will serve as a model for the DCA Frame.
7. 2. DCAF-SYSTEM
The three elements of the assessment frame will be substantiated into a concept model
according to Koper’s domain model for e-Learning. Therefore, Koper’s model will be looked
into briefly.
According to Koper, it is important to get a clear picture of e-Learning and its applications in
a relevant context. In education, e-Learning should be a system which “supports the primary
educational functions” (Koper, 2005). Those who are learning form the primary input. This
implies the educational system should primarily concern itself with turning out transformed
learners, well-equipped with acquired competencies. The educational process can be
explained as a transformational development, in which lecturers and organisation should be
taken as factors determining input and output. I will now investigate Digital Competence-
based Assessment and set it up as a system analogous to Koper’s model of e-Learning.
Diagram 19: Concept DCAF-system
Dissertation for Master of Science - 68 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
The division into three elements of assessment are similar to the three processes described in
the EA Model. The EA processes will serve as a model for our construction of the DCA
Frame.
7. 2. 1. DESCRIBING THE DCAF CONCEPT
The functional and conceptual parts of the DCA Frame have been founded on the tripartite
division into processes as shown in appendix 3. Activities of students and lecturers during
any of the process stages “scenario of use” have defined the shape of the model. These
activities may have consequences for the digitalisation to be generated by the DCAF system.
Possible ambiguities have been explained into more detail and can be found back in the
concept as points for special attention.
Dissertation for Master of Science - 69 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
A-COACHING - POP - PDP
Activity Points of interest DCAF-system
Student - The student sets up his POP (PDP) for a semester;
planning, learning activities, reflecting on personal learning and development.
- The student decides what competencies he wishes to acquire for that period;
- The student indicates how he wants to develop his personal learning process, and what learning activities he will start
Learning environment (micro level) - The student’s career coach gives his first feedback on
the student’s plans to see if these are realistic - The career coach and theme coach for any semester
decide positively or negatively if plans and competencies to be acquired meet the course criteria, and if these can be achieved within that period
- If this is a positive, the student is allowed to carry out his plans.
- If this is a negative, the board of examiners should be consulted.
[Learner; student] [Assessors; career coach, theme coach] [Learning objects; POP - PDP]
[Registration & Management] System of basic data containing personal information, previous education, Log In. [Assessment Methods and Tools] For an adequate use of the instruments Portfolio and Pop, Assessment Methods and Tools should be related to the DCA Frame [Items and Content] Course- or Domain competence criteria and their credits. The student can select the desired competencies for his POP (PDP) for that period Forms and templates helping the student to execute his personal learning route. In addition, a student should have the means to do a digital learning style test and/or SWOT analysis These may be entail course-specific parts.
Dissertation for Master of Science - 70 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
A-COACHING - POP - PDP
Activity Points of interest DCAF-system
Some points of special attention should be mentioned as towards a further development and realisation of assessment during the coaching process. Working out the DCAF system raises the following questions: What should be recorded where, and how; Concern here is the student/learner who set up his POP (PDP) in his Portfolio. If his career coach and theme coach have approved his POP (PDP), he now has a document which represents a particular value; it indicates which selected competencies give shape to his personal learning process per period. Where and how, then, did he record his POP (PDP), making it available for the lecturer involved; and how will it be authorised. Who is the proprietor of what, and who has responsibility for what; who has access to what, how are documents made available for who; who authorises documents and digital evidence.
It is important to decide on unambiguous rules and procedures in this matter. Everybody should know his rights and duties, as well as the degree of freedom to be enjoyed.
Dissertation for Master of Science - 71 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
A-EDUCATION - FORMATIVE
Activity Points of interest DCAF-system
Student - the student participates in a theme project group. He and
his fellow-students work on a case collaboratively and execute specific professional tasks;
- the student takes responsibility for tasks and assumes roles in accordance with his chosen competencies, in order to deliver the professional products in question;
- the student collects formative feedback moments which illustrate his tasks; he may have himself evaluated, for instance, as chairman of the project group.
Learning environment (micro level) - The theme coach guides a project group working on a
case. If requested, he will give formative feedback or information about the subject. The coach guides the students through the working process, encourages the group as a whole and challenges them to broaden the theoretic case or improve professional assignments.
- In any of his roles, a lecturer may be requested to give expertise formative evaluations of students
[Learner; student] [Assessors; lecturers (roles of), peers and the theme coach] [Learning objects; case, project definition, assignments]
Dissertation for Master of Science - 72 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
A-EDUCATION - FORMATIVE
Activity Points of interest DCAF-system
[Registration & Management] Departing from his ISP, the student signs up for a semester theme. After registration and placement, a specific CSCL environment will be assigned to him. [Assessment Methods and Tools]Having collected all necessary materials, the student requests his formative tests by means of the RAS. Tools: CSCL environment, RAS and Portfolio. [Items & Content] Course and domain competencies as laid down in his ISP are in accordance with a student’s personal CSCL environment. Professional tasks and products to be delivered can be realised in that very environment. A selection can be made of the various formative tests available
In an electronic learning environment, the CSCL platform should contain relevant formative test. Personal organisation of a CSCL environment per semester theme should be visualised clearly. This goes for students as well as teachers. The students themselves should manage the accommodations for the CSCL project group. How can formative tests take place in a specific CSCL theme environment? How can individual tasks and group assignments be executed with respect to students’ portfolios
(Think of right of ownership, responsibility and authenticity)
Dissertation for Master of Science - 73 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
A-ASSESSMENT - SUMMATIVE
Activity Points of interest DCAF-system
Student - the student compiles an assessment portfolio in his DPF,
centring around his Defence. It should refer to the evidence.
- The student requests an assessment for his assessment portfolio
- The student has an assessment interview with two assessors
Learning environment (micro level) - the assessment coordinator checks if the student’s
assessment portfolio meets the requirements; it should contain: the Defence referring to the evidence, a minimum of three competencies, per competence at least one feedback moment
- If the portfolio meets the requirements insufficiently, the student receives an e-mail telling him this
- If it is accepted, the assessment coordinator will choose two subject assessors and arrange for an assessment interview, space and facilities. Everybody is informed.
- The assessment coordinator sends the portfolio to the appointed assessors.
- The assessors interview the student and determine which competencies have been acquired.
[Learner; student] [Assessors; assessment -coordinator, lecturers as assessors] [Learning object; Pleidooi, reflection, professional tasks and practices]
Dissertation for Master of Science - 74 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
A-ASSESSMENT - SUMMATIVE
Activity Points of interest DCAF-system
[Registration & Management] The student requests an assessment for the competencies in question via the RAS. [Assessment Methods and Tools]The student requests an assessment via RAS and sends it together with his assessment portfolio Tools: DPF & RAS. [Items & Content] Course/domain competencies of his ISP are described in his request.
The RAS concept offers students an opportunity to request an assessment which is not time- and place-dependent. As a means to facilitate a digitally supported, competence-based assessment, it is a valuable tool in electronic learning environment. The level of competency can be tested conditionally by means of the RAS With a view to right of ownership and authenticity of the materials to be evaluated – this includes formative tests - it is essential documents are securely saved outside the student’s portfolio as well. As soon as the student sets up his ISP and chooses his competencies, the following issues should be taken into account by those involved: inspection and checking of the criteria, conditions of feedback tests and finally, registration of the assessment. (Pass/Fail)
Dissertation for Master of Science - 75 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
7. 2. 2. THE DCAF AND THE LEARNING CYCLE The conceptual description of the DCAF shows that we are dealing with a number of
independent applications and three database systems underlying it. The six componements of
the DCAF frame have a mutual relationship, depending on user and function. The
components of the DCAF Frame previously referred to will be explained in terms of the
learning cycle.
Diagram 20: Components for DCAF
ORIENTATION
The student sets up his new pop (or he continues working on his former)
He decides how his ISP will look like, indicating which competencies he wishes to acquire in
the coming period, and how. He may add acquired competencies from the SIS (Student
Information System).
He discusses POP and ISP with his career coach, who may or may not approve of it. Next, he
saves the POP as a piece of evidence in his personal DPF and in the DMS (Document
Management System), this in order to secure the authentic document. Date, relevant period ,
name of the career coach etc. should be included.
Dissertation for Master of Science - 76 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
PLANNING
The ISP can now be activated; competencies from the Course Competence System (Du.:
OCS) are officially recorded in the student’s ISP. These data are taken up into the RAS as
well. The ISP is saved in the same secure way as the DMS (Document Management System). EXECUTION
Working on his tasks, the student collects all kinds of materials in his DPF, such as
documents, professional products, tests of conduct, diagnostic tests, formative feedback etc.
The student works on his tasks and requests for an assessment via the RAS on a regular basis.
His request should contain references to the evidence in his digital portfolio. The lecturer
gives a formative assessment; the evidence and the formative feedback are recorded in the
RAS, and acknowledged as “completed” in the DMS. EVALUATION
As soon as the student has collected sufficient materials, he checks if he meets the criteria. If
he thinks he does, he will request an assessment. The assessment coordinator checks his
request, after which an assessment interview can take place.
The request, date and location for its execution as well as assigned assessors are recorded in
the RAS. Defence and evidence in the Portfolio are sent to the assessors.. After the
assessment interview, the assessors will write a protocol of successfully completed
competencies to the RAS. Defence and evidence are recorded in the DMS; the acquired
competencies are added to a student’s course profile.
7. 3. THE DCAF, PPS AND THE 4C/ID
When describing the processes Education and Assessment, we indicated the competencies
regarding Knowledge, Capacities and Attitudes should be integrated into assessment. In
addition, we mentioned multidisciplinary complexity as an aspect to pay attention to. Further,
it is essential to strife for competence acquirement to take place in a professional
environment, or at least in an environment which is as authentic as possible. Levels of
complexity in competence acquisition should be defined clearly in order to test them
adequately.
The course BI & IDM has formulated a division into five levels of progressive complexity:
Dissertation for Master of Science - 77 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
Diagram 21: Five levels of competence
Examples of levels:
Level 1 - a student can function as chairman in a homogenous group;
Level 5 – a student can lead a meeting of a project group in which people from various
disciplines and departments have a debate about the pros and cons of an issue.
Criteria describing attitude/task and (partial) professional products have been determined per
level; the student is tested accordingly. During the assessment interview, in which a student’s
working process and results are discussed, it is evaluated if the three main elements of a
competence (Knowledge, Capacities and Attitudes) meet the criteria and can be proved. The
assessors will determine which competencies have been acquired.
Diagram 21: Authenticity performance pyramid according to Miller (1990)
Dissertation for Master of Science - 78 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
Assessment will mainly concern itself with actions involving context and complexity of the
competence level in question; Miller’s pyramid serves as an assessing tool. This model checks
underlying knowledge and capacities for their supportive value (diagnostic test); it checks if
the student acted professionally; it broadens complexity of the task and judges a student’s
capacity to perform independently.
One of the respondents said only those competencies can be assessed that have been
thoroughly defined:
“It is essential that assessment takes place according to the criteria of a
competence. The professional task must be made operational by following a
standard set of criteria. This set makes out the elements to be assessed.” [R12]
Criteria defined per competence, then, are essential for an effective assessment of
competencies.
7. 3. 1. PPS AND THE DCAF
In order to make adequate use of the Protocol Portfolio Scoring (PPS), the competencies and
their criteria should meet certain requirements as to quality. Straetmans labels these
requirements as follows:
• Accuracy: the criteria for the competencies should be formulated unequivocally,
so as to avoid differences of interpretation among assessors in various situations.
• Generalization; the competence to be assessed should not depend on the task in
question: ‘An assessor must be able to rate a student’s accomplishment in
competence acquisition, no matter if task, criteria and context differ
• Extrapolation; to what extent does the performed task reflect the actual
professional context. In this respect, it is important to guarantee the authenticity of
a case, and preferably, to perform it in a professional environment. The latter is
usually not realistic, this with an eye to expenses. An educational workshop may
offer a solution for this obstacle (Straetmans, 2004/2006)
Assessing the acquired competencies needs to be reliable and representative. Therefore, the
Deventer School of CII decided to have assessment interviews take place in a setting of
student and two assessors. The PPS definitely requires a blend of testing methods to evaluate
the various criteria of a competence. Applications of the PPS methodology are explained
below: Dissertation for Master of Science - 79 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
a. A student should collect three formative assessments with a minimum of two
different testing methods per competence level. A competence criterium, then,
should be tested twice.
b. The assessors determine the following formative aspects:
the competence level tested formatively
• What testing method was used,
• What was tested: task execution, feedback on a document or product,
• Advice should be taken down,
• Which criteria could be tested unambiguously (not all situations will clearly
define criteria),
• The assessor rates the criteria using a five-points scale: [- -] [-] [-/+] [+] [++],
c. The student collects formative assessments and learns from them:
• For executing tasks he may request a second formative moment,
• A student knows to improve his professional product by taking the feedback
into account. He keeps documents recording his old and new version,
• He keeps a process log, in which he mentions how he assimilated the
formative feedback.
d. When a students requests an assessment, he can judge for himself if his portfolio
meets the requirements, it should contain:
• A minimum of three competencies,
• Per competence a minimum of three formative assessments, each of them
tested at least in two different ways,
• All criteria have been assessed twice.
e. Having checked the request, possibly by means of the RAS, the assessment
coordinator appoints two assessors. The assessors have the Defence, evidence,
survey of implemented testing methods and rating per competence criteria a their
disposal (see table 6)
f. At the end of an assessment interview (IBM), the assessors fill in a protocol,
indicating which competencies have been completed successfully. If the student
has failed any of them, they will give good arguments for their decision. The two
assessors rate a student’s personal and professional performance during the
interview according to the Dublin-descriptors. The Dublin- descriptors are:
knowledge and understanding, there application; forming of judgement,
Dissertation for Master of Science - 80 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
communication and learning capacity. OSC, SIS and RAS are used to register the
assessors’ decisions.
In this way, the PPS methodology could make out an important part of the DCA Frame.
Formative test survey of……………… Competency: SETTING UP INFORMATION REQUIREMENTS
Level 3 For a homogenous target group Assessment/evidence
01 Assessment/evidence
02 Assessment/evidence
03 Assessment/evidence
04
Criteria Test Score Test Score Test Score Test Score
1 X + X -/+ 2 X - X -/+ 3 X -/+ X + 4 X + X -/+ 5 X ++ X + Test = Test performed Table 6: Formative competence assessment survey (based on PPS, Straetmans, 2004)
7. 3. 2. PPS & 4C/ID IN COMBINATION WITH THE DCAF
The concept of an integral competence assessment via the PPS methodology may bring about
application possibilities for the 4C/ID model. When describing competence-based assessment,
we mentioned diagnostic tests to be implemented in supportive education. There are
workshops for exercising skills, which can be judged as formative capacity tests in a real life
environment. During their work on partial tasks, students may request professional formative
feedback from the lecturer who coaches him or from externals. The collection of relevant
formative evidence can be presented in a portfolio; subsequently, a summative assessment
with two assessors will then take place in order to evaluate the entire professional task. This
integral testing method may be linked to 4C/ID, the four components curriculum design.
(Sluijsmans, 2005)
When a thematic project is developed, directions per activity component should be taken up
as to which testing methods students should use to gather his evidence.
Dissertation for Master of Science - 81 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
Diagram 23: 4C/ID concept model in combination with the PPS for the DCAF
Table 5, “suitable testing methods”, presents possible utilisations of testing methods for the
DCA Frame in combination with the concept model 4C/ID and the assessment system of the
PPS the following assessment forms could be used.
FORMATIVE:
Supporting information Testing method
An underlying theory for carrying Diagnostic test
out tasks
JIT – Just-in-time information Testing method
Necessary information for carrying Test of conduct or capacity
out Standard professional actions
Partial task exercise Testing method
Instructive exercises for learning Hands-on, Professional feedback,
desired Professional skills Presentation of professional task
360 degrees feedback (co/peer/self)
Dissertation for Master of Science - 82 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
SUMMATIVE
The entirety of performed professional tasks and delivered products will be related to the
competencies to be assessed. The Assessment portfolio should be compiled along the lines of
the DCAF concept; it should contain a Defence referring to evidence. This evidence has been
collected, rated according to the methods of PPS, thus constituting a survey of formative
competence testing.
Following this system, a last, final evaluation (Du.: IBM) is established during an assessment
interview with assessors who will decide on the competencies acquired and rate them
according to the Dublin descriptors.
Dissertation for Master of Science - 83 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
Dissertation for Master of Science - 84 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
8. CONCLUSIONS AND RECOMMENDATIONS
Two concepts evolved from this master’s dissertation: the requirements for a competence-
based learning environment were founded on the Educational Analysis Model (EA Model),
and as a result the Digital Competence-based Assessment Frame was constructed in an
explorative way.
8. 1. CONSIDERING THE EA MODEL
The EA model proved an adequate instrument to describe an educational situation. The
decision to describe the environment in Higher Education from a perspective of the three EA
approaches resulted in a generic image of changes in society, and the consequences of those
changes for education.
• The approach according to the three levels, Macro-Meso-Micro generated a clear
framework to describe layers in the educational field. Due note should be taken of the
fact that Macro and Meso level regularly seemed to merge into each other. The School
of Higher Education as an institute, seen from the perspective of processes and
participants, was regularly placed on macro level; the academy/course on meso level,
and actual education on micro level.
• The changing roles of the participants became clearly discernable: students should be
more active; and from a variety of functions, lecturers should chose the one that suits
them best. The changes extend to supportive personnel; they, too are expected to
function in a different way.
• The digital environment is an increasingly essential provision in education. With
respect to this, attention should be paid to sufficient training for lecturers; only then a
paradigm shift in education can take place.
• The educational processes Coaching, Education and Assessment will come into their
own and are not necessarily interdependent. They are to be seen as three autonomous
processes; it is important to adequately organise transfer (between these processes).
• The innovated role of “Coaching” should be the foundation of the new way of
learning. Education will have to provide a broader and more integral support to meet
the demands of students. Competence acquisition requires a learning environment
which is as authentic as possible, in other words, resembles professional situations.
Dissertation for Master of Science - 85 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
This change affects competence assessment; it will have to focus on a student’s
personal development towards professionalism.
A separate study is recommended to investigate how processes and transfer between cubes
within the EA Model should be realised.
I quote from professor W. Jochems’s comments on the EA model:
“…An analytic model can be descriptive; in that case it represents the relevant
dimensions for describing features and characteristics of an issue…
…and it can be prescriptive; the model will then give indications and directions for
developing an issue…”
Considering Jochems’s statement, we conclude that the EA model served as a descriptive
model in this master’s dissertation
8. 2. POSSIBILITIES FOR THE DCAF
The concept of the Digital Competence-based Assessment Frame offers a structure to realise
the process Assessment in a more effective and efficient way. The requirements for a
functional design of the DCAF as a system logically connect to the described sequence
Coaching – Education – Assessment. The DCAF concept was placed in the learning cycle of
the Saxon PLW project and as such, served to describe how applications and database
systems in a learning environment should look like and how to employ them.
The following applications were described:
DPF Digital Portfolio
POP/ISP Personal Learning Process/Individual Study Programme
RAS Route Assessment System
The following database systems were described:
SIS Student Information System
OCS Course Competence System
DMS Document Management System
The DCAF concept may hold prospects for a promising fusion with the PPS methodology and
the 4C/ID model (Four-Component Instructional Design).
The Sakai project, an educational open source community, may offer possibilities to actualise
the DCAF Frame even further.
Dissertation for Master of Science - 86 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
Dissertation for Master of Science - 87 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
DEFINITIONS This master’s dissertation makes use of various terms and definitions which may be explained differently if considered in another context, or seen from another perspective. Below, we seek to describe the definitions that will provide the reader with an unequivocal understanding of the context they were used in. ACCREDITATION The term ‘accreditation’ stands for a hallmark, which proves certain requirements have been met. Since the introduction of the BaMa model all courses in Academic and Higher Education are expected to employ these hallmarks, which have been established by the NVAO for all Dutch and Flemish courses. ADAPTIVE LEARNING The term “Adaptive Learning” aims to purposefully create a learning environment in which students with different capacities can carry out their tasks and deliver products together FRAME The term ‘Frame’ is used to describe a part of the EA Model. Within the frame of this model, the mutual relations between the cubes are described. PARADIGM The term ‘paradigm’ has philosophical and linguistic implications. In a linguistic sense, we are dealing with declensions or conjugations. The meaning used in this dissertation departs from a philosophical point of view; ‘paradigm’ is to be seen as a coherent system. The EA model represents educational reality and will be used as a model for this dissertation. POP The abbreviation ‘POP’ stands for Personal Learning Process. ‘Learning’ here implies ‘student-centred’, and refers to the word ‘course’ as well. A more specific term is PAP – Personal Activity Process, which is not used in this dissertation. Considering it is related to the POP, we mention it under this heading. EDUCATIONAL LEARNING COURSE The entirety of activities developed by a student in competence-based learning constitutes his educational learning course, in which he attempts to acquire his competencies in every respect.
Dissertation for Master of Science - 88 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
AC - Assessment Centre ADC - Assessment & Development Centre BaMa - Bachelor Master model for Higher and Academic Education BI - Information Management CAF - Competence-based Assessment Frame CII - Communication, Information Technology and Information Management CMS - Content Management System CSCL - Computer Supported Collaborative Learning DAP - Digital Assessment Portfolio DCAF - Digital Competence-based Assessment Frame DCA Frame - idem DMS - Document Management System DPF - Digital Portfolio DU - Digital University EA Model - Educational Analysis Model ECTS - European Credit Transfer System EVC - Competencies elsewhere and previously acquired HBO-raad - Council of Higher Education Association HRM - Human Resource management IA - Information Architecture IBM - Integral Assessment IDM - Information Service- and Management ILA - Instruction – Learning – Assessment ISP - Individual Study Programme MKB-Ned - Retail Sector NVAO - Dutch-Flemish Accreditation Organisation OCS - Course Competence System OCW - Ministry of Education, Culture and Science OER - Examination Regulations in Education OWP - Educational Workshop PBL - Project-based Learning PLW - Personal Learning Course POP - Personal Learning Process PPS - Protocol Portfolio Scoring RAS - Route Assessment System SIS - Student Information System SLB - Career Coaching SLB-er - Career Coach VNO-NCW - Dutch Association of Employers –
Dutch Christian Association of Employers VSNU - Association of (Dutch) Universities 4C/ID - Four-Component Instructional Design
Dissertation for Master of Science - 89 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
DIAGRAM / TABLES Diagram 1: Educational Analysis Model – EA Model © De Vries, 2005 Diagram 2: Education triangle Diagram 3: Outline interviewees Diagram 4: Levels of EA Model Diagram 5: Cyclical education model Diagram 6: Place of the instruments (PLW-PLP) Diagram 7: Role from lecturer-guided to self-regulating learning Diagram 8: Participants of EA Model Diagram 9: Spiral model for reflection (Korthagen) Diagram 10: 4C/ID Model Diagram 11: Processes of EA Model Diagram 12: Learning cycle and lecturers’ roles (Saxion PLW Project) Diagram 13: Learning cycle and portfolio (Saxion PLW Project) Diagram 14: e-Coaching from four perspectives (Smits e. a. 2006) Diagram 15: SALDO Themes for collabortive learning Diagram 16: Model owner and responsibility Diagram 17: Stages of assessment (BI & IDM) Diagram 18: Assessment Frame - EA Model Diagram 19: Concept DCAF-system Diagram 20: Components for DCAF Diagram 21: Five levels of competence Diagram 22: Authenticity performance pyramid according to Miller (1990) Diagram 23: 4C/ID concept model in combination with the PPS for the DCAF Table 1: Changed insights in education (Dochy & Bouwens, 1992; Kayzel, 2004) Table 2: Features of innovation in society, Hoeksema (2002) with own addition Table 3: Points of attention for an electronic learning environment Table 4: Characteristics for Assessment (Segers e.o. , 2003) Table 5: Suitable testing methods Table 6: Formative competence assessment survey (based on PPS, Straetmans, 2004)
Dissertation for Master of Science - 90 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
REFERENCES Baartman, L.K.J., Bastiaens, T.J. en Kirschner, P.A., (2004); Requirements for
Competency assessment Programmes, Paper at the Onderwijs Research Dagen 2004, Utrecht / OU NL, Heerlen.
Birenman, M., (2003); New insights into learning and teaching and the implications
for assessment in Optimising new modes of assessment: in search of qualities and standards, edited by Segers, M., Dochy F.J.R.C., & Cascallar, Kluwer Academic Publishers, Dordrecht/Boston/London.
Blok, H., (2004), Adaptief onderwijs; Betekenis en effectiviteit, Pedagogische
Studieën, jaargang 81, nr. 1. , p. 5-27 Boekaerts, M. en Simons, P.R., (1993). ; Leren en instructie. Psychologie van de
leerling en het leerproces; Assen: Dekker & van de Vegt. Bolognaverklaring, (1999); De Europese Onderwijsruimte, Gezamenlijke verklaring
van de Europese ministers van Onderwijs, EU, Bologna/Brussel. Boshuizen, H. en Kirschner, P., (2005); Coaching en training in geïntegreerde
elektronische leeromgevingen. In: Een geïntegreerde benadering van e-learning (H12), onder redactie van Jochems, W.M.G., Van Merriënboer J.J.G., Koper, E.J.R., Bastiaens Th.J., Wolters-Noordhoff Groningen/Houten.
Bouman, A. en Lisdonk, E. van, (2005); Naar een digitaal portfolio, Een onderzoek
naar de aansluiting van beleid met betrekking tot het digitaal portfolio op ervaringen van studenten in het hoge onderwijs, DU Digitale Universiteit, Utrecht.
Bruijns, V., Eringa, K., Rietveld J. en Zwaal, W., (1999); Het gebruik van assessment
en development centers in een competentiegerichte leeromgeving. In: Competentiegerichte leeromgevingen (H16) onder redactie van Schlusmans K. , Slotman R. , Nagtegaal C. , en Kinkhorst G., deel 2, Lemma, Utrecht.
Castells, M., (2001); The Internet Galaxy. Reflections on the Internet, Business and
Society, Oxford University Press Inc., New York Cohen, L., Manion L. en Morrison, K., (2000); Research methods in education, 5th
edition, RoutledgeFalmer, London. Dochy, F.J.R.C., Nickmans, G., (2005); Competentiegericht opleiden en toestsen:
theorie en praktijk van flexibel leren; Lemma BV, Utrecht. Dochy, F.J.R.C., Bouwens, M.R.J., (1992); Toetsing en Toetsservicesystemen voor
individuele leertrajecten in open leersituaties, Open leren Inforeeks, Spruyt, Van Mantgem & De Does BV, Leiden.
Dissertation for Master of Science - 91 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
Draaijer, S. e.a.,(2004); Flexibilisering van Toetsing, Creëren van voorwaarden voor een breed toetsrepertoire in het Hoger Onderwijs, DU Digitale Universiteit, Utrecht.
Driessen, E, Beijaard, D., Tartwijk, J. van., Vleuten C. van der, (2002); Portfolio’s,
Wolters-Noordhoff Groningen/Houten. ELF, DEST, JISC-CETIS (2004); e-Learning Framework, Service-Orientend
Frameworks: Modelling the infrastructure for the next generation of e-Learning Systems; A paper prepared on behalf of DEST (Australia), JISK-CETIS (UK), and Industry Canada.
Everwijn, S.E.M., (1999); Het hoe, wat en waarom van competentiegericht
onderwijs. In: Competentiegerichte leeromgevingen (H4) onder redactie van Schlusmans K., Slotman R., Nagtegaal C., en Kinkhorst G., deel 2, Lemma, Utrecht.
Geerligs, T. en Veen, T. van der, (2001); Lesgeven en zelfstandig leren, Koninklijke
Van Gorcum BV, Assen (9de druk) Geloven, M. van, Smulders. P, en Vries, M. de, (2006); Meerwaarde van e-learning
in de praktijk, Student, docent en manager aan het woord, DU-HO Praktijk, Wolters Noordhoff, Groningen/Houten.
Hendriks, P., (2006); Handboek Assessment, Gedragsproeven, deel 1, onder redactie
van Schoonman, W. ; Koninklijke Van Gorcum BV, Assen. Hoeksema, K. & Oosterhout K. van, (2002); Competenties voor de informatie-
samenleving, stakeholder centraal, In Informatief, nov.’02, SDU, Den Haag. Jager, H. de, Mok, A.L. en Sipkema, G., (2004); Grondbeginselen der sociologie,
Wolters Noordhoff, Groningen/Houten (12de druk). Janssen-Noordman, A.M.B. , Merriënboer, J.J.G van, (2002). Innovatief Onderwijs
Ontwerpen: Via Leertaken naar Complexe Vaardigheden, Wolters Noordhoff, Groningen.
Jochems, W. M. G. (1999); Competentiegericht onderwijs in een elektronische
leeromgeving, 15de Dies Rede, OU Open Universiteit Nederland, Heerlen. Jochems, W.M.G. en Schlusmans K., (1999); Competentiegericht onderwijs in een
elektronische leeromgeving. In: Competentiegerichte leeromgevingen (H3) onder redactie van Schlusmans K., Slotman R., Nagtegaal C., en Kinkhorst G.; deel 2, Lemma, Utrecht.
Kayzel, R., (2004); De belofte van het nieuwe leren. Een evaluatie van het
constructivisme, Hogeschool van Amsterdam, Amsterdam.
Dissertation for Master of Science - 92 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
Kirschner, P.A., Martens, R.L., & Strijbos, J.W., (2004); CSCL in Higher Education?
In: What we know about CSCL, And it implementing in Higher Education; redactie: Jan-Wilem Strijbos, Paul A. Kirschner en Rob L. Martens, OU NL, Kluwer academic publishers, Boston, Dordrecht, New York, London.
Koper, E.J.R., (2005); Een domeinmodel voor e-learning In: Een geintegreerde
benadering van e-learning (H5) onder redactie van Jochems, W.M.G., Van Merriënboer J.J.G., Koper, E.J.R., Bastiaens Th.J., Wolters-Noordhoff Groningen/Houten.
Korthagen, F. & Vasalos, A., (2001); Maatwerk bij coaching. Handboek Effectief
Opleiden, NVP; Elsevier HRM, Den Haag. Knottnerus, A. en Renique, (2004); Domeincompetenties in het HBO als sleutel voor
herkenbaarheid en flexibiliteit, in TH&MA5, Elsevier Overheid, Den Haag. Maastricht Communiqué, (2004); Over de toekomstige prioriteiten voor intensievere
Europese samenwerking in het beroepsonderwijs, EU Maastricht/Brussel. Merriënboer, J.J.G van (2005); Didactisch ontwerp voor e-learning In: Een geintegreerde
benadering van e-learning (H1) onder redactie van Jochems, W.M.G. , Van Merriënboer J.J.G., Koper, E.J.R., Bastiaens Th.J., Wolters-Noordhoff Groningen/Houten.
Oblinger, D. en Oblinger, J., (2005); Is it age or IT: First steps towards understanding
the Net Generation, Chapter two of Educating the Net Generation, Educause, Orlando, USA
Onderwijsraad, (2002); Competenties: van complicaties tot compromis, Over schuifjes
en begrenzers, onder redactie van: Van Merriënboer J. J. G. , Van der Klink M. R. en Hendriks M., OTEC, Open Universiteit Nederland, Onderwijsraad, Den Haag.
Preece, J. a. o. (2002); Interaction Design. Beyond human-computer interaction, John
Wiley & Sons, Inc. New York. Rubens, W., (2003); Samenwerkend leren met behulp van ICT binnen het Europese
onderwijs: ervaringen met Synergeia in vier landen, ILVOS Universiteit Utrecht. SALDO projectgroep, (2004); Samenwerkend Leren Digitaal Ondersteund, Handboek,
DU Digitale Universiteit, Utrecht. Salmon, G., (2004); E-moderating: The Key to Teaching and Learning online,
Routledge Falmer, London/New York. Sanden, J.M.M. van der (2001); Opleiden vanuit een constructivistisch perspectief. In: Human
Resource Development: organiseren van het leren Kessels, J.W.M. & Poell, R.F. (Red), Uitgeverij Samson.
Dissertation for Master of Science - 93 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
Saunders, M.N.K., Lewis, P., Thornhill, A., (2004); Methoden en Technieken van onderzoek, derde editie, Pearson Education Benelux, Amsterdam.
Schreuder Peters, R.P.I.J., (2002); Methoden & Technieken van Onderzoek, Principes
en Praktijk, 2de druk, Academic Service, Schoonhoven. Segers, M., Dochy F.J.R.C., & Cascallar (2003); The Era of Assessment Engineering
In: Optimising new modes of assessment: search of qualities and standards, edited by Segers, M., Dochy F.J.R.C., & Cascallar; Kluwer Academic Publishers, Dordrecht/Boston/London.
Simons, P.R.J., (2002); Digitale Didactiek: Hoe (kunnen) academici leren ICT te
gebruiken in hun onderwijs, Universiteit van Utrecht. Shuel T. J. (1988); The role of the student in learning from instruction. Contemporary
Educatonal Psychology, 13; Elsevier, Internet tijdschriften. Sluismans, D. en Martens, R., (2005); Performance assessment in e-learning In: Een
geintegreerde benadering van e-learning (H5) onder redactie van Jochems, W.M.G., Van Merriënboer J.J.G., Koper, E.J.R., Bastiaens Th.J., Wolters-Noordhoff Groningen/Houten.
Smits, M.H.S.B., Stijnen, P.J.J., Bie M.D. de, Bastiaens, Th.J., (2006); De E3
Begeleider - Effectief, Enthousiast, Elektronisch, Wolters-Noordhoff Groningen/Houten.
Straetmans, G.J.J.M., (2004); Protocol Portfolio Scoring, Een methode voor het
systematische scoren en vaststellen van competenties, BVE en HO Brochurereeks Perspectief op Assessment 4, Cito-groep, Arnhem.
Tartwijk, J. van, Driessen, E., Hoeberigs, Kõsters, J., Ritzen, M., Stokking, K.,
Vleuten C. van der, (2003); Werken met een elektronisch portfolio, Wolters-Noordhoff Groningen/Houten.
Van Dale Lexicografie, (2004), Woordenboeken; Utrecht/Antwerpen Veldhuis-Diermanse, A. E. (2002); CSCLearning? Participation, Learning activities
and knowledge construction in Computer-Supported Collaborative Learning in Higher Education, Proefschrift Universiteit van Wageningen, Veenendaal.
Dissertation for Master of Science - 94 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
INTERNET RESOURCES Bologna-verklaring, (1999); De Europese Hoger Onderwijsruimte, De Europese Unie
(EU), Brussel: UK: http: //www. europa. eu/pol/educ/index_en. htm NL: http: //www. europa. eu/pol/educ/index_nl. htm ELF, e-Learning Framework (2004); Modelling the infrastructure for the next
generation of e-Learning Systems; UK/JISC-CETIS: e-Framework: http: //www. e-framework. org/resources/ HBO-Raad, Raad vereniging van Hogescholen: http: //www. hbo. nl/
navigatie onderdeel > Domeincompetenties NVAO, Nederlands-Vlaamse Accreditatieorganisatie, Nederland en Vlaams België,
Den Haag: NL/Belgie: http: //nvao. net/ / http: //nvao. be/ OCW-HBO (2005), Presentatieagenda 2005 OCW – HBO-raad, Min. OCW
Den Haag: http: //www. minocw. nl/ho Sakai (2005-06), Synchronised Architecting of Knowledge Aquisition Infrastructue:
SURF Special Intrestgroup: http: //e-learning. surf. nl/docs/e-learning/sakai Edusite/Educause 2005: http: //www. edusite. nl/edusite/specials/15311 Sakai organisatie: http: //www. sakaiproject. org
Saxion Hogescholen, Deventer/Enschede Algemeen: http: //www. saxion. nl Academie CII locatie Deventer: http: //cii. saxion. nl SAXION UNIVERSITIES FOR APPLEID SCEINCE - DOCUMENTS Saxion Onderwijsvisie, (2000); Onderwijs op maat in een activerend klimaat,
Onderwijs visie van de Saxion Hogescholen, Enschede. Project PLW, (2006); PLW Integrale Rapportage Instrumenten & PLW
Werkdocument 2004-2009, Stuurgroep PLW, Saxion Hogescholen, Enschede. Schoonman, W., (2004); Assessment voor en door iederéén, Lectorale rede, Saxion
Hogescholen, Enschede. Straetmans, G.J.J.M., (2006); Bekwaam beoordelen en beslissen, beoordelen in
competentiegerichte beroepsopleidingen, Lectorale rede, Saxion Hogescholen, Enschede.
HBO-V, (2005); Handleiding reflectie in de verpleegkundige beroepsuitoefening
tijdens het duale leertraject; Academie AGZ, Deventer/Enschede.
Dissertation for Master of Science - 95 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities
Digital Competence-based Assessment Frame
Toetsen, (2004); Toetsen en beoordelen in een competentiegerichte leeromgeving, samensteller T. Geudeke, afdeling Onderwijs en Kwaliteit, Saxion Hogescholen Apeldoorn/Deventer/Enschede.
SAXION SCHOOL OF CII DOCUMENTS Kleinpaste, J. , Overbeek L. , Balsma, G. , (2001); Naar een nieuw curriculum met
het BAKLOV-model, Saxion Instituut I&I, Deventer. SLB Handboek, (2005/06); Studieloopbaanbegeleiding in het competentie onderwijs
van BI & IDM, red. T. de Bruyn, M. de Vos en H. de Vries, Academie CII Deventer.
MSC ELMAC STUDYDOCUMENTS / CASE STUDY Vries, J. de, (2003, Dec.); DAP for CII Deventer, Plan for testing the use of the
Digital Assessment Portfolio (DAP), Deventer/Nijmegen, Communication Consultancy and Change management.
Vries, J. de, (2004, May); Research Proposal & Pilot Research DPF with the
Projective Picture as Instrument, Deventer/Nijmegen, Research Methodology. Vries, J. de, (2005, June); DAP for CII Deventer, Report of testing the use of the
Digital Assessment Portfolio (DAP), Deventer/Nijmegen, Project Studies.
Dissertation for Master of Science - 96 - Hans de Vries E-Learning, Multimedia and Consultancy School of CII Sheffield Hallam University | HAN University for Professional Education Saxion Universities