Alternative assessment in teacher education

Dysthe, O. and K. S. Engelsen (2004) Portfolios and assessment in teacher education in Norway: a theory-

based discussion of different models in two sites. I Assessment & Evaluation in Higher Education, Apr

2004, Vol. 29 Issue 2, p239,

E-assessment 2013 1

Introduction • “The Quality Reform of Higher Education” 2001 • Portfolios were specifically mentioned • The project: “Alternative assessment in teacher education” • Involves three Norwegian teacher institutions:

– The Department of Teacher Education at the University of Oslo – Departments of Teacher Education at the University Colleges of Vestfold – Departments of Teacher Education at the University Colleges of

Stord/Haugesund • The project is based on social constructivist and sociocultural

perspectives • The focus in this article is on describing and discussing the portfolio

models and typical learning and assessment processes in the two sites, with a particular emphasis on the learning potential and areas of improvement.

E-assessment 2013 2

Main questions

• What are similarities and differences in the portfolio models that are implemented by our two sites?

• What are particular areas for further improvement?

• What are critical factors in portfolio work as regards students' learning processes?

• How is the digital aspect of portfolios utilized and what are unused potentials?

E-assessment 2013 3

Definition of portfolio

“A portfolio is a purposeful collection of student work that exhibits the student' efforts, progress, or achievements in one or more areas. The collection must include student participation in selecting contents, the criteria for judging merit, and evidence of the student`s self-reflection.” (p. 60) (Paulson, F. L., Paulson, P. R. & .“Meyer, C. ( 1991) What makes a port folio a portfolio? Educational Leadership. 48(5), 6O - 63.)

E-assessment 2013 4

A model of analysis for portfolio processes

• The model presented in Figure 1 is based on a collection-reflection-selection framework

• Want to raise awareness of advantages and disadvantages of different ways of practicing portfolios

• Primarily a descriptive and analytic tool • Purpose is to illustrate how the learning potential at different

stages in the portfolio processes depends on what choices are made regarding some key factors in each of the 'learning phases‘

• Particularly interested in how the portfolio can support student learning.

• Meant to help identify critical factors for improvement, especially in institutions where they have just started using portfolios

E-assessment 2013 5

E-assessment 2013 6

Figure 1. A model of analysis for portfolio processes in the project «Alternative assessment in teachers education in Norway»

Learning phase 1 Questions: • Organization

– How are learning processes organized? – What combinations?

• Individual-collective – How is the relationship between individual and collective/collaborative work? – What kinds of work dominate?

• ICT – Is ICT used primarily for individual writing and for organization of portfolio processes, or also

for feedback, discussion and collaboration? • Writing and feedback practices

– How are the recommended writing strategies? – Who gives feedback and how is it organized?

• Meta-processes – To what extent are reflection as well as discussion and negotiation of criteria integrated in the

work with theoretical and practical aspects of the subject area?

Teacher and peer feedback are widely recognized as crucial aspects of formative assessment.

E-assessment 2013 7

Learning phase 2 Questions: • Self-assessment

– Are students involved in self-assessment as part of selecting what goes into the PP, and do they use criteria and reflection in doing so?

• Criteria – Have students been given explicit criteria for what counts as good

quality work? – Have these been developed in cooperation with the students or

negotiated with them? • Reflection

– Are students asked to write reflective text s for their PP? Several studies have documented that self-assessment of portfolio work is an important aspect of formative assessment as well as students' need for criteria

E-assessment 2013 8

Learning phase 3 Some critical factors: • Object and format of assessment

– What is being assessed? – The portfolio itself? – Portfolio-based oral performance or written essays? – Combination of portfolio and traditional exam?

• Student involvement – Is the student directly involved? – What degree of control does the student have over the object of assessment?

• Criteria – Are the criteria implicit or explicit? – Do the criteria include process or just product?

• Assessor – Is the traditional Norwegian pattern of external assessors + teacher still dominant? – Are peer and self-assessment used for summative purpose?

• Reflection – Is there any space for setting new learning goals on the basis of assessment results?

E-assessment 2013 9

Portfolios in the light of sociocultural theories of knowledge and learning

From sociocultural perspectives knowledge and learning are viewed as situated, social, distributed and mediated, dependent on language and dependent on participation in communities of practice. • Portfolios as mediating artefacts

– Portfolios are mediating cultural tools, and the physical and cultural aspects of them are important, as well as the rules and routines and processes the portfolios are surrounded by and embedded in.

– Digitalization of portfolios provides an interesting example of how learning processes change when the mediating tool changes.

• Learning as situated, social and distributed – When knowledge development and knowledge production is in focus instead of knowledge reproduction,

the notion that knowledge is distributed becomes of practical importance in the learning activities – From a sociocultural perspective language is not just a mediating tool for learning, but closely bound up with

thinking itself – Collaborative learning processes are dependent on shared oral and written information and on dialogic

interactions at all stages of knowledge production. • Learning as participation in communities of practice

– In order to define a teacher education class or a peer group as 'a community of practice', the students must share a mutual engagement in learning, be willing to define the particular learning task as a joint enterprise and share 'repertoires', for instance strategies for group work, for how to give response or for how to create hypertextual links.

– An important question to ask is therefore whether portfolio assignments pre-supposes students' participation in and across different communities of practice or only individual engagement with the tasks.

– One implication is the close connection between process and product. – In portfolio artefacts the process is implicit in the product.

E-assessment 2013 10

Some general findings • Portfolios influence the pedagogical processes in both sites and have improved formative

assessment. • There is considerable emphasis on social learning processes in both sites. • Unless cooperation and collaboration are built into the portfolio assignments, individual work

continues to be dominant. Assignments are therefore crucial. • Developing an integrated portfolio and ICT pedagogy is a major change and takes tune. • Written Net-based dialogues complement oral dialogues both among students and between

students and teachers. • Increased workload is a problem both for teacher and students, especially the first semester

due to the demand of developing the specific competencies in using the mediating tools • None of the sires utilize fully the learning potential in all the three phases of the model of

analysis. Some improvement areas are: – (a) criteria work in phase l and 2; – (b) student written reflective texts; and – (c) self-assessment and selection.

• The summative forms of assessment can be characterized as hybrid in the sense that traditional exams are mixed with portfolios in various ways. The most successful innovation in summative assessment is a portfolio-based oral exam.

E-assessment 2013 11

How do the sites utilize the learning potential in each of the phases 1?

• Learning phase 1: Formative assessment – The use of portfolios and ICT has resulted in changes in the way

student learning is organized. – Efforts to change from lecture-based teaching to project- and

problem-based learning. – Portfolios afford more systematic use of peer feedback, mainly

individual, but also in the form of response groups. – Developing and negotiating quality criteria for student work

may be a necessary next step. – There is a need for developing expertise on collective portfolios. – All sites are still mainly teacher and curriculum controlled and

that moving the locus of control to students is a hard and difficult process.

E-assessment 2013 12

How do the sites utilize the learning potential in each of the phases 2?

• Learning phase 2: Formative assessment. – Phase 2 is weakly represented in the project. – Neither of the sites have a strong focus on selection,

criteria work and self-assessment. – There is much evidence in the international portfolio

literature that the selection process promotes self-assessment and focuses students' attention on quality criteria.

– There is a considerable improvement potential in our two sites in the area of self-assessment.

– The portfolios are not assessed directly but indirectly in a portfolio-based oral or written exam.

E-assessment 2013 13

How do the sites utilize the learning potential in each of the phases 3?

• Learning phase 3: Summative assessment – There are three main models of assessment where portfolios are involved:

• the portfolio itself is being graded • the portfolio is the source from which students extract material to present digitally at an oral

exam • written exams, where students more or less directly develop some aspects of portfolio contents

– None of the disciplines in the study base students' grades solely on the portfolio.

– Neither is self- or peer assessment involved in the summative assessment process.

– The assessment commission, consisting of external assessor and the teacher, determines the final grade.

– Seems to be a discrepancy between the importance of the portfolio in learning phase l and the lack of importance given to it in the summative assessment.

– A positive aspect according to the students is the new oral, portfolio-based exam, where students are given 10 -15 minutes for a digital presentation of selected portfolio work, followed by questions and discussion.

E-assessment 2013 14

Discussion of three critical aspects of portfolio work 1

• Negotiated portfolio assignments – Portfolio assignments must provide opportunities

for rich and complex learning situations – Assignments should signal the need for collaboration – What kind of practice do we want students to

document in the portfolio? – Problems need to be complex even if reification of

individual cognitive activity is the goal – Assignments need to be carefully designed – Such assignments need to be discussed and

negotiated with students

E-assessment 2013 15

Discussion of three critical aspects of portfolio work 2

• Reflection/self-assessment – It is important to build reflection and self-assessment into the

portfolio assignment itself – These features of portfolio work are elements of students' identity

building – Reflection and self-assessment strengthen these identity-forming

processes – It is vital that they are part of the portfolio work and also documented

in the form of text – Quality criteria are important for students in order to self-assess – These criteria need to be negotiated in order for students to make

them their own – The learning potential in reification and self-assessment for lie in

sharing them with trusted peers and teachers instead of in grading such texts

E-assessment 2013 16

Discussion of three critical aspects of portfolio work 3

• The format of summative assessment – They argue for the students' participation in the

summative assessment process – There is no reification without participation – Assessment formats, which allow students'

participation and open up for negotiation of meaning, can bring out more of the 'iceberg' of the process

E-assessment 2013 17

Concluding remarks • The core issue in the assessment discussion seems to be the

relation between formative and summative assessment. • The model of analysis emphasizes collaborative and meta-reflective

aspects, criteria development and self- and peer assessment • A stronger focus on these aspects involves a shift of focus from

summative to formative assessment which seems fundamental for increasing the learning potential of portfolios

• Changing the assessment system has more to do with learning in general than with assessment as an isolated phenomenon

• lf portfolio assessment is introduced into a culture dominated by a traditional view of knowledge and learning, the outcome may either be an instrumentalization of portfolios or a change in the culture itself.

E-assessment 2013 18

The paper's contribution to the field of e-assessment

You can read my text here

E-assessment 2013 19

Reflections on how to apply the idea of this paper in a relevant educational

context

You can see my reflection on this Evernoteside Flesberg 19.03.13 Jan-Arve Overland

E-assessment 2013 20