Acknowledgements

Promoting a deep approach to professional learning in the field

Development and evaluation

of electronic personalised learning

G Finnerty, M A Volante, L Rockingham,

J MacLaren and M ODriscoll

Centre for Research in Nursing and Midwifery Education

Division of Health and Social Care

Faculty of Health and Medical Sciences

June 2008

Acknowledgements

We would like to thank:

All the students who participated in the project from Cohorts 1 and 2 from the Mentorship module.

The four students who gave of their time to speak to us about their mentor experience.

Colleagues in the Mentorship teaching team.

Colleagues from the University of Surrey e-learning units and in particular Dr Steve Rowett whose resourcefulness made the project possible.

Members of the Steering Group.

Dr Andrew Comrie and Professor Pam Smith for advice, guidance and support throughout the project.

This project was funded by the University of Surrey Fund for the Strategic Development of Learning and Teaching.

CONTENTS

51.0Summary

62.0Introduction

73.0Background

84.0Module design, development and delivery for personalised learning plans...

125.0 Literature Review

186.0 Project approach and evaluation

227.0 Findings

428.0Discussion

459.0Conclusions and recommendations

48APPENDIX ONE

Glossary of terms

E-tivities

Electronic activities

ICT

Technology

KSF

Knowledge and Skills Framework

PDA

Personal Digital Assistant

SPT

Specialist Practice Teacher

VLE

Virtual Learning Environment

1.0Summary

Aim of project

To develop an e-portfolio to manage student progress and assist with authentic assessment on the Mentorship module (FHMS).

Objectives

To evaluate student experience of peer-supported reflective learning using electronic portfolios

To increase staff understanding and potential of e-portfolios as a way to engage students in meaningful learning.

Evaluation Methods

Nominal group technique and e-learning questionnaire involving e-voting by students

Semi-structured interviews with module tutors in FHMS and staff from e-learning unit

Biographical narrative interviews with a sample of students following successful completion of the Mentorship Module.

Key Findings

Mentor students managed their learning and progress through use of the Virtual Learning Environment (U-Learn). Support was presented through multiple strands, for example, handbooks, lectures, peer engagement, a help desk, IT services and the E-Learning Unit.

Mentor students actively engaged in the reflection process to inform their learning and, importantly, their teaching and assessment.

Tutors felt they had ownership of the module, which they described as organic and living.

Learners described a deeper understanding of teaching styles, theories and new ways of performing in challenging mentoring roles. The blended learning approach was generally viewed as a positive innovation to inform mentoring as an art and a lifelong skill.

Personalised learning within dynamic e-portfolio is a product of the learners own history, their future aspirations and how they engage with the design of learning activities, including the use of e-technologies.

A route map which addresses learning strategies and content is part of the scaffolding necessary for learners to organise information into meaningful learning outcomes. This is demonstrated by the building of an e-portfolio of personalised learning products.

Capturing student biographies is an effective approach to illuminate understanding the modification of personal learning schemata.

2.0Introduction

This report presents the outcomes of an education development and evaluation of technology supported personalised learning undertaken during 2005 2007. The work was funded by the University of Surrey Fund for the Strategic Development of Learning and Teaching. The aim of the project was to: develop an e-portfolio to manage student progress and assist with authentic assessment. The project used the continuing professional development Mentorship module (30 credits level 3) to pilot the development. The specific module context provided the project with a secondary aim of seeking to provide an effective and efficient learning experience that will enhance the supervision of the student experience in practice settings recognising the wider system in which mentoring and personalised learning plans are located.

Specifically the project had the following objectives

use e-portfolios to further develop the student experience of peer-supported reflective learning

support learner progress and assessment through the development of an e-portfolio as a tool for capturing evidence of learning outcomes and for ongoing professional development

enhance the supervision experience of mentor students and their learners on professional placements

increase staff understanding and capacity of e-portfolios as a way to engage students in a meaningful learning experience.

The aims and objectives were addressed using an action research design in which two members of the module teaching team worked with an education developer, and researcher to enquire into technology-supported personalised learning. The two teachers worked closely with an e-learning technologist and the other members of the module teaching team to design the planned e-tivities for students to build their e-portfolio. The action research design intended to facilitate staff engagement with the pedagogical practices necessary for the design and support of student centred e-learning. A programme of staff development for teacher moderated on-line discussion was put in place. A literature review on e-portfolios was undertaken and the e-tivities supporting personalised learning plans were evaluated using questionnaires and semi-structured interviews with staff. To understand the learner mentor experience within the wider context of the professional learning in the field, narrative interviews were undertaken after the completion of the module. This report summarises the outcomes of the development and evaluation as a basis for conversation and discussions with colleagues on e-developments for professional learning.

Section 3 provides contextual background to professional learning in the field.

Section 4 outlines the design, development and delivery of the e-learning education interventions to support personalised learning and the building of an e-portfolio.

Section 5 presents a review of selected relevant literature.

Section 6 descibes the project approach and methods for evaluation.

Section 7 presents findings using three perspectives. Firstly, the student perspective with findings from the standard module evaluation questionnaire, the e-learning evaluation tool and classroom field observations. Secondly, the teaching staff perspective with findings from the semi-structured interview and finally presenting the findings from learner mentor biographies using narrative interview.

Section 8 draws together the key issues emerging from the evaluation of electronic personalised learning plans.

Section 9 provides some conclusions and offers recommendations for action for consideration by relevant teaching and learning communities of the University.

3.0Background

The project used the mentor preparation module as a case for enquiry into the development of e-learning by means of an electronic portfolio that supports learners to manage their progress and assist with meaningful assessment processes. A technological advantage of e-portfolios over traditional portfolios is that they are sharable, which facilitates interaction between student and student as well as teacher and student (Cotterill et al. 2004). The flexibility of the presentation methods and the embedding of interactive elements in the building of e-portfolios have been found to deepen learning for trainee teachers and to also reduce workload pressure in comparison with producing traditional portfolios (Woodward and Nanlohy 2002). Peer sharing and interaction was made possible through the discussion tool of ULearn.

Whilst it was intended to supply the students with their own on-line learning space to capture meaningful learning and build an e-portfolio, early in the project it became clear that this was not going to be feasible with the information technology support available. Instead, following discussion with the e-learning unit and Dr Rowett (in particular), the functions of the universitys virtual learning environment (ULearn Web CT Vista) were used to enable students to experience the use of technology to support personal learning intentions and to show the personal process of enquiry undertaken within the overall learning strategies of the module, for example, reflective essay incorporating feedback from e-mediated peer discussion. The decision was taken during the delivery of the module that the students would be asked to submit a conventional paper based portfolio. Whilst some of the students may have wished at the end of the module to have submitted their portfolio of learning electronically, consideration also needed to be given to where the teachers had reached in their understanding of e-learning as well as organisation processes for example, the examination office staff.

Undergraduate teaching and learning in the Division of Health and Social Care is delivered through programmes designed around classroom based learning in the university and the development of clinical competence through practice learning and reflection with mentors (workplace supervisors) in the local health economy. Methods of assessment are based on engaging students in meaningful learning activities, the products of which form a portfolio of evidence of professional knowledge, understanding including learning how to learn, and clinical competence. The portfolio is used to both track student progress and to assess student achievement of practice learning outcomes at the appropriate academic level. Mentors are a crucial resource in the assessment of practice learning outcomes and clinical competence and are required by the professional body of the Nursing and Midwifery Council (NMC), to undertake mentor preparation through University accredited and NMC approved courses (NMC 2006).

The Division of Health and Social Care is contracted for continuing professional development by the local Strategic Health Authority to provide mentor preparation (amongst other provision). The module is also used by other healthcare employers. Over one academic year the module is taken by approximately 400 learners. During the year 2006/07 the module ran 11 times. This project planned to use data from the first and second student cohorts of 2006/07 and the lessons learned through ongoing evaluation by the teaching team of the following cohorts has informed module design and delivery within the academic year.

4.0 Module design, development and delivery for personalised learning plans

In this section the e-learning educational intervention is presented and outlines the curriculum design and development of the module; the student constituency; the university teachers and staff development for e-learning, and the specific e-tivities supporting the building of a portfolio of meaningful learning experiences.

4.1 Module development

Blended campus and distance learning (15 credits at HE2/3)

Assessed by profile of reflective writing and mentor assessed learning development plan (competency document)

Each cohort was run by a different pair from a group of 6 module tutors

4.2 The student constituency

The majority of students who undertake the module are registered practitioners working in health services for example, nurses, midwives, operating department practitioners and ambulance service personnel. All are adults who are working and taking the module for professional development either in relation to their own personal aspirations or because of employer requirement. The range of ICT skills within health care is well known (Laurillard 2005) and a typical early posting on ULearn during the project reflects student difficulty:

I am struggling with this way of learning, not sure its for me? Its really quite stressful .

4.3 The staff

The Mentorship module is resourced by a core group of six teachers who have a particular interest in learning and teaching within practice settings and have been supporting the module over various lengths of time. The teachers had been involved in the working group undertaking the curriculum review and redesign of the module to meet the NMC standards for supporting learning and assessing in practice settings (NMC 2006).

The teachers were afforded a range of staff development in relation to ULearn and teacher supported peer mediated on-line group discussion. All university teachers are required to undertake the introduction to ULearn provided by the University e-learning unit in order to become a ULearn user. The project made funding specifically available for the six teachers to undertake the Oxford Brookes course peer learning on line. Four teachers took up the opportunity. Two members of the teaching team who supported cohorts 1 and 2 were key participants in the action research design for implementation of the e-portfolio development project. They were supported by an e-learning technologist from the university e-learning unit. The table overleaf summarises the structure and content of the e-tivities undertaken by the mentor students during Units 1 and 2 of the Mentorship Module

Table 1: Summary of e-tivities within overall module

Unit of learning

Coursework to be produced

E-tivities

Other Activities/ comments

Students to undertake pre-course checklist to ensure they have requisite knowledge and skills to be able to participate in the online learning environment

ULearn log-in session

Hopes, fears and expectations discussion thread

Introduction to

module caf for social interaction and discussions

sources of help including the Help-at-hand area

Discussion of library resources and online access of books, journals etc.

Threads throughout module

1

Learning and Teaching in Practice Level 2 & 3

For this assignment you should develop a teaching plan that you will use with your learners. You should submit this plan with a 500 word reflective critique which explores the effectiveness of your plan in the light of relevant theoretical literature, and personal and professional experience. Within this unit you should include a discussion of how you used learning theories and domains of learning to support the development of your teaching plan.

Discussion thread: Priorities of workload and the learning environment (level specific - students access the discussion at the level they are studying)

Uploading of draft assignments and receipt of feedback

Assignment Draft uploaded for review by personal tutor by end of day three of module

1 and 2

Collaborative online small group working to produce a seminar on their chosen topic. This combines enquiry based learning and group ULearn discussions for classroom presentation

Group specific discussions with allocated online tutor

Student led seminar presentations on day four

2

Assessing and Evaluating Learning

Level 2

Prepare a 500 word reflective discussion which identifies two types of assessment used to assess learners in your clinical practice. You should compare and contrast these methods / strategies in relation to their strengths and weaknesses, paying particular attention to the concepts of reliability and validity.

Level 3

You should prepare a 500 word reflective critical analysis of the concepts of reliability and validity in assessment in relation to your assessment of a learner in your practice area. You should draw upon a range of resources, including personal and professional experience.

Level specific discussions based around individual learning journeys (My learning journey) in preparation for part 3 reflection my learning journey.

face-to-face student feedback and evaluation of seminar presentations

A written reflection:

Unit Three: My Learning Journey

Level 2 & 3

This final piece of work for your profile is a 1500 word reflection on your development as a mentor. The title of this is Reflecting on my journey from the beginning of the course how do I now feel about the mentorship module and my practice as a mentor? You should draw upon your experiences of undertaking the module, in attending, studying and mentoring learners in your practice area. You should demonstrate evidence of appropriate reading around the subject of mentorship by referring to the relevant theories you have learnt about.

Your work should have a clear reflective structure, and clearly demonstrate description, analysis, synthesis, evaluation and self-awareness, and give an idea of how you feel you may further develop your mentorship whilst fulfilling all the learning outcomes.

My Learning journey and group discussions remain open

To be submitted 1 month following completion of the module with unit 1 & 2 reflections and learning and development plan completed in practice with mentor (paper copy)

To be submitted (paper copy), 1 month following completion of the module with unit 1, 2 & 3 reflections

Learning and development plan

Competency assessment in practice Four proficiency statements must all be signed off and include comments on your practice at both the formative and summative stages.

5.0 Literature Review

5.1 Introduction

The purpose of this review of literature is to highlight the key elements which have informed the development of electronic resources within the Mentorship module in the Division of Health and Social Care. Evaluation of these new developments has entailed reviewing and analysing electronic portfolios used for medical training and teacher training. The literature review is organised by firstly stating the purpose of e-portfolios and defining terms. How e-portfolios are structured will be examined before discussing problems which have been reported through evaluations in Australia, the United States of America (US) and the United Kingdom (UK). This review then incorporates evaluations which include views of students, teachers and educators who have used these electronic records over the last decade. The intention is to complement other work in progress involving e-portfolios and electronic tools for recording learning across the University.

5.2 Purpose of e-portfolios

The primary purpose of the e-portfolio is provision of a pedagogical tool for recording and managing learning. Garrett and Jackson (2006) suggest the electronic portfolio should be used as a tool to contextualise and embed knowledge in the workplace but Hennessy and Howes (2004) assert that, although useful as a collection of evidence of learning, the e-portfolio should not be merely a transcription tool or archive (Herbert, 1998). It would appear that e-portfolios are increasingly being used for job applications and promotions and also as a tool to show development of thinking (Butler, 2006). As Cambridge (2004) asserts, the electronic portfolio should represent what a person (or organisation) has learned over time through a collection of authentic and diverse evidence. Across the literature, it would appear that what has the most meaningful pedagogical influence is how learners are enabled to analyse their own evidence and take responsibility for their learning.

The e-portfolio has been shown to be a valuable tool to empower learners through use of self-evaluation and reflection, assisting with planning of work-based learning (Graham 2004). This is of particular value to mentors as a tool to assist with planning supervision and assessment of students in diverse workplace settings. Cotterill et al (2004) also emphasise the positive value of transferability of data to support life-long learning. It is this capacity for recording personal and professional growth which makes the electronic portfolio a versatile instrument in current fast-changing health environments. The social networks made possible through use of e-portfolios also has potential to promote lifelong learning (Cambridge, 2004). Additionally, the e-portfolio can assist learners to see patterns in their work over time and to process feedback from tutors and peers. Importantly, e-portfolios provide learners with the opportunity to see their progression and to visualise how all the learning and activities link together (Williams 2000).

Georgi and Crowe (1998) cite Gardner (1987) who suggested the portfolio provided:

Naturalistic sources of information about how peoples around the world develop skills important to their way of life (p189). Hamilton (2004) terms these skills transcendent as they emerge through the e-portfolio building process and can encompass such things as heightened critical thinking and understanding diverse societies and cultures. Cambridge (2004) claims that the test of this durable learning is that it can last beyond a module or programme.

The e-portfolio is now frequently used as a tool to showcase examples of best work, for example through video-clips or podcasts. The increasingly interactive elements provided by creative hardware, software and institutional VLEs (Sharpe et al 2004) mean that active learning is promoted through providing learners with a platform to dialogue with peers and tutors. This provides a medium to engage students in meaningful learning experiences. This is particularly relevant with a new emphasis on the student voice (Hughes, 2006), which has only recently received attention (Kilpatrick 2007).

Discussions abound as to the extent to which e-portfolios can be used for performance-based assessment (Georgi and Crowe, 1998). This important issue will be covered in more detail later in the review.

5.3 Structure of e-portfolios

Individual, faculty and institutional models are used across the United States (Cambridge 2000). Throughout the literature, decisions regarding constructing and implementing an individual e-portfolio should include:

Agreeing the purpose of the electronic portfolio, i.e. for learning, assessment or professional development

Prescriptiveness of the guidelines for the creation of an e-portfolio

How pieces of selected information should be organised, for example, around themes selected by the student or lecturers?

Input of tutors, lecturers and peers, including feedback mechanisms.

Learners are being encouraged to record their experiences using a variety of media, including mobile clinical e-portfolios, using wireless Personal Digital Assistants (Garrett and Jackson, 2006). As asserted by Laurillard (2007) mobile technologies such as pod-casting also enable continuity between contexts and provide opportunities for social construction of knowledge (Grob 2008). There is still a need, however, for provision of tangible flags or sign posts for students. Laurillard (2005) emphasises how essential explicit learner goals are at the outset of an e-portfolio. This is because if learners focus on the operational aspects of a task, rather than its meaning or content, surface learning only is likely to ensue.

It is the learners choice of modes and materials selected which makes the e-portfolio such a learner-centred activity (Mason et al, 2004). The key to presentation of a successful e-portfolio is not the technological facilities used but the demonstration of reflective learning using a digital interface:

The most effective electronic portfolios link student reflective practice with products and performances, which indicate hands-on and applied acquisition of practical skills and knowledge involving learning technologies (Jonassen 2001, p 7).

Laurillard (2007) has analysed various modes of mobile technology used by students and teachers within a Conversational Framework. Her research has demonstrated how the richness of mobile learning can be enhanced and digitally facilitated due to immediacy of actually being in the practice environment. Certainly, Pebble Pad enables inclusion of multiple social networks for learning and recording practice under one umbrella (Grob 2008, personal communication).

5.4 Medical portfolios

In the UK, e-portfolios are structured in a variety of ways, depending on the users and audiences. They have most commonly been used in medicine. These are often structured as an evaluation tool to assess performance in authentic contexts. As Butler (2006) suggests, this is imperative to record continuing learning and to keep pace with rapid advances and developments in health. Driessen et al (2005) assert that medical portfolios are unique because the focus of reflection is on actions. Problems arise, however, because the portfolio in this format is often presented merely as a dossier of evidence. Butler (2006) suggests that a misunderstanding of portfolio pedagogy is apparent across the medical literature. Haig (2007, personal communication) disputes this, suggesting that General Practitioner trainees in Scotland report that competencies are linked with the national curriculum and lecture time is subsequently reduced. Implementing a tool which records student progression across professions has been suggested (Cotterill et al 2004). A generic e-portfolio system is in use in Newcastle meaning that collaborative connections can be made, linking with other web-based electronic systems. Cotterill et al (2004) highlight the necessity for e-portfolios to be flexible and easy to customise.

5.5 Nursing portfolios

Having examined a selection of web e-portfolios used in medicine in 2005 (Cotterill et al 2004), it could be said that structures are traditionally rigid. In contrast, nursing portfolios have been criticised for lacking any unified understanding in the literature (Spere and El-Ansari, 2004). This is, however, improving with increased use of digital formats for reflection on practice. The Royal College of Nursing is the largest European professional body and has recently been implementing an e-portfolio for supporting nurses continuing professional development and re-accreditation (Cambridge 2004).

5.6 Benefits of e-portfolios

Help to focus student thinking, and translate theory into practice (Butler 2006)

Aid management of work for engineers through use of a personal website (Campbell and Schimdt, 2005)

An instrument to enhance employability (Cambridge 2004)

Can help art and design students form their artistic identity

Can help individuals to manage and distribute personal digital information and therefore contribute to knowledge management (Cambridge, 2004)

Provide a tool for asynchronous discussion within a large group

Can help to support diverse students and enhance undergraduate learning (Cambridge 2004)

Have potential to improve patient care (Butler, 2006)

Provide opportunities to network and communicate both synchronously and asynchronously (Greenberg 2004)

5.7 Problems with e-portfolios

Portfolio construction is notoriously time-consuming and reliant on staff support (Butler 2006). Extra time therefore needs to be set aside for preparation to use an electronic interface (Scholes et al. 2004). Additionally, the technological novelty or troubleshooting may overshadow the purpose of the portfolio (Woodward and Nanlohy, 2002). As Laurillard (2005) asserts, although the aim is to promote discussion using communicative media, any pedagogical benefits depend on a healthy dialogue between student and tutor or student and student. Gomez (2004) found that being able to track and guide student learning remotely was beneficial for students on long placements. Conversely, students compiling electronic personal development profiles at Bournemouth University felt there was a Big Brother effect which affected authenticity (pdp.bournemouth.ac.uk).

Ownership of e-portfolio data has emerged as a contentious issue. As Kilpatrick (2007) asserts, there is currently confusion as to an institutions right to retain data and responsibilities of an institution to store information about students in this format. Woodward and Nanlohy (2002) argue that ownership needs to be clearly established by all authors using electronic and digital tools.

Problems also occur with assessing portfolios. The audience needs to be clearly established and defined (Herbert, 1998). Cayne (1997) described problems with lack of guidelines for assessment. For this reason, Webb et al. (2003) stress that marking and grading criteria need to be extremely clear and question whether portfolios are, in fact, valid forms of learning and competence. Problems with assessing professionalism in medicine have also been documented (Butler 2006). A supportive educational environment is therefore essential so that students feel comfortable revealing any weaknesses or mistakes (Pinsky and Fryer-Edwards 2004).

5.8 Views of students

Darling (2001) found that students described feeling confused and anxious if guidelines for portfolio-building were not well defined. Students appeared to benefit if examples of past portfolios were shared and if the scope, nature and value of the development portfolios was made explicit. Resentment by students was common if a balance did not exist between an emphasis on construction which was student-driven and over-prescribed guidelines. Students in Darlings (2001) study also expressed concern with the subjectivity of evaluation of their portfolios.

Conversely, a sample of pharmacy students (n=154) in Manchester reported that the portfolio was an effective tool to summatively assess work, to support their learning and also prepared them for their future continuing professional development (CPD) (Ashcroft and Hall, 2006). E-portfolios have also been reported to be valued by learners for accessing of information from a central place (Hennessy and Howes, 2004).

Pebble Pad was used by students undertaking a Post-Compulsory PGCE course at Wolverhampton University, to record their progress (Kilpatrick 2007). A significant outcome noted by several students in video transcripts, was the longevity of the tool for blogging within a personalised learning environment. Daily entries to the e-portfolio were being made more than nine weeks after completion of the course. This demonstrates the power of virtual communities to sustain lifelong learning.

5.9.1 Views of trainee teachers

Trainee teachers in one study described how the flexibility of presentation methods and the interactive nature of building an e-portfolio, helped deepen learning and reduce workload pressure (Woodward and Nanlohy, 2002).

Attractiveness for students of graphics, movies, sound and other computer applications help to build a running multi-media record of a students growth and development found by student teachers (Bartlett 2000 p 296).

Ford and Olhausen (1991) surveyed the attitudes, beliefs and habits of 115 teacher education students in the US. Just 1% reported that their beliefs about assessment had changed and the assessment of their own portfolios played a critical role.

As part of teacher education, a reflective portfolio is often used to help embed theoretical principles, skills and knowledge in the classroom. Interestingly, there appears to be a dearth of research with a focus on e-portfolio implementation for trainee mentors or trainee nursing and midwifery teachers. With new Nursing and Midwifery Council standards in place (NMC 2006), nurses and midwives with a mentoring role will have to demonstrate that they have attained certain standards and criteria to sign off students practice as professionally competent. It is possible that the e-portfolio will provide a central medium to display and validate this evidence.

5.9.2 Views of teachers/educator in the literature

E-portfolios have been reported to provide an immediate way to assess learning by teachers (McKenzie et al. 2002). In one study, focus groups were held with medical educators and questionnaires were issued (Hennessy and Howes, 2004). Although staff initially found the e-portfolio web interface difficult to use, some described how overall computer skills improved and benefited as a result of working with the e-portfolio.

Designing Constructivist Learning Environments (CLEs) through portfolio building

Jonassen (2001) asks; how will we know if students are engaging in meaningful learning? Technologies need to be introduced which enhance, not dilute meaningfulness in learning. To help guide learners to construct their learning, a rubric can be introduced. This is a tool or model presented to learners at the beginning of an assignment which:

Aids the learner in refining the product.to include the desired content aspects, while still maintaining the freedom to develop a product-oriented assignment which reflects the learners personality, style and knowledge (Crawford 2002: 5).

There are a variety of rubrics to suit a range of educational purposes, for example, for holistic critical thinking scoring and as authentic assessment tools. Crawford (2002) suggests that a rubric can function as an evaluative tool, guiding a learners expectations for an assignment. Jonassen (2001) emphasises the need to assure that students learning is:

Active ; Constructive; Collaborative; Intentional; Complex; Contextual; Conversational and Reflective.

The following diagrammatic representation of a rubric shows how the attributes and characteristics of meaningful learning are interconnected. These characteristics can be used to aid discussion in the classroom (Jonassen 2001) or used as criteria for assessment to identify students progress, understanding, knowledge and skills (Williams 2000).

Figure 1

Constructive

Collaborative

Active/manipulative

Intentional

Complex

Contextualised

Reflective

Conversational

Constructive

Collaborative

Active/manipulative

Intentional

Complex

Contextualised

Reflective

Conversational

This flexibility and use of a constructivist paradigm obviously offers potential as a focal tool to underpin e-portfolio development. Jonassens selected characteristics are so relevant to personalised learning that this rubric will be used as a framework to underpin analysis of interview data from the mentor students. (Please see section 7.3, p 34).

Summary

This short review of the ever-expanding literature on electronic portfolios has covered the key pieces of work which have informed the development of research tools to help present a robust evaluation of implementation of an e-portfolio in the Mentorship module. Although problems with e-portfolios have been identified (for example, issues of ownership of student material) the literature demonstrates that benefits to learners, teachers and institutions may be profound. Key advantages cited are pedagogical benefits of mobile and electronic technologies such as immediate and site-specific communication, enabling live channels of conversation and new styles of reflection between students and their peers and even tutors. The Wolverhampton Pebble Pad project showed the potential for on-going and meaningful peer learning. It is evident that signposts and maps for students need to be clear in any constructivist learning environment. The central aim of the current study is to develop an electronic portfolio to manage student progress and assist with authentic assessment. The literature therefore supports the rationale for the project and highlights the need for teachers to engage with students using alternative platforms (such as an electronic portfolio) to create a meaningful, personalised learning experience. The study design has been partially derived from the literature and how the project objectives will be met will be described in the following section.

6.0 Project approach and evaluation

The normal module classroom evaluation process was used to evaluate the implementation of the e-tivities and teaching and assessment strategies supporting learners on the mentorship module to build a personalised learning plan and produce an e-portfolio. The standard module evaluation form was used along with the university e-learning unit feedback questionnaire. These two tools have consistently been used for all of the 11 cohorts undertaking the module, and provide teachers with data as a basis to review their actions in an action research process of teacher enquiry. The evaluation format was extended to incorporate qualitative in-depth feedback on the production and development of an e-portfolio within the professional life course of the mentorship module student. Teaching staff were also invited to participate. The intention of the evaluation is to illuminate how the supported student enquiry process used by the student builds a personalised e-portfolio for assessment of the module outcomes. The approach draws on that of illuminative evaluation to compare planned activity with student perceptions and experience of building an e-portfolio.

The main measures to identify project outcomes are:

Collation of student views of the experience of the process of building an e-portfolio to manage their progress

The level of engagement, formative input and levels of support in the process of reflective practice learning

The value of electronic portfolios for assisting with managing the clinical supervision of pre-registration students and also to manage continuing professional development

Teachers understanding of e-portfolios as a means to provide evidence of students learning and as a tool to enhance meaningful learning

Sample

The sample for the project and its evaluation consisted of:

The first two cohorts from a total of 11 cohorts undertaking the mentorship preparation module during the academic year 2006/07 (n= 62)

Teaching staff of the mentor preparation module team and the e-learning unit (n=7).

Ethical considerations

Professor Desombre (Chair of University Ethics Committee) was consulted in October 2006 regarding status of this evaluation for ethical approval. It was agreed that, because the participants had informed consent, including the project teams intent to publish, a University ethics opinion was not necessary.

All participants, students and staff were provided with written information sheets describing the project evaluation. These were issued by the evaluator during Day Five (Evaluation Day) for both Cohort One and Cohort Two in November, 2006. In March 2007 the evaluator collected the module results list from the module administrator, which was confirmed by the examinations office. All students who had passed the module were sent a formal letter inviting them to participate in an in-depth biographical narrative interview. Students were approached to participate in the interview after the module assessment results were in the public domain to avoid compromising student achievement of learning outcomes during module delivery. The decision was made that only those who passed the module were invited to participate so as not to intrude on retrieval processes.

Data sources and analysis

Three sources of data were used for the evaluation to understand how students were supported and experienced building an e-portfolio and their experience of the mentorship module within their professional life course. Also staff views of participating in the delivery of the mentorship module and educational e-tivities interventions were captured. The data sources were as follows:

1 Classroom evaluation: Standard module evaluation form, e-learning unit survey and classroom evaluation observation

2 Semi-structured interview with mentor preparation module tutors and e-learning unit staff

3 Biographical narrative interview

These are now presented in turn.

1 Classroom evaluation

Standard module evaluation form

All students completed the self-report standard module evaluation form which uses a 3-point satisfaction scale of very satisfied, satisfied and unsatisfied. Specific questions on ICT and module teaching and assessing strategies were selected for analysis using frequency counts. Responses were analysed in greater depth using the responses to the e-learning survey.

E-learning survey

An e-learning questionnaire was developed by the e-learning unit to evaluate student perceptions of using an electronic platform ULearn. Responses were obtained from cohorts 1 and 2 using an electronic voting system capturing individual feedback via handsets. This was carried out in the classroom as a group activity (see photograph)and therefore only aggregate frequency count data was available for each cohort.

Figure 2

Photo courtesy of SR from e-learning unit on use of electronic voting system to capture responses to e-learning questionnaire.

All students engaged in the exercise and, when giving informal feedback about the method, the majority stated that they particularly liked the immediacy and visual impact of the graphic scoring. In presenting the data analysis Cohorts 1 and 2 are amalgamated except where large differences were found across Cohorts.

Observation

The observations recorded by the evaluator took place during the classroom evaluation session following a nominal group technique exercise. The evaluator was present during the nominal group activity for both cohorts and captured some feedback from the student discussion of Cohort 1 and the closing circle discussion of Cohort 2. The data in field note form of what was said during these closing discussions is complementary to the quantitative data sets of the standard module evaluation form and the e-learning questionnaire. The data were used to illuminate the quantitative findings.

2 Semi-structured interviews with tutors and personnel in the e-learning unit

During the lifetime of this project, the Mentorship module teaching team comprised seven teachers of whom five agreed to participate in the interview. The purpose was to ascertain views of setting up a new style of module and to also provide a space for them to share how they visualised the in-depth component of the evaluation taking place. The five teachers from the mentorship module teaching team were interviewed at the university using a semi-structured interview schedule. Each interview lasted approximately thirty minutes. The same questions were posed to all five respondents. One member of the e-learning unit was interviewed and provided detail about the setting up of electronic portfolios on the University VLE. The interviews were analysed using thematic content analysis (Patton 1990).

3 Biographical narrative interview (BNI)

Four students responded to the letter of invitation with an information sheet to participate in the interviews following feedback on their assessment results. The interviews were planned to take place after the completion of the module and submission of the portfolio of evidence for assessment to avoid compromising student achievement of learning outcomes during module delivery. All students were from Cohort One and had the permission of their line manager. The interviews were carried out by GF at the working sites of three out of the four respondents. This assisted in gaining contextual information about the workplace teaching and learning environments described by the students. Each interview lasted one hour and participants were given a high street store voucher for 25.

To gain a complete biographical narrative, it is important to have a defined beginning, then sequence of events, then an ending (Wengraf and Chamberlayne, 2006). The BNI method necessitates asking firstly a single question to induce narratives or stories from the respondent. The question asked of the four participants was designed to capture their story, beginning from a time that was significant for them:

We are interested in practitioners experiences in becoming a mentor. Please tell me about your experiences of the mentorship module from when it became important in your life and up to now. Please take all the time you need. Ill listen first and take some notes as you are speaking and ask questions when you have finished talking.

The BNI method is structured around two sessions (Wengraf 2007). The first session involves free speaking of the participant, which is uninterrupted. The interviewee is encouraged to speak from their frame of reference rather than the interviewers. During the interview interaction, emphasis is on active listening by the interviewer so that cues can be recorded, in preparation for the second session. This is centred on capturing particular incident narratives (PINs) considered to be the nearest to the actual experience when it occurred. Questions are asked in the same order as presented by the interviewee in their response to the first single question. The interview was closed by the interviewer saying, Thats all my questions. Thanks very much. Is there anything else you would like to tell me before we end the session? PAUSE. What else are you doing for the rest of the day?

The interviews were transcribed, read and analysed using thematic content analysis (Field and Morse 1985, Patton,1990). The underpinning framework used to codify the emerging themes was Jonassens rubric (2001) as presented on page 17. These themes were validated independently by the project manager and then confirmed with other members of the project team.

7.0 Findings

Results from the group evaluation are presented using descriptive statistics. Cohort One evaluated the module separately from Cohort Two.

7.1 Student perspective

This section provides results of the standard module evaluation and e-learning questionnaire and qualitative comments from observation of the evaluation process.

7.1.1 Standard student evaluation form

A large amount of data were available from the standard module evaluation data. Taking up the advice from Cousin (2007) to think with the data, eight statements pertaining to assessment, student motivation and ICT were selected from the standard student evaluation. The results are presented in Figure 7.1 Levels of satisfaction with how the module supported learning.

With the exception of responses to the statement there was clarity of information about assessment criteria for Cohort 1, in general the students were satisfied with the module. Of the seven statements, the response of two thirds of cohorts 1 and 2 were satisfied to where it was used IT helped me learn, is below that of the percentage satisfaction level for the other statements. For each of the statements Cohort 2 is considerably more satisfied because the proportion very satisfied is much higher for cohort 2 (note this difference is not illustrated in Figure 7.1).

The difference in satisfaction levels between Cohorts 1 and 2 is both large and consistent across all the statements. We went back to the teachers to work with them and reflect on the data to consider why such a consistently large difference might have occurred (Cousin 2007). In the conversations, what emerged was that during the week between C1 and C2 starting, the teachers met and reflected on their teaching experience with C1 and made planned changes to the teaching approaches for implementation with C 2 including how ULearn was introduced. The alterations in the teaching and learning strategies in the delivery for C 2 are presented in table 7.1.

Table 2 Alterations to teaching and learning strategies for C2

Day 1: 15 minute presentation about ULearn prior to logging on session

utlised buddy approach to classroom activities

Library session compulsory

Hopes and fears ice-breaker sheets collated and posted on ULearn

Status passage reading taken home

Day 2: ULearn refresher session held at 4pm and not lunchtime

Day 5: Posted hopes and fears revisited within evaluation

The design of the evaluation is not robust enough to make the claim that such changes explain the differences in cohort satisfaction levels but they can suggest explanations. It is possible that the methods and techniques used to support explanation for understanding from the student perspective enabled C 2 students to engage in meaningful learning and a process of meta-cognition which connected and extended their current schemata of understanding (Eraut 2000). The interpretation has some support from the field observational feedback data of C 2.

One student said: Professionally as well as personally, I think Ive developed

Another stated that the module had been empowering

There were varied cohort responses to the statement; there was clarity of information about assessment criteria. 48% of C1 was satisfied compared to 100% of C2.

The findings from the e-learning survey are used to explore in more depth the response to a) where it was used IT helped me learn of the standard module evaluation form and b) the student experience of e-tivities as part of a managed learning process for the purpose of supporting the students approach to meta-cognition.

Figure 3 Levels of satisfaction with how module supported learning (selected standard module evaluation form questions).

82

79

63

48

88

90

86

67

71

85

96

67

71

100

91

95

100

100

100

98

81

0102030405060708090100

learning helped develop analytical and problem solving skills

module stimulated enthusiasm for further learning

module has improved confidence in tackling unfamiliar problems

where it was used I.T. helped me to learn

the number & nature of assessments (with submission dates) was made

clear to me

the assessment/s related to the module outcomes

there was clarity of information about assessment criteria

all

c2

c1

7.1.2 E-learning questionnaire

ULearn usage

The e-learning questionnaire was administered at the end of the module, after the students had been working with ULearn for 11 weeks. At the end of the module, 29% of C1 and C2 respondents rated their expertise with using computers in their studies as high or very high; just under half (45%) describe their expertise as OK. Confidence in C2 appeared to be higher than in C1; 20% of C2 rated their expertise very high compared to 4% of C1; and 11% of C1 rate their expertise as very low compared to 0% of C2 (see table 7.2).

Figure 4 Expertise in using computers in your studies

11%

22%

44%

19%

4%

0%

20%

45%

15%

20%

6%

21%

45%

17%

12%

0%5%10%15%20%25%30%35%40%45%50%

Very low

Low

OK

High

Very high

average c1+c2

C2

C1

The learner constituency of the mentorship module live off campus and are full-time working professionals who are released by their employers to attend the university on a part-time study basis. Students had internet access through a range of computers, but work and home were the most predominant location for accessing ULearn. Both C1 (83%) and C2 (85%) experienced technical problems using ULearn and the open feedback responses on how to improve the students experience indicates technical resolution is likely to make a difference to student engagement with e-learning.

Address ULearn technical issues regarding downloading information and parking assignments

Stop it crashing / freezing every 5 minutes

The difference in cohort experience emerges in identifying and getting help to deal with the technical problems. Across the questions concerning help and support C2 consistently indicates a greater level of negative perceptions with the help, support and guidance offered compared to C1. Overall C1 found ULearn easier to use than C2.

C1 showed greater frequency of use of ULearn with 61% of C1 using ULearn once a week or more compared to 53% of C2. Although C1 are logging on more often, a much higher proportion of C2 are using ULearn for 5+ hours per week during the module (21% compared to 8% in C1). C2 are having fewer but longer sessions on ULearn. The open questions on the E-learning questionnaire shows that the use of ULearn does realise some benefits.

` "Made me feel more comfortable using a PC than I was previously

Challenged me to learn something new

The self managed learner

Response data from four questions from the e-learning questionnaire were used as an indicator of student management of their learning. The questions and responses are presented in the following table:

Table 3 Self managed learning.

Attitudes to statement

'Using Ulearn helped me manage my studies for this module better'

7

15

33

33

11

10

35

25

25

5

05101520253035404550

strongly agree

agree

not sure

disagree

strongly disagree

Cohort 2

Cohort 1

Although C2 appear to hold more negative perceptions of technical support for ULearn, they hold more positive perceptions of ULearn than C1 in helping them manage their studies and in particular a perception that they would like to use ULearn for all their modules. It is possible that with C2 having a greater level of expertise in using ULearn for their studies than C1, C2 has more awareness of how ULearn can support their learning intentions. That said, both cohorts are similar in distribution of their perceptions that using ULearn has helped them understand the module. So it is not simply a matter of understanding the module content, there also appears to be a student orientation to e-learning technology emerging. From the field observation of developmental negative feedback data, C 1 predominantly identify technical and student feedback issues, even although they perceived they got the help and guidance they needed with technical difficulties more than C2.

Cohort one developmental negative comments

Need a prerequisite level of computer skills prior to coming on the course

Felt unsupported if not computer literate, when accessing remotely

Can feel discouraged due to emphasis on IT

Teething problems with U-Learn eg parking the assignments

Problems with downloading because home computer didnt have Java

Hidden costs of going to an internet caf and printing

More time needed to prepare for the presentations

Assignment guidelines need to be given earlier eg Day 2

Difficulty accessing a traditional tutorial

Prefer individual to generic feedback

Although C1 held more positive perceptions of gaining help and guidance when needed than C2, lack of preparedness for ULearn seemed to predominate in their perceptions of control of their learning. This may also explain why C2 were more highly satisfied in the standard module evaluation form data given the different approach to preparing the cohorts for ULearn and C1 and C2 perceptions on clarity of the information on assessment criteria. From the observational data both cohorts were of the view that Everything could/can be submitted on U-Learn including being able to submit final work online.

E-mediated discussion

In response to the question I would be able to study equally well on this module if I had never logged into ULearn 65% of C2 agreed compared to 34% of C1. In similar vein only 25% of C2 disagreed with the statement compared to 41% of C1. The responses to the question I felt comfortable using the ULearn discussions to learn on this module showed 58% of C1 agreed compared to 42% of C2. The data illuminate differences in how cohorts felt about studying using ULearn discussions. More marked is the difference in perception between C1 and C2 on the extent to which course materials that were available on ULearn helped them learn on the module, 79% of C1 agreed with the statement compared to 55% of C2. It is possible that the lack of help and guidance when needed by C2 is linked with their perceptions of the course materials including e-discussions. What is consistent across cohorts is the level of agreement on the lecturer feedback. Whilst discussion may have a place what makes the difference is the feedback from the teacher and teacher support in the discussions. The following quote is from a students open response e-learning questionnaire:

Discussion forum was great. Enjoyed linking with peers and tutors

It is not e-learning technology per se that facilitates a self managed learner, rather how the overall learning experience is designed, constructed and experienced and e-learning located within that. The positive comments and also possibly the negative comments from the field observation data are supportive of such an interpretation table 7.3.

Table 4 Comments from observation of the module evaluation

Cohort 1 positive comments

Cohort 2 positive comments

We had enough support from the personal tutors-with a prompt response

U-Learn was very good

Help was available for assignments

Useful to split module into three sections

Enjoyed group support

Balance of days was good-liked the pattern

Structure of the course was good

Relevance of teaching sessions to written work

Cohort 1 negative comments

Cohort 2 negative comments

It was slow torture to hear flexibility of writing assignments

Its exhaustingI had to go on the computer at midnight when sharing access with children in the family

The observational data below from the closing circle of Cohort 2 shows the context to which e-learning on the module both contributes and creates.

Empowering

Relevant

Really friendly

Useful to students

It changed my way of thinking towards students. I now look at everything in a different light

It made me more aware of mentorship-and the whole journey of learning

It made me reflect more on my practice and be a good role modelI didnt realise how important it was until this course

In summary, the findings from these data sources demonstrate how essential student support was, in terms of how the learning experience is constructed and timely access to teacher and technical input. The design of the learning activities helped them manage their learning rather than necessarily ULearn per se. E-learning is an element that offered a possibility of peer supported learning when not at the University. Whilst the cohorts experience is variable, what is generalisable is the power of carefully designed and executed learning activities for meaningful student learning.

7.2 Teaching staff perspective

A convenience sample of five (out of seven) tutors was invited to participate in a semi-structured interview. Questions centred on adaptation of the individual tutors to delivery of the mentor module using a personalised e-portfolio approach.

It is significant that some tutors were themselves using an electronic interface and U-Learn for the first time in their teaching practice. This created some anxiety as the tutors were expected to move away from the traditional structure of paper portfolios. Four themes were generated from the thematic content analysis of the six interviews. Embracing the virtual culture; Developing digital communities of learning; The organic nature of e-portfolio building; and Being an invisible presence. These are presented using interview extracts to reveal some of the realities of promoting a reflective platform for students using electronic activities blended with face to face, traditional learning and teaching approaches.

(i) Embracing the virtual culture

The following quote emerged in response to question three, regarding tutors feelings about which evaluation strategies would be most appropriate to capture students views:

The nominal group technique is the first opportunity to be open and honest about engaging with the technology (there were people who didnt log on at all)-and I need to be one step ahead of the students! (MPT2)

Another tutor visualised embracing the virtual culture for both mentor preparation and ongoing mentor development:

We need to change the mentor update culture, therefore we need to know about e-portfolios to make a web-page for them. Once weve broken the habit well be able to put on mentor development days around a wider scope. For example, we could put activities on the web and look at inter-professional learning. They could attend a day then mentors could write a reflection. This could have potential for the KSF (Knowledge and Skills Framework) (MTD)

However, one tutor raised concerns about embracing the virtual culture, questioning the student experience:

The new style is a good idea but what value do the students place on it? And are the students ready for it? (MPT3)

There appears to be some hesitancy concerning whether the virtual environment was, in fact, embracing students as equal stakeholders.

Having designed activities and tutored these students on-line, I feel weve got more tools in the toolbox to deal with more complex issues (MPT2)

To prepare them for supporting the students and teaching using electronic methods, four tutors in the sample had completed, or were in the process of completing an e-tutoring course. These tutors provided rich information related to how the format of the course changed their views and approach to working within a virtual culture:

The [e-tutoring] course was very interestingIts all virtual, on-line. All the emails and posts were openIt helped me build my portfolioI understand what the students are going through to prepare them-and how working with a group on-line is managed. You only have a week to do a presentation and youre only given a title. As for whos going to do what etc, its all on-line (MPT1)

In the above quote, the respondent articulates the benefits of openness of email postings to inform her own electronic portfolio. Barriers appeared to occur through having no face to face contact with team members prior to engaging in group work using team activities. Another tutor who had undertaken the e-tutoring course actually indicated that on-line ice breakers were enough to enable contribution to on-line group discussions:

Ice-breakers in class and on-line gave time to get to know the groupIt helped with summarising posts. I can go on-line and pick up the discussion threadsI learnt for myself (MPT2)

The following quote highlights the necessity for clarity of learning outcomes if presented on-line:

When we set up, weve got to be mindful of embracing everybodys learning styles and learning needs. Also, [the e-tutoring course] wasnt very organised because of lack of on-line outcomes. There were too many links so we wasted loads of time. Id have liked them to be more prescriptive in week one (MPT4)

The same tutor described the disembodied nature of communicating on-line without having an indication of who they were communicating with, or even which gender!

I hated the course. I got so frustrated!...I didnt know who I was talking to. With any e-learning they have to see who theyre talking toI was talking to disembodied names. Some of the posts were too long and were just tirades. Some were posing (MPT4)

The issue of feeling that the person on-line was posing and using long posts and messaging seemed to mitigate against meaningful dialogues occurring. The use of the word tirade indicates that the messaging may have been antagonistic as well as lengthy.

(ii) Developing digital communities of learning

The tutors sometimes revealed how inspired they were by students digital capabilities:

Its much more about lifelong learning rather than the single end of the courseThese students will be much more digitally adept than ussome have their own blogs and are even developing their own communities of practice (MPT2)

It was recognised by one tutor, however, that accessibility was not always universal:

..Blogs could be used but there are access issues (MPT3)

Having an individual space was seen as progressive, in combination with on-line discussion groups:

its about making a contribution to the body of knowledge and producing something to say: This is who I am. Its also about the student being more collaborative on-line (MPT2)

Some concern was expressed regarding anonymity and being faceless. It is interesting that the following quote is from the tutor who spoke of frustration at communicating with disembodied names. This had evidently influenced her teaching style:

With team teaching its anonymous. Things will change now and we can nurture the cohort. Ive got a good knowledge of the cohort nowThey know who theyre talking to and are therefore more ready to share (MPT4)

The presenting of group feedback by tutors had initially been set up as group presentations had been given. However, this was not always individual enough:

We give group feedback but students find it difficult to decide whats pertinent to them. Individual feedback is probably better (MPT3)

The use of an electronic approach to encouraging student reflection within a digital community was seen by one tutor to have added potential to widen participation of students:

This has been a learning curve for me. Ill have to be quite pro-active. But its something I believe in. One day we may see students by video link only. I am all for embracing the concept. It allows for widening participation (MPT4)

(iii) The organic nature of e-portfolio building

An important element in supporting students for the mentoring role involved encouraging students to engage in live discussions and reflections on teaching and assessing in practice. The following quote provides an excellent vision from a tutor of the movable and organic nature of learning using the electronic medium:

I dont see the portfolio as static but a living document. An experience at work builds into the portfolio, eg using an image, a picture or a photo, or a flow diagram depicting a train of thought to express the journeyI dont want it to be linear, but movable, like a jigsaw, with an elusive piece at the end; a growing thing, like a plant. You can prune it. You can choose whether you water it or not. Learning expands and contracts (MPT4)

Several tutors intimated that the virtual environment should enable self-expression of the students:

A sustainable evaluation needs to have a focus on the self. You need to capture the feelings and barriers (MPD)

The inclusion of both professional and personal aspects was viewed as positive:

I[Portfolio-building] means creating an evidence base of what youve done and how youve done it-for academic life or personal life-or both (MPT3)

Importantly, the development of the portfolio using U-Learn was seen as a useful tool by one tutor, to enable learners to visibly see their learning journey (Williams 2000):

The e-portfolio is an authentic record of their learning, so they can see their journey-and a way of connecting experiences (MPT2)

(iv) Being an invisible presence

One tutor alluded to the invisible work of checking students on-line engagement and contributions, or lurking as it is known in the virtual world:

There needs to be a culture change in thinking about contact hours. Your teaching, not administration will need to be identified. For example, care is needed with lurking. And tutors may look in on groups theyre not allocated to (MPT1)

The quote highlights the possible tension with guiding students without spying or observing interactions without the students explicit knowledge of this. There was also subtle concern that the electronic format provided access to the whole teaching team.

Interview with E-Learning Unit stakeholder:

The respondent from the E-Learning Unit at the University was asked primarily about e-learning initiatives across the University, about how students were assisted to manage their learning, and also how it was envisaged that tutors should be prepared to teach using the VLE.

(i) Preparing the tutors

The respondent provided useful insights into the benefits of tutors undertaking an e-tutoring course prior to teaching using a blended approach:

One of the most powerful things is if tutors have studied on line themselves first. Its powerful to understand what its like to be a student in the on-line environment. They can have a shift in their head from understanding the issues. It also increases confidence. (ELU1)

This shift in the tutors heads described by the E-Learning respondent is congruent with the paradigm shift described by some of the tutors themselves.

(ii) Capturing individual student experiences

The respondent suggested that the most effective method for capturing experiences of e-learners was through use of qualitative methods and through use of voting electronically:

There needs to be more talking to learners and one to one contact. With questionnaires, the qualitative part can be illuminating. There is now the power of e-voting as well Straight away, they could see what everyone else thought (ELU1)

The respondent indicated that even in questionnaires, the qualitative section was often the most revealing. This finding is relevant to literature with a focus on student voice. To fully capture the student experience, the respondent stressed the centrality of individual feelings in the ELU, saying:

We deal with the human side (ELU1)

(iii) Provision of technical support

When asked about the more technical aspects for students of managing their learning through U-Learn, the respondent alluded to the importance of the interface between IT services and the helpdesk. It was suggested that problems related to browsers and logging in had reduced through simplifying the technology.

It was felt to be important to ensure that learners received on-line help within 24 hours. Interestingly, a common problem for learners was reported to be management of time on-line:

On-line learning still means giving a structure. It may be based on weekly tasks. Its not a free-for-all. What are you expecting students to do and in what timeframe? So students know whats required of themStudents with families are often better at managing their time (ELU1)

The respondent emphasised the need for a thorough induction for each student undertaking the mentor preparation module, suggesting that it could make or break a students experience:

Work was done on the induction for students. More cohorts are going through therefore there has been some refining as weve gone through. One hour in the lab can increase confidence and knowledge about help thats available etc. Induction can make or break a student. The induction is crucial for students, particularly those who arent IT literate. So even if you have an hour with them in the lab, its about how you make sure they leave in that hour happy, confident, positiveknowing where they can get helpThe tutors [module leaders] were very goodvery reassuring. And that sets the tone (ELU1)

The above quote emphasises the value of face to face contact at the outset in order to guage reactions of students if new to e-learning.

(iv) Managing the learning journey

The respondent emphasised the importance of the tutors being explicit and expectations for the module and also conveying all available options to students:

Its very useful for the tutor to be explicit about the avenues available. You need to include a page, early on, that says If youve got a problem you can post a message on the discussion board or email. Weve had examples where weve set up discussion rooms and absolutely nothing has happened in them at all!

The respondent went on to stress the importance of providing explicit instructions and support:

Be explicit about what the rules and expectations are. You have to model the practice I think. Bearing in mind that its the first time that some students have been exposed to this type of learning, I think you have to be almost over-explicit, until they kind of learn the rules for themselves (ELU1).

To help guide the students through a module delivered through blended learning, the nature of tips necessary to manage the experience was offered:

Time and structure are important Its how you design the opportunities that are there. Everyone will slightly adapt them for their own needs. A range of techniques are needed eg make sure you log on frequently. Just short but frequent logging on. Generic tips like that (ELU1)

It is evident that students need to be equipped with a range of strategies to assist building of their portfolios and undertaking group work and projects on-line.

Summary

The interviews with the teachers provided interesting insights into their approaches to implementing and developing an electronic portfolio with no template. Several tutors described how they had embraced the concept and had moved from traditional modes of teaching to working within an e-tutoring model. The data provide evidence that the project objective concerning raised staff understanding of e-portfolios as a way to engage students meaningfully was largely met. The enthusiasm with which all the tutors used the ULearn interface to promote deeper learning for students was commendable.

7.3 Learner mentor biographies

Four mentor learners from the two successful cohorts of 67 students volunteered to participate in the biographical narrative interview. The settings were varied and reflect the range of placements offered to students at the University of Surrey. They comprised:

A district nursing setting (Melanie)

A community mental health setting (Aisha)

A hospice setting (Ruth)

A school nursing setting (Pat)

Using thematic content analysis five themes were generated from the interview data:

1 aligning expectations with reality

2 circumventing technological challenges

3 experience of learning within a blended approach

4 strategies for learning and teaching

5 views of a modernised module

These themes are presented in the table below alongside five of the constructivist learning categories from Jonassens (2001) rubric: intentional, contextual, collaborative, reflective and constructive.

Content analysis theme

Constructivist learning category (after Jonassen 2001)

Aligning expectations with reality

Intentional: all human behaviour is goal directed. The goals are either simple or complex. To achieve their cognitive goals and meet the module objectives, the respondents needed to actively and intentionally align their expectations with reality.

Circumventing technological challenges

Contextual: Learning is more meaningful and transferable to new situations if it is situated authentically. For this reason, the respondents needed to identify any technological obstacles to enhance learning within new contexts.

Experience of learning within a blended approach

Collaborative: Learning occurs through social support and modelling in knowledge building communities. The experience of learning within a blended approach meant the respondents had to collaborate on learning projects through on-line cafes and discussion groups.

Strategies for learning

Reflective: articulation of learning from a technology-based curriculum assists in a fuller understanding and better use of the knowledge the learners have constructed in new situations. Through describing their strategies for learning, the respondents were reflecting on processes and decisions which informed their learning.

Views of a modernised module

Constructive: To make meaning, learners need to integrate their prior knowledge with new ideas so they construct their own meaning from different phenomena

Aligning expectations with reality (Intentional)

As suggested by Jonassen (2001), respondents needed to consider their biographical learning histories in order to align expectations of the mentor preparation module with the reality of the actual programme and the challenges of undertaking new tasks. The cognitive goals (including undertaking a team assignment) and the intentionality involved, were displayed in various ways across this data set:

Because Melanies training as an enrolled nurse had not involved teaching, she described feeling as if she had missed the loop.

I feel I missed the loop because enrolled nurse training in 1982 didnt sort of encompass looking after students I never really understood it through the eyes of the student nurse. ..So it was very sort of hit and miss (Melanie p6, part 2)

Melanies expectations, however, differed from reality as she was unaware that the module had been re-designed:

But it wasnt until we got here that the tutor said that theyd re-designed it and within the course I think they called it U-learning? And so I was a bit worried because I havent got huge computer skills and I was worried that Id be struggling on it. So ur I was quite frightened about doing it really (Melanie BNI).

Aisha vocalised performing teaching on auto pilot prior to undertaking the mentorship module:

when youre in practice for a long time without studying you get used to the way you do things and you dont necessarily, you do a lot of it on auto pilot, especially with regards to your teaching. You had like a student shadow you and I dont think I thought about it as much as I did when I did the course (Aisha BNI)

Aisha also described her intentional, goal-directed approach as a participant constructing her own learning:

but I guess its reinforcing the fact that youve got a lot to learn yourself, youre always on a learning curve and I guess its important to kind of ensure youre kind of directing that and not just kind of going along with day to day stuff(Aisha)

Ruth was a little more critical of the pre-course computer literacy preparation and described how her goals had not met with the module objectives:

We did have a questionnaire about computer literacy and what we knew about computers. Um Mines basic. I can cope-but only just. So Id have appreciated perhaps a bit more of that and might have gone on the computer more before I started. (Ruth p1)

The following respondent (Pat) highlights the importance of a mentor to help align expectations of all involved and to assist in realistic goal setting. This is to avoid feeling thrown in to challenging teaching situations:

And I also feel that because of [tutors] perceptions, I was thrown in at the deep end with the student and just left to get on with her and not have the experience of an SPT [Specialist Practice Teacher] really, close to hand, to help me through the course. I was just left to get on with it. (Pat P2)

The above quote was from Pat, who described immense difficulties she had experienced when mentoring one student in practice. Pat had evidently expected far more hands-on support and therefore expressed disappointment.

Circumventing technological challenges (Contextual)

The nature of the technology-led emphasis in the new module meant that the learners needed access to software such as Java and compatible hardware. This was not always easily obtainable. Melanie described her anxiety related to work getting lost in the ether:

I couldnt download the Java so I couldnt use it at work. I had to use it at (pause) home. So I had to ask my son to sort it out for me, to make my computer compatible to the Universitys. So that was all a bit anxious for me but we managed to sort it outAnd that was another thing because the Java wasnt acceptable in the format that I had on my computer [I was] worried that work had got lost in some sort of black hole (Melanie BNI3)

Technical support from the E-Learning Centre was evidently available and useful:

But as time went on-and um there was quite a lot of support from the tutors and one of the chaps that designed the um U-learning sort of computer system, I felt a lot more happy about it. (Melanie BNI1)

Other respondents also described challenges of parking of assignments on-line.

I got a bit confused with parking assignments and retrieving them. And actually there was a fault with that because we were able to get everybodys essays and things at one point. I think that was then rectified. Because I couldnt park mine, my modules, um essays. It just wouldnt park. And I found that really difficult. not a positive thing (.) from it. (Ruth)

The contextual elements were intrinsic to these students learning. As found in the literature, the asynchronous nature of communicating on-line was not always conducive to vibrant discussion within internet cafes:

The things about the U-learn, that has provoked a lot of agitation I think, a lot of anxiety in the beginning especially I enjoyed that side of it, but I think its about anxiety if people couldnt get onto a computer and that had an impact on our group work. So several times there was only me actually on it doing some work, myself and one other person. Pat P3

Greenberg (2004) suggests that, to avoid isolation (perceived by three out of four students) network communication tools need to be both synchronous and asynchronous. The virtual learning environment at the University proved to be a double edged sword. Pat describes the value of the VLE for group work and individual searching, but not assessment:

I found U-learn very helpful for the initial things that we did in the teaching. For instance, in our group work we chose motivation and it was very, very useful. Things were put on the U-learn site for us to go in or we did our own searches on the internet and that was very, very usefulWhen it came to assessment though, I dont remember U-learn being very helpful at all really. There didnt seem to be anything on there that was particularly helpful,

(Pat p10, part 2).

As described earlier, Pat had been influenced by a negative incident with a student and therefore seemed to view assessment as a barrier to learning using technology.

Experience of learning within a blended approach (Collaborative)

As found with the e-portfolio system used at Wolverhampton University (Pebble Pad), the on-line community used the digital format as an electronic reflection and communication tool. The following quotes provide information about the actual experience of blended learning, which is rather sparse across the literature :

It wasnt just about computer systems. We had to come in and have some taught sessions which I thought were really good and really up to date. And I thought there was plenty of time to get the work in. And I thought with the people on the course there was a really good mix of people. You got ..rather than having just one type of nurse you know from one area. So there was a really good mixture which kind of enriched the team.(Melanie)

Melanie provides information about how she managed her time on-line:

Because we had our little group given to us. And we all agreed to go um on-line at 8 oclock in the evening and have a chat, like a web chat about what we were going to do this presentation on and I sat there-8 oclock, went on line and there was nobody there ( Melanie part 2)

The above quote provides evidence of the challenges for some mature students in undertaking group work asynchronously. Aisha appeared to work in a group with a different approach to on-line discussions:

the e-learning, I think we could have made more use of it, like my group made, because you could discuss your group work and we utilised it quite well but that was because most of the group had access to it so people checked on it quite a lot(Aisha)

Ruth described some of the barriers and frustrations inherent in attempting to communicate on-line:

The other thing as regards using U-Learn. You were put into groups to do a presentation. I found that extremely difficult a) because I was out of the area and b) because no-one was on line the same time as I was And I think I sent about 30 emails and didnt get any replies for a long time. So trying to formulate a presentation with the rest of the group was difficultWorking shifts isnt conducive to being in contact at a certain time of day, really (Ruth).

What appeared to compensate for this, was the immediacy of tutor support from the University:

Tutors did respond to emails. I was surprised that someone could respond that quickly. Almost spontaneously really. They seemed to be on line all different times of day and night. Which is quite reassuring really. And gave you know, positive. You know, that feedback. And helpful. Answered the questions really that were asked. That was good. YehI think (.) the staff were very supportive (Ruth).

Pat, who was probably more used to presenting independently in the community, described how positively her group gelled:

The teaching side I really enjoyed. I enjoyed all of the group work that we did and we had to get into groups and do a presentation and I think I enjoyed that, probably because in my role I do do lots of presentations anywayI think also in our group we were quite lucky, although it was randomly, we seemed to gel really easily and seemed to get on very well, so that was good (Pat)

Strategies for learning and teaching (Reflective)

As found in the literature (particularly that related to views of trainee teachers), quite intense preparation was required prior to students using technology for the first time (Woodward and Nanlohy 2002, Scholes 2004, Butler 2006). Tensions with attempting to cope with new technologies are evident in the following quote:

Ive got to move on with the technology because I cant communicate and move my knowledge forward if I dont start using the new technology. So I have this sort of push me, pull me feeling about how I cope. Melanie, P2

Ruth described how the face to face teaching compensated for her lack of technological confidence:

The actual lectures and things were excellent. I felt really supported (.) They were taught really well I felt. Um. There was a lot of humour involved. And thats how I learnUm. Certainly the hand-outs and the reading lists that we got were excellent. And I gained a lot from that. I think more role play in the module would be good. (Ruth p2)

Ruth went on to share how confidence in technology to assist formal presentations had increased, adding that the blended approach used at the University had triggered using new approaches to assess students in practice:

So Ive developed in the fact I can use computers better, despite all the hiccups. Yknow I could do a Powerpoint and even get the bits coming in from the side and all the other bits. So thats been of value and I can use that in my teachingAnd weve altered our approach to students to their evaluationIm more of a practical teacher I think, rather than a theoretical oneBut Ive certainly learnt the the research and the knowledge behind it now. I think thats the value. (Ruth, part 2)

Aisha described how reflection was informing her own teaching style:

[I] selected the information according to what [the student is] more likely to come across rather than something thats kind of common, so it would be useful to her I structured it in relation to [the mental state examination] to make it more meaningful to her (Aisha p2, part 1)

The individuality of the students was being encompassed to provide a relevant practicum for the student and to also influence her divergent thinking:

all students are different and theyll bring something and make you think in a different way(Aisha P8, part 2)

Pat described how the module had inf