IGI P Raith, U. of Education Heidelberg, Germany Ken Reeder, U. of British Columbia, Canada Thomas...

Antonie Alm, U. of Otago, New ZealandKikuo Asai, Open U. of Japan, JapanElena Benito-Ruiz, Universidad Politécnica de Valencia, SpainDiana Bannister, U. of Wolverhampton, UKFrançoise Blin, Dublin City U., IrelandStephen Bronack, Clemson U., USAAaron P. Campbell, Kyoto Sangyo U., JapanMohamed Amine Chatti, RWTH Aachen U., GermanyAlice Chik, City U. of Hong Kong, Hong KongMichael Coghlan, TAFE, AustraliaJohn Collick, Promethean Co. Ltd, UKJohn Cook, London Metropolitan U., UKCristina Costa, Salford U., UKEdward Dixon, U. of Pennsylvania, USADarren Elliott, Nanzan U., JapanJames A. Elwood, Tsukuba U., JapanOla Erstad, U. of Oslo, NorwayPatrick Foss, Tokyo Medical and Dental U., JapanNicolas Gromik, Tohuko U., JapanSara Guth, U. of Padova, ItalyRegine Hampel, Open U., UKMirjam Hauck, Open U., UKDon Hinkelman, Sapporo Gakuin U., JapanJane Hunter, U. of Western Sydney, AustraliaMarcia Johnson, Waikato U., New ZealandTony Jones, U. of Melbourne, AustraliaHelen Keegan, Salford U., UKJeremy Kemp, San Jose State U., USA

Jaroslaw Krajka, Warsaw School of Social Psychology, PolandMark Lee, Charles Sturt U., AustraliaJ. P. Loucky, Seinan JoGakuin U., JapanSteve McCarty, Osaka Jogakuin, JapanGeorge MacLean, Tsukuba U., JapanDave Miller, Keele U., UKGary Motteram, U. of Manchester, UKTony Mullen, Tsuda College, JapanDiane Nahl, U. of Hawaii, USAAnna Peachey, Open U., UKNik Peachey, ICT Consultant, UKMark Peterson, Kyoto U., JapanThomas Raith, U. of Education Heidelberg, GermanyKen Reeder, U. of British Columbia, CanadaThomas Robb, Kyoto Sangyo U., JapanKarl Royle, U. of Wolverhampton, UKBernd Rueschoff, Universität Duisburg-Essen, GermanyMathias Schulze, U. of Waterloo, CanadaKieron Sheehy, Open U., UKLeslie Shield, Open U., UKSukunesan Sinnappan, Swinburne U. of Technology, AustraliaSandra Sutton Andrews, U. of Northern Arizona, USAHagit Tal, Open U. of Israel, IsraelRicardo Torres, Citilab-i2Cat, SpainTracy Villareal, U. of Texas at Austin, USACharles Wankel, St. John’s U. - New York, USAPeter Worrall, Media Consultant, UK

Editor-in-Chief: Michael Thomas, U. of Central Lancashire, UK

InternationalAdvisoryBoard: Axel Bruns, Queensland U. of Technology, Australia Edward Castronova, U. of Indiana, USA Thomas M. Connolly, U. of the West of Scotland, UK Gavin Dudeney, The Consultants-e, Spain Steve Higgins, U. of Durham, UK Carey Jewitt, U. of London, UK Rob Koper, Open U., The Netherlands Agnes Kukulska-Hulme, Open U., UK David McConnell, Glasgow Caledonian U., UK Hayo Reinders, Middlesex U., UK Roo Reynolds, BBC Vision, UK Jonathan Richter, U. of Oregon, USA Niall Sclater, Open U., UK George Siemens, U. of Manitoba, Canada Jeroen van Merriënboer, Open U., The Netherlands Mark Warschauer, U. of California - Irvine, USA Crispin Weston, SALTIS, UK

AssociateEditors: Michael DeMers, New Mexico State U., USA Daniel Livingstone, U. of the West of Scotland, UK Barbara Payne, U. of Hawaii, USA Mark Pegrum, U. of Western Australia, Australia Jonathan Richter, U. of Oregon, USA Michael Vallance, Future U., Japan

BookReviewEditor: Sarah Robbins, Ball State U., USA

IGIEditorial: Heather A. Probst, Senior Editorial Director Jamie M. Wilson, Assistant Director of Journal Publications Chris Hrobak, Journal Production Manager Gregory Snader, Production and Graphics Assistant

InternationalEditorialReviewBoard:

IGI PublIshInGwww.igi-global.com

IGIP

IJVPLEEditorialBoard

The Editor-in-Chief of the International Journal of Virtual and Personal Learning Environments (IJVPLE) would like to invite you to consider submitting a manuscript for inclusion in this scholarly journal.

missionThe mission of the International Journal of Virtual and Personal Learning Environments (IJVPLE) is to study and disseminate research about the design, development, and evaluation of online learning environments. IJVPLE is committed to encouraging the best teaching and learning practices by examining the role of technology enhanced learning in the emerging area of virtual and personal learning environments.

Topics of inTeresT (include buT are noT limiTed To):• 3D online worlds• Adaptive and intuitive environments• Applications of the Semantic Web• Blended learning• Collaborative learning• Computer aided language learning (CALL)• Computer-mediated communication• Computer-mediated conferencing• E-assessment• Educational mashups• E-learning• Emerging technologies• E-portfolios• Gaming and learning• Intelligent content• Knowledge management and learning• Lifelong learning• Mobile learning environments and applications• Multimedia applications and virtual reality• Networks/grids for learning• Platforms and authoring tools• Remote and virtual laboratories• Responsive environments• Virtual learning environments• Web-based learning (WBL)

Allsubmissionsshouldbee-mailedto:MichaelThomas,[email protected]

An official publication of the Information Resources Management Association

InternationalJournalofVirtualandPersonalLearningEnvironments

Please recommend this publication to your librarian. For a convenient easy-to-use library recommendation form, please visit:

http://www.igi-global.com/ijvple.

Ideas for Special Theme Issues may be submitted to the Editor-in-Chief.

CALL FOR ARTICLES

ISSN 1947-8518eISSN 1947-8526

Published quarterly

January-March 2012, Vol. 3, No. 1

EditorialPrefacei Michael Thomas, University of Central Lancashire, UK

ResearchArticles1 PersonalLearningEnvironments:ConceptorTechnology?ClickIfYouWanttoSpeak:

ReframingCAforResearchintoMultimodalConversationsinOnlineLearning Marie-Noëlle Lamy, The Open University, UK 19 TheDevelopmentofE-PortfolioEvaluationCriteriaandApplicationtotheBlackboard

LMSE-Portfolio Gary F. McKenna, University of the West of Scotland, UK Mark H. Stansfield, University of the West of Scotland, UK

37 AVirtualWorldWorkshopEnvironmentforLearningAgileSoftwareDevelopmentTechniques

David Parsons, Massey University - Auckland, New Zealand Rosemary Stockdale, Swinburne University, Australia

55 UtilizationofIntelligentSoftwareAgentFeaturesforImprovingE-LearningEfforts: AComprehensiveInvestigation

Mandana Farzaneh, University of Tehran, Iran Iman Raeesi Vanani, University of Tehran, Iran Babak Sohrabi, University of Tehran, Iran

BookReview69 Second LanguageDistance Learning and Teaching: Theoretical Perspectives and

DidacticErgonomics Iffaf Kahn, University of Essex, UK

InternatIonal Journal of VIrtual and Personal learnIng

enVIronments

Table of Contents

mnl2

Sticky Note

Insert space between question mark and colon

i

Editorial PrEfacE Michael Thomas, University of Central Lancashire, UK

Edition 3(1) of the International Journal of Virtual and Personal Learning Environments (IJVPLE) contains four research articles and one book review. In the first paper, “Click If You Want to Speak: Reframing CA for Research into Multimodal Conversations in Online Learn-ing, Professor Marie-Noëlle Lamy from the UK Open University focuses on the neglected area of data collection vis-a-vis audiographic platforms. Emphasising the importance of re-searching conversations in language learning, Lamy argues that neither computer-assisted language learning (CALL) nor conversation analysis (CA) has adequately produced a methodology for analysing these interactions. In conclusion Lamy suggests that it is neces-sary to move to a crossdisciplinary approach in which constructs from CA are used alongside those from social semiotics to provide a way forward for researchers.

In the second paper, “The Development of E-Portfolio Evaluation Criteria and Application to the Blackboard LMS E-Portfolio,” Gary F. McKenna and Mark H. Stansfied address a gap in the literature on e-portfolios following an extensive process of research concerning publications between 1995 and 2010. Although e-portfolios have become increasingly popular

across the educational spectrum few research studies have focused on the development of e-portfolio effective practice frameworks. The paper examines the use of an e-portfolio to support Personal Development Plans and provides a much-required starting point for further research on the topic.

In “A Virtual World Workshop Environ-ment for Learning Agile Software Development Techniques,” David Parsons and Rosemary Stockdale present findings from a longitudinal research project using Multi-User Virtual En-vironments (MUVE). Adopting a design-based research approach grounded in cycles of learn-ing and the paper examines the trajectory from a real world workshop to its virtual incarnation in the MUVE. The paper concludes by discuss-ing the insights vis-à-vis learner perceptions and practical implementations arising from the research.

In the final research paper in this edition, “Utilization of Intelligent Software Agent Features for Improving E-Learning Efforts: A Comprehensive Investigation,” Mandana Farzaneh, Iman Raeesi Vanani, and Babak Sohrabi illustrate the importance of compre-hensive categorization of intelligent software agent features which can be used in the virtual

ii

world of learning and training. The agents are particularly important for the implementation and improvement of e-learning information systems, and the paper provides an extensive exploration of agent-related international papers focusing on a wide range of internationally ac-cepted features. The agents enable the online learning mechanisms to be potentially more efficient and effective than traditional learn-ing systems. The paper argues that the use of agents can help instructors transfer knowledge easier, faster, and in a more targeted fashion to the intended audience.

This edition is completed by Iffaf Kahn’s book review of Second Language Distance Learning and Teaching: Theoretical Perspec-tives and Didactic Ergonomics by Jean-Claude Bertin, Patrick Gravé and Jean-Paule Narcy-Combes and published by IGI Global.

We hope you enjoy reading this volume and that you will consider contributing to the journal in future editions.

Michael ThomasEditor-in-ChiefIJVPLE

International Journal of Virtual and Personal Learning Environments, 3(1), 1-18, January-March 2012 1

Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

Keywords: Audiographic, Computer-Mediated Communication, Conversational Analysis, Embodiment, Multimodality, Second Language Acquisition, Social Semiotics

INTRODUCTION

This paper reports on research into conversa-tions undertaken in voice-enabled learning platforms by learners seeking to enhance their oral and written fluency in a second or foreign language. The aim of the study is to establish methodological principles to address a gap in current research, which relates to the way that data collected from such conversations is analysed. Focusing on the phenomena of turn-taking, face-saving and use of space in online language tutorials, the study takes Conversa-tional Analysis (CA) as its theoretical starting

point but adopts a multimodal perspective that appears to shed greater light on the meaning-making processes involved in interactive digital environments.

We will start by establishing the need for an understanding of what learners are doing when they converse via learning platforms such as Lyceum, Elluminate, Centra Symposium and others. We will then discuss the applicability of CA to multimodal conversations online. Fi-nally, drawing from communication theory and multimodal social semiotics, we will identify three frameworks which may provide guiding principles for a methodology with the potential to answer questions raised by our data.

Click If You Want to Speak:Reframing CA for Research into Multimodal

Conversations in Online LearningMarie-Noëlle Lamy, The Open University, UK

ABSTRACTThis paper addresses the lack of formalised methodology for analysing learner interaction data created in conversations on audiographic platforms. First the author shows the importance of conversations in language learning and the need for researchers to understand how users learn from these interactions. Then the author establishes that appropriate methodologies for investigating interaction data collected from online platforms have as yet emerged neither from the field of computer-assisted language learning nor from conversation analysis (CA). Three brief multimodal conversations involving language learners in platform-based tutorials are analysed. The author shows that linguistic means of communication are only one way in which to achieve learning aims and other communication modes are identified. The author concludes that the analysis and interpretation of such exchanges can be improved by a crossdisciplinary approach which consists of augment-ing constructs drawn from CA with selected constructs from social semiotics.

DOI: 10.4018/jvple.2012010101

2 International Journal of Virtual and Personal Learning Environments, 3(1), 1-18, January-March 2012


CONVERSATIONS FOR LANGUAGE TEACHING AND LEARNING: FROM FACE TO FACE TO ONLINE SETTINGS

Conversations are seen as beneficial to language learners for two reasons. Firstly oral conversa-tions have been core to the communicative model of language teaching for half a century, as part of conversation classes. In these, learn-ers, in the safety of the classroom, are invited to experience the “pressure of conversation” (Cook, 1991, p. 61) that it is assumed they will face when called upon to talk with na-tive speakers in the target country. Secondly, research carried out since the mid-80s within Second Language Acquisition (SLA) theory, including in computer-assisted settings where for a long time only written exchanges were possible, has established a consensus about language learning. Specifically it is agreed that, providing interactions are structured so as to require participants to negotiate meaning, con-versations can indeed promote socio-cognitive progress (Gass & Varonis, 1994; Long, 1983; Gass, Mackey, & Pica, 1998; Chapelle, 2004).

To attain these learning outcomes, re-search has shown that conversations have to satisfy certain criteria. They need to be part of a constraining task: typically, a group of learn-ers might be asked to address a problem then reach a negotiated consensus. Synchronous tutorial platforms can offer such opportunities: they are structured so as to allow learners to negotiate in groups online (through the use of ‘grouping’ tools and ‘breakaway rooms’), to respond individually (by text-chat or audio) or collaboratively (with written documents and graphic objects co-created during the interac-tion). Multimodal platforms, which provide group-management tools, and written as well as aural/oral facilities, are thus of great interest to language teachers and they are gradually be-ing adopted in language courses as vehicles for conversation practice around language tasks.

RESEARCHING LANGUAGE LEARNER EXCHANGES IN TECHNOLOGICAL ENVIRONMENTS

However, teachers as well as researchers often fail to take proper account of the nature of the new environments in which exchanges occur. For example, noting teachers’ preference for linguistic communication over other means available in such environments, Karabulut and Correia (2008) judiciously warn them that “if not carefully designed, web-based videoconfer-encing sessions may turn into mediocre lectures because not full advantage of the medium is taken. Converting in-class interactions into synchronous environments requires extensive design considerations” (Karabulut & Correia, 2008, p. 481).

Concerning researchers in the computer-mediated language-learning community, with some exceptions to which we return below, they have drawn from online learner data in order to investigate aspects of the language acquisition and socialisation processes, rather than examine the online learning event as a whole. For ex-ample they have studied group dynamics (Vetter & Chanier, 2006; Reffay & Chanier, 2003), task design (Rosell-Aguilar, 2005; O’Dowd & Ritter, 2006; Hauck & Youngs, 2008), affective variables (Payne & Whitney, 2002; Hauck & Hurd, 2005), fluency and accuracy (Xiao, 2007) and “electronic literacy” (Helm & Guth, 2010). This literature asks to what extent the tools help or hinder the pedagogical processes that are accepted as the most effective for language learning. However, in its often stated concern – with which this author sympathises – to avoid technology-centered interpretations of human learning, it focuses away from the materiality of the hardware and software, failing to heed Kress’ (2003) advice to see “the material stuff” (Kress, 2003, p. 32, original emphasis) that sur-rounds us as a resource for making meaning, an insight that we will show to be relevant to the analyses in this article.

Approaching multimodal environments with greater awareness of their nature, Hampel



(2006) advocates tasks that “take account of the affordances of the different modes in the online medium by using certain modes for certain purposes” (Hampel, 2006, p. 113), and makes recommendations as to how this can be achieved. However, while educational designer research of this kind lies what we consider to be the appropriate stress on the material features of the medium, nevertheless of its own admis-sion it stops short of providing a way into the complexity of multimodal discourse itself.

Berglund’s (2009) ecological perspective comes closer to our own orientation, in that she recognises the importance of the design and materiality of the environment, and she describes, as will we, later on, some of the ways in which seemingly incoherent conversations can be reinterpreted as coherent when all the artefacts are taken into account (Berglund, 2009, p. 198). However her ecological approach is mainly theorised in terms of Common Ground Theory, a psychological theory with a focus on common elements facilitating communication. Again, this offers a lens through which to view multimodal conversation data, but one which we believe is limiting because (in contrast to CA) it ignores subversive communication practices and face-saving issues, and (in con-trast to semiotics) its focus is not primarily on meaning-making.

Research addressing the methodology of data collection and interpretation in language-learning projects in these environments is also scarce, with the exception of Smith, who investi-gated concepts from SLA, such as learner uptake of linguistic form and negotiation of meaning (Smith, 2007), and self-repair (Smith, 2008). In the latter study, this author uses dynamic screen capture software to collect self-corrections that chat participants made to their messages prior to clicking the ‘send’ button. Thus Smith (2008) widened the window of observation to include different kinds of spaces in which participants operate: screen spaces and private physical spaces. This stress on space will prove important in the discussion section of the current study.

CA AND TECHNOLOGY-MEDIATED CONVERSATIONS

Other disciplines have long worked with CA to research technology-mediated exchanges. For example, in sociology, drawing on Goffman’s (1967) work on mother-tongue face-to-face interactions, Sacks, Schegloff, and Jefferson (1974) used CA to analyse everyday conversa-tions in a technologically-mediated setting, the telephone. Hutchby (2001) further extended the field of application of CA to Internet-mediated (including video-enhanced) everyday interac-tion. In applied linguistics, Mondada (2004) has used CA to explore video-mediated professional conversations, for example telesurgery involv-ing surgical teams using both mother tongue and the foreign language. In educational technology, a comprehensive examination of the relationship between CA and technology has been offered by Mazur (2004). In this work, she encourages researchers working with online learner talk to attend to methodological issues pertaining to both CA and to technology, for example the use of tools for visualizing conversations, and she shows how to use CA to investigate knowl-edge construction and community building in computer-mediated environments.

However in a review of language learning research, Lamy and Hampel (2007) found that CA was not a big part of language researchers’ methodological armoury. Certainly Garcia and Jacobs (1999) made an early attempt to introduce CA into a piece of foreign language learning research and demonstrated CA’s appropriate-ness for analysing turn-taking in text-chat.

Yet when Lamy et al. (2007) looked at pa-pers written between 1999 and 2005, to identify those which specifically named the research methodology used, they found that Garcia et al.’s (1999) early example had not been followed. Only 12 of the papers in the corpus explicitly identified their analytical approach (Garcia et al., 1999 p. 56). Among them, 10 studies said they used Discourse Analysis, leaving only two with a declared commitment to CA, Negretti (1999) and Tudini (2003), and one other with some influence of CA, Payne and Ross (2005).



None of them included a visual or multimodal element.

Some of the issues raised by Negretti’s (1999) attempts to use CA as a means to investi-gate learner communication in CMC arise anew in the current research. Her study analyzed text chat between natives and non-natives, and iden-tified turn-taking, turn design and paralinguistic features as observable phenomena that could tell us about how “participants deal with restrictions on sense-making procedures imposed by the Webchat environment” (Negretti, 1999, p. 86). Also focused on text-chat, Tudini (2003) and Payne et al. (2005) used elements of CA to argue that there is a relationship between written chat proficiency and oral proficiency in the foreign language. However to date there has been no study of conversational meaning-making in a multimodal second-language environment, with the exception of our own work (Lamy, 2004, in press), where we use empirical data to explore aspects of the semiotics of audiographic platforms and personal videoconferencing environments respectively.

In summary, existing research on analysing conversations in technological environments has in our view stopped shy of embracing the complexity of multimodality, either because the researchers have shown little interest in matters that they believe lie outside the linguistic part of the conversational exchange, or because, while aware of multimodality issues, their priority has lain in a specific direction (task design, com-munication, acquisition, community-building or other). Furthermore, some of the work that has attempted to marry the CA methodology with online analysis has historically been confined to synchronous written text environments. In order to widen the lens and pick up where preceding research left off, the present paper adopts an open, inductive approach to the data, looking at technology and conversations systemically.

THE NATURE OF THE RESEARCH GAP: CA AND SYNCHRONOUS MULTIMODALITY ONLINE

Among the tenets of CA, we will select two as be-ing pertinent to the data to be introduced shortly (for a full account of CA see Wooffitt, 2005). Like all CA principles, they can account for conversational material both when participants abide by them and when they breach them. In a simplified form they can be summed up as fol-lows: (a) turns of speech alternate and interlink, since a basic principle known as ‘conditional relevance’ asserts that given a question, you may expect an answer, given an apology, expect an acknowledgement, given a topic, expect that it will be pursued and that violating this rule is a conversational move that CA takes as a proper part of its object; (b) ‘face-saving’ is a major preoccupation in any conversation, because conversants are in a relationship of perpetu-ally converging and diverging interests with their fellow-conversants: you may try to save your own face or to protect the face of others, and it is also possible to threaten one’s own or others’ faces for strategic reasons; again, the CA methodology can account for the ways in which through these manoeuvres, interactants achieve conversational ends.

In face-to-face conversations, linguistic resources are mobilised in order to carry out such strategies, but in the environment under study, discourse partners use many semiotic systems, including linguistic (written and spoken), iconic and symbolic systems. They may use these sys-tems in rapid succession (for example type in the text-chat box then draw on the whiteboard), quasi-simultaneously (for example speak into the microphone while hitting the ‘send’ button to post a written message) or they may choose among systems to make meaning in particular ways. For example, a user may close a conver-sation by typing ‘Bye for now’, by clicking a specific button, or by announcing their with-drawal orally. Different semiotic systems are also involved in responding to such a move. For example, if users of a synchronous environment

mnl2

Sticky Note

Instead of (Lamy, 2004, in press) this bracket should now read (Lamy, 2004, 2011)



signal their impending departure orally, their discourse partners will receive an aural input, made up of linguistic and non-linguistic infor-mation, e.g., a phone overheard ringing in the location of the departing participant may give remote partners a clue to the person’s reasons for terminating the conversation. Or a user may type a valedictory message into a box and send it. In this case, their message will be displayed on the shared screens and will remain a part of the ongoing conversation even after the sender has disappeared from the environment. Or again users might avail themselves of the system’s telepresence indicators, e.g., icons of different colours signalling that users are ‘of-fline’, ‘online’ or ‘on standby’. Or the system may display symbolic changes to the objects on the screen, such as the greying-out of the name, or the fading-out of the photo of the departed partner. Choosing to leave the conversation by clicking an icon that means ‘temporarily away’ is a strategy likely to influence the direction of the conversation in a way that is distinct from the effect produced by disconnecting altogether (in which case the name or image would vanish), or from the effect of leaving all connections ostensibly active but walking away from the computer and remaining absent for the rest of the session.

Thus in environments with multiple meaning-making devices, the combinations and interlinking of semiotic systems and effects can become extremely complex, and there is scope for observing conditional relevance and face-saving at work, through studying the us-ers’ conventional use of the tools as well as any ‘workarounds’ that they may adopt.

THE PROJECT: ENVIRONMENT, PARTICIPANTS AND LEARNING CONTEXT

In the rest of this paper we seek theoretical support when attempting to grasp this complex reality, by studying three brief conversations carried out on an audiographic platform as part of an English-for-Special-Purposes project at

the University of Franche-Comté in France. Before describing the project and introducing the data sets, however, a terminological note is required. Central to our project is a focus on ‘multimodality’, defined for the purpose of this study as the co-availability of several modalities, this term ‘modality’ itself being understood as the material through which the conditions for the creation of meaning are made available to users. For example the relationship between any technological feature (e.g., a hyperlink) and a semiotic resource (e.g., the meanings in the pop-up that appears when the hyperlink is clicked) is a relationship of modality. Additionally the pop-up itself carries two types of meaning. First, as part of the overall web text is has structural meaning, through the very fact of its being a pop-up rather than, say, a note on the main page, or a hyperlink to a different sub-page. Second, the pop-up’s material appearance (in a static or floating frame, opaque or translucent) is also a resource with semiotic potential. Multimodal-ity research is concerned with all the possible meanings. With multimodality, several semiotic resources (e.g., linguistic and visual signs) may be associated with a single modality (e.g., meanings created when a pop-up appears) or a single semiotic resource (e.g., written language) may be associated with two different modalities (e.g., user-created meanings, in chat messages, and codified system-generated ones in ‘alert’ or ‘status’ windows). These considerations define the ways in which we will be using the terms modality and multimodality in the rest of this paper.

In the project under study we used Lyceum, a synchronous audiographic groupware system designed to facilitate social learning in distance tutorials. Figure 1 shows the characteristic tools and spaces of this tutorial platform. On Figure 1 from top left, anticlockwise on the image: a facility for creating ‘rooms’ for small-group work, a space representing the participants (here four people, with names withheld), buttons for indicating agreement or disagreement (Yes/No buttons, also known as ‘Tick’ icons), a window for textchat input and, in the centre of the image,

mnl2

Sticky Note

Insert a space between 'with' and 'multimodality'



a space for collaborative writing, here shown with a brainstorming session in progress.

Figure 2 shows icons representing the conversational status of participants, displayed to facilitate turn-taking. Participant T has his ‘hand’ raised, indicating that he is waiting for a turn. Participant G is talking, as shown by the loudhailer icon.

While technically the University’s servers may accommodate hundreds of connections at a time, the logistics of turn-taking on this plat-form is such that as many as 10 to 15 people may participate in each tutorial room. How-ever the subdividing of spaces for plenary groups into separate sub-spaces for groups of 3 to 5 students (an optimum number for effective language-learning conversations) can be achieved by clicking, and is almost instanta-neous. Within each sub-space, different tools and shared documents or whiteboards can be displayed and collaboratively used. A turn-taking management system based on clickable icons is another feature of such platforms that has relevance for our study.

The extracts discussed below originate in a project known as Copéas, a partnership be-tween the University of Franche-Comté and the UK Open University. Sixteen French-speaking students studying for a Professional Masters in Open and Distance Teaching (ODT) worked in two groups of eight, connecting from their homes in various parts of France. Each group had an English native speaker tutor connecting from the United Kingdom. The groups met dur-ing 10 sessions of over an hour each. The course had a dual objective, linguistic and professional, which was the development of competences in ODT through spoken and written English. The less proficient of the two groups (French students who were rated as ‘false beginners’ in a pre-test and had wide variations in their knowledge of English) provided the extracts discussed below. Extracts 1 and 3 involved a sub-group made up of students H, C and A, while Extract 2 was taken from a sub-group comprised of students A, P, G and C.

Figure 1. Screen image of a Lyceum brainstorming session

mnl2

Sticky Note

This line looks as though it needs some spaces creating between the words.



THE DATA: THREE MULTIMODAL CONVERSATIONS

The three extracts occur within the last seven minutes of the last session in the course. Tables 1 and 2 give simplified versions of a tabular transcription of the screen videos. Reading the top row of Table 1 from left to right, the tran-scripts show: participants’ initials (User), the speech turns (T, characterising all inputs, rather than simply those in ‘speech’), chronological

sequences (Time) and data collected from the four modalities (in the four rightmost columns).

When the first extract begins, the learners are engaged in an oral conversation. Three students, H, C and A are negotiating an agreed statement about what they think they learned during the course. The statement then needs to be typed on the shared screen for later presen-tation to the plenary group. In Extract 1 (Table 1), H twice tries to pronounce the phrase tech-nical vocabulary (at T1 and T7) but stutters and hesitates each time. As C attempts to help H by

Figure 2. Screen image of Lyceum turn-taking tools

mnl2

Sticky Note

Create a space between 'characterising' and 'all', and between 'all' and 'inputs'.



typing vocabulary learning in the shared document (T3) and to elicit H’s approval of this formulation (T5), H holds to his original word-ing by writing it in the chat window (T6). This results in C modifying (T8) what he had typed on the shared document, which H finally ap-proves (T9) by clicking the Tick icon. Arrows show topic-maintenance by H (in other words compliance with the CA notion of conditional relevance). To compensate for his articulatory problems in English, H deploys an alternative conversational strategy that takes advantage of the environment’s multimodality. His strategy cannot be made visible through conversation analysis conducted in its classical form since CA relies on the sequentiality principle, and the conversation, if read vertically down the second column of Table 1, has no sequencing that could be sensibly interpreted. Sequential-ity is in fact present, but can only be detected by analysing the four rightmost columns, which provide an understanding of phenomena across modalities.

Just before the start of Extract 2, the tutor had launched the conversation by asking the members of the sub-group (A, P, G and C) to introduce themselves. The extract in Table 2 shows the conversation proceeding orally (shaded in grey), with related input typed into the text-chat window (shaded in grey and sur-rounded by a frame), while two of the learners (A and P) carry out an unrelated conversation, concerned with an auditory difficulty, in the text-chat window (unshaded). In the recording which was used for the transcription, A speaks quite audibly but, possibly due to his home setup or for server-related reasons, P does not hear him. So A and P run a conversation in parallel with the main conversation initiated by the tu-tor. They use chat to construct a dialogue on a different theme, without apparently disturbing the ‘main’ conversation which is proceeding via the audio channel. This strategy may be seen as non-transgressive in terms of saving the tutor’s face. In a face-to-face classroom, holding a parallel conversation while the

Table 1. Re-interpreting sequentiality (Extract 1)

User T Time Audiotranscript Chatlog Tick Doc

H 1

28:17 vocal er bulaire er tech-nical vocabulaire je sais pas comment on dit

C 2 28:25 vocabula vocabulary <laughter>

C 3 28:28 C writes: vocabu-lary learning

A 4 28:33 er ah yes maybe er

C 5 28:36 do you think it’s er ok, H?

H 6 28:46 technical vocabulary

H 7

28:50 er technical er vo-cabulary er technical vocabulary

C

8

28:51 C deletes previous and inserts: to learn technical vocabulary

H 9 29:02 √

mnl2

Sticky Note

Provide spaces between the words of the preceding line.



teacher is talking can easily be construed as face-threatening for the tutor, who may take steps to stop it, should it persist for longer than a few seconds. The face-saving principle helps explain A and P’s conversational moves across modalities, i.e., out of the audio channel and into the small space dedicated to text-chatting. The example shows that the face-saving principle of CA needs to be re-interpreted for the online platform environment.

In the last example, Figure 3, we see how learner C uses the four modalities at his disposal: audio, text-chat, Tick icon and shared document.

The chart in Figure 3 highlights discrepan-cies in the ways each participant (H, C and A) uses these four modalities: the horizontal axis shows the modalities available while the verti-cal axis gives the number of speech turns per individual. Learner C’s inputs are mainly via the audio channel (45 audio speech turns, 4 text-chat turns). In the sub-group, C has the highest level of proficiency, as evidenced by his score in the pre-experiment test and by his ease with English structures such as ‘would you repeat?’, ‘we must choose’, ‘we can’t answer’, ‘do you want to add something?’, ‘don’t you think so?’ or ‘are you OK with what I write?’.

Table 2. Re-interpreting the face-saving principle (Extract 2)

User T Time Audiotranscript Text-chat

A 1 30:36 Er I do my presentation er my name is er A donc I come from er I live er at Sète er a town near Montpellier and er I come from er west [inau-dible] but I er I was born in Toulon er in a town er south er of France

P 2 30:57 A, I don’t read* you [P uses this verb to mean ‘hear’ perhaps influenced by war film cliché]

Tutor 3 31:13 ok thanks er A er so C is C … if C is there would you like to talk a bit about yourself C?

[…] C answers Tutor’s question (in audio) at length […]

A 4 36:14 P, I don’t write

P 5 36:59 no, A, you need to put the volume of your microphone more hight

C 6 37:08 er yes <laughter>

P 7 37:12 I think…

Tutor 8 37:14 ok and do they install they install motors as well as in in in er in houses or that kind of thing?

G 9 37:45 I don’t understand

C 10 37:49 can you repeat your question please?

P 11 37:50 me too

A 12 39:41 sorry, P, but I can’t

Tutor 13 39:54 sorry did I did you say fourteen to seventeen years old?

P 14 40:03 ok ok



Content analysis of his input shows that in over half his spoken turns (28 out of 44) he is asking for others’ opinions. Yet all his text-chat inputs are language accuracy checks. Additionally, C makes much more use of the shared document than do his two colleagues. In data collected from the parts of the session that precede and follow Extract 3, 90% of C’s turns in the shared document are directly preceded by a turn else-where (mostly in the audio channel) seeking confirmation of the group’s approval of what he has written.

We hypothesize that C uses the text-chat facility in order to check the accuracy of his English rather than to progress the conversa-tional agenda of the class, which requires that the trio reach a consensual statement. He prefers to move through the class agenda by using the shared document, but only after obtaining consensus from his peers orally. This pattern of use could relate to two converging factors: C may be specialising particular conversational aims to particular spaces on the screen. His representation of himself as a communicator may also play a role in his choice of modality.

In order to interpret his modal preferences and their effect on the multimodal conversation,

analysts of his strategies would need to assess the degree of visual salience of the text-chat window in contrast to the aural salience (in-trusiveness) of the sound of his voice coming through each group member’s headset. As for self-representation, we may posit that face-saving issues are involved: for example C might be prepared to give an image of himself as a confident English-language speaker in the audio channel while utilising the chat window more specifically for risk-taking face-threatening activities, such as asking for help with English forms. A similar explanation might be offered for his use of the shared document: the visual organisation of the screen when that document is uploaded to it (as seen in Figure 1), the central position of that document and its status as the ‘official’ record of the group’s collaboratively negotiated view may explain both C’s self-appointed guardianship of it and his diffidence in committing material to it, unsupported by his peers. However, CA alone offers no way of accounting for physical perceptions, or of factors such as visual or aural salience. A wider conceptual framework is therefore needed.

Figure 3. Use of modalities by participants in Extract 3



DISCUSSION

On the basis of the findings from the extracts presented earlier we suggest that while CA remains a useful approach to the understanding of sense-making in real-time online multimodal settings, it needs to be rearticulated. Our sug-gestion is that such reframing can be done in the light of three theoretical frameworks draw-ing from affordance theory, multimodal social semiotics and geosemiotics respectively. Our reading of the data showed that two dimen-sions of traditional CA, sequentiality (Extract 1) and face management (Extract 2) could be used to characterise conversational moves within electronically-mediated multimodal conversations, albeit redefined, because when more than one modality was available, both sequentiality and face management operated across modalities. Hutchby’s (2001) work is, we suggest, relevant in this respect.

CA Re-Interpreted in the Light of Communicative Affordances

Hutchby (2001) reminds us through the example of the early history of telephone communication that technology is in a reciprocal relationship with its users. The telephone was originally sold as an instrument for transacting business or for exchanging practical information. Early marketing stressed this functional use to the extent of advising subscribers to wait until late at night if they really needed to use the machine for personal chat. However users were not persuaded to follow these commercial exhortations and they soon appropriated the telephone as a medium for family and other intimate conversations, forcing the telephone companies to review their marketing strategies.

“[...] While designers may be said to have some control over the features they design into an artefact, and while they may have some idea about the range of uses to which the artefact should be put, they have little control over the artefact’s communicative affordances – over the range of things it turns out to enable people to do” (Hutchby, p. 123).

We witnessed this mechanism at work in the project extracts, where learners appropriated modalities in diverse ways. For example the text-chat was re-appropriated in the contexts of different conversations in order to support control over content (Extract 1), to provide technical assistance (Extract 2) and to enact face management (Extract 3).

We also observed that individual partici-pants engaged with different communicative affordances to satisfy identical communica-tive needs. For example legitimation of self as a turn-taker (and therefore control over the conversation) was achieved via moves from audio to text-chat then to the Tick icon by H (in Extract 1), but for C (in Extract 3) by tog-gling between the audio channel and the shared document. The construct of communicative affordance – as described by Hutchby in the last line of the quotation above – also helps to understand the different ways in which hu-man actors in different technological settings solve a single problem such as (to stay with the example of the telephone) answering a landline call, a mobile phone or a computer bleep from an Internet-telephony system. The same conversational problem is involved in each case (how to respond to a conversational invitation from a remote caller), but differ-ent discursive solutions are appropriate. For instance, a person called on a landline without user ID will in most ordinary circumstances pick up the handset and initiate their side of the conversation by uttering a conventional query (e.g., hello, allô, pronto) in a rising tone followed by a pause, in the expectation that the caller will identify themselves. With user ID, or in Internet telephony where the caller’s name and sometimes photograph appears on screen, the person called is likely to respond instead with a greeting usually followed by the caller’s name. A pause, if provided by the person called, will not be heard by the caller as an invitation to identify him/herself; instead it will require interpretation and negotiation so that the conversation may progress.

Thus CA continues to be central to un-derstanding conversations in electronically-



mediated multimodal settings. However only by paying attention to communicative affordances will we be able to do the necessary work of re-interpreting its principles in terms of their manifestations and discursive adaptations across modalities.

The Influence of Platform Design on Discourse in Multimodal Environments

In our view, the materiality of the environment influences the dynamics of conversations. For example in Extract 2, with two conversations progressing in parallel, it is not accidental that the tutor-initiated one was carried out orally while the students’ discussion of sound prob-lems was conducted in text-chat: all we need to do to become persuaded of this is to mentally invert the modal choices and imagine that the tutor led his tutorial via postings in the text-chat while students talked about other topics in the audio channel. It is unlikely that the group would accept such a position for the tutor, and we draw from multimodal social semiotics to help explain why.

In the coming section, the discussion of the semiotics of screen displays is based on an analogy with Kress and van Leeuwen’s (2001) work on the semiotics of multimodal pages in newspapers and books. These authors identify four dimensions in the potential for meaning-making of such artefacts (discourse, production, dissemination and design), of which design is particularly relevant to the current study. The designers of the platform in our project can perhaps be compared with the architect used by Kress and van Leeuwen to explain the relation-ship between design and discourse:

“An architect, for instance, designs (but does not build) a house or a block of apartments. The discourse provides a certain view of how houses are lived in the way they do, and argu-ments which critique or defend this way of life. The design of the house then conceptualises how to give shape to this discourse in the form of a house, or a type of apartment” (Kress & Leeuwen, 2001, p. 6).

The relevance of this architectural example will become clear as we look at two very different designs of environments that have been used for language learning, Lyceum and Traveler (DiPaola, 1997), through comments made by learners. Lyceum (Figure 4), as we have seen, is an academic tool designed to look like a university campus building, which is a traditional design choice for electronic learning environments as pointed out by Bayne (2008). In contrast, Traveler (DiPaola, 1997) is an avatar-based system with a ‘fantasy’ feel inherited from the world of games, as shown in Figure 5 further down.

Just as the architect has provided a shape for the cultural discourses of human habitats, so, we would argue, the designers of elec-tronic environments contribute to determining the kind of discourse that users deploy there. For example here are responses from two of our users (for Lyceum) and two of Örnberg Berglund’s (2006) users (for Traveler), on be-ing asked about their feelings of ‘presence’ when online:

• Lyceum user 1: ‘Quand le prof rentre dans la salle, cela ne dérange pas. Je sais pas comment l’expliquer’ (When the teacher enters the room, it’s not intrusive. I don’t know how to explain it).

• Lyceum user 2: ‘Le style du prof joue, mais le fait qu’il est invisible, il ne peut pas s’imposer de la même façon qu’en présentiel’ (The teacher’s style is a factor but the fact that he’s invisible, he can’t impose himself in the same way as in face-to-face).

• Traveler user 1: ‘It took me to another world and was a real adrenaline buzz. It was on my screen and I was conscious of it always, but I was definitely virtually gone from my usual habitat’.

• Traveler user 2: ‘I’m always immersed.[…] It doesn’t matter that the environment is artificial. […] I think of the place as real’.

Whereas these two Traveler users produce a discourse of emotions and escapism, the



discourse of Lyceum users reflects school-like representations of a particular type: teacher-led classes. We make two comments here. Firstly, although these differences in perception may not be surprising given the strongly contrasted visual identities of Figures 4 and 5, the question is whether two groups using these environ-ments for achieving the same language-learning objectives might have very different types of conversations in each environment. The second observation relates to designers and the uses made of their designs. Lyceum’s design was underpinned by a democratic and participa-tive pedagogical posture: “We have imposed minimal technical constraints on ‘floor control’: anyone can speak anytime” (Buckingham-Shum, Marshall, Brier, & Evans, 2001, p. 4). Yet the users’ comments show a preoccupation with teacher control. It is likely that this is part of their pre-existing non-virtual educational culture, in which case the question can be asked: to what extent and in what ways can the design features of interactive learning environments transform the users’ representations of self? The answer to this question is another deter-miner of sense-making in these environments. Finally, regarding the device which introduced

this section, i.e., the proposal that the Copéas tutor could conduct core tutorial business in the chat box while the students conversed orally, evidence from Lyceum users’ perceptions sup-ports the view that the system’s design provides a shape for the cultural discourses of traditional teacher-centered classrooms. But based on multimodal social semiotics’ understanding of design, there is no in-principle reason why other types of design could not work to support other cultural discourses, producing distinct types of conversations.

The Role of the Body in Relation to Space in Multimodal Environments

While Scollon and Scollon (2003) acknowledge the importance of Kress et al.’ s (2001) design dimension, they re-interpret it in order to focus on the interactants’ bodies as resources for meaning-making, relating these resources to the use of the spaces within which the interac-tants interact. Although Scollon et al.’s (2003) insights were drawn from offline situations, for example observations of groups of people in public places, we have found their accounts useful for understanding the way that our online

Figure 4. The design of a Lyceum screen, with whiteboard displaying learners’ work



users perform conversations by choosing among the different spaces offered to them within the interface of the virtual environment.

Scollon et al. (2003) conceptualise the spatial dimension of social semiotic resources by presenting a framework which they call ‘geosemiotics’and which they define as follows:

“Geosemiotics is the study of meaning systems by which language is located in the material world. This includes not just the location of words on the page you are reading now but also the location of the book in your hands and your location as you stand or sit reading this” (Scollon et al., 2003, pp. x-xi)

The authors structure geosemiotics into three sub-sets: the interaction order, visual semiotics (on which we will not elaborate here, as this concept comes close to Kress et al.’s (2001) notion of design mentioned earlier) and space semiotics.

The interaction order provides a construct for understanding how individuals perceive the interactional value of the space they choose to use. In their description of the interaction order, the authors include perceptual spaces and interpersonal distances. Salient perceptual spaces are visual and auditory spaces (while ‘less

noticed’ ones are respectively olfactory, thermal and tactile). The addition of the construct of interpersonal distances – as a scale of values inspired by Hall’s (1969) work on proxemics – allows geosemiotics to ask questions about the relationship between space, sound, bodies and socialisation. For example, the auditory space which I perceive, and my perceived intimacy or distance with the body of the individual vo-calising the sound that I am hearing, together create the semiotic resource through which I embody meanings. Applying this framework to Copéas participants, in particular to the parallel conversation mechanism in Extract 2 and to the multimodal preferences of the learner in Extract 3, the question becomes: how do participants co-construct interpersonal values (intimate, per-sonal, social, public) into conversations which proceed simultaneously through visual spaces of varying salience and through an auditory space defined by the spatially and tactilely intimate device of an earpiece or headset?

Space semiotics, in Scollon et al.’s (2003) words, is the most fundamental part of geose-miotics, because it asks where in the world the sign or image is located and because it aims to account for “any aspect of the meaning that is predicated on the placement of the sign in the material world” (Scollon et al., 2003, p. 146,

Figure 5. The design of a Traveler screen



emphasis added). In terms of multimodal elec-tronic environments, space semiotics provides the basis for asking questions such as: how do users decode and encode meanings in a material situation involving their computer and its vari-ous peripherals (keyboard, mouse or keypad, webcam) as well as other stimuli around them (possibly another computer, a video screen or a person physically present in the vicinity)?

LIMITATIONS AND CONCLUSION

The data collection method and the framework chosen for the present analysis brought limita-tions. A limitation in data collection was the fact that for budgetary reasons it was impos-sible to provide every participant with screen capture software. Thus not only was some linguistic data lost (for example hesitations and auto-corrections before sending a finalised chat post), but also lost to the researcher were cursor movement (i.e., data on learner use of the spaces on screen), and traces of strategies such as scrolling up the chat history to find conversational resources. The latter has a direct bearing on CA, since as Smith (2007) observes, ‘if scrolling detracts from a learner’s attention to incoming messages, then our interpretation of why, for example, a learner “missed” some salient information about a topic, lexical item, etc., might be different […]’ (Smith, 2007, p. 73, original emphasis).

One of the limitations of the chosen frame-work, social semiotics, is that it leaves aside an aspect that has been well-developed within Grounding theory: the notion of the cognitive and communicative ‘cost’ of transactions in interaction (Örnberg Berglund, 2009, p.188). Just as Örnberg Berglund signals different types of cost incurred in the environment that she studies (FlashMeeting), so in the platforms in our work, avoidance of some modalities may be related to the cognitive and communicative costs incurred by users as they try to orchestrate them. Including a model of the costs involved in

multimodal meaning-making may be of interest in future analyses.

Two considerations should be offered in conclusion, one reflective and the other concerned with application. Firstly, due to the complexity of the phenomena under study and the pluridisciplinary nature of the analysis, we have only touched on principles for a methodol-ogy. A clear direction for further work, then, is to complement and operationalise these principles. Secondly, we are aware of use of social semiotics for researching some online environments, such as the semiotics of multiplayer immersion games (Lemke, 2006), and of social tagging (Huang & Chuang, 2009), but not for investigating the multimodal learning platforms at the core of the current study. Similarly, in the educational field, although Jewitt and Kress (2003) have used social semiotics as part of their work in a field sometimes called ‘multimedia semiot-ics’, and have sought to lay the foundation for an understanding of learning via multimodal texts in educational software (Jewitt, 2004), neither on-screen interaction nor the effects of synchronicity of communication are part of such work to date.

As a response to both the reflective and the application dimensions just mentioned, we see a useful research agenda emerging: to test out, with a large volume of learner interaction data collected from different platforms, in set-tings both educational and non-educational, the methodological claims made in this paper according to which such data can be better understood through the synergistic use of con-versation analysis, multimodal social semiotics and geosemiotics.

ACKNOWLEDGMENT

We thank Thierry Chanier, the Université de Franche-Comté and the Ministère de la Recherche (France) for their support in the Copéas project.



REFERENCES

Bayne, S. (2008). Higher education as a visual prac-tice: Seeing through the virtual learning environment. Teaching in Higher Education, 13(4), 395–410. doi:10.1080/13562510802169665

Buckingham-Shum, S., Marshall, S., & Evans, T. (2001). Lyceum: Internet voice groupware for distance learning. In Proceedings of the 1st First European Conference on Computer-Supported Col-laborative Learning, Maastricht, The Netherlands. Retrieved from http://kmi.open.ac.uk/publications/papers/kmi-tr-100.pdf

Chapelle, C. (2004). Technology and second language learning: Expanding methods and agendas. System, 32, 593–601. doi:10.1016/j.system.2004.09.014

Cook, V. (1991). Second language teaching and language learning (2nd ed.). London, UK: Arnold.

DiPaola, S. (1997). Virtual worlds. Retrieved Feb-ruary 5, 2011, from http://www.dipaola.org/steve/vworlds.html

Garcia, A., & Jacobs, J. (1999). The eyes of the beholder: Understanding the turn-taking system in quasi-synchronous computer-mediated communica-tion. Research on Language and Social Interaction, 32(4), 337–367. doi:10.1207/S15327973rls3204_2

Gass, S., Mackey, A., & Pica, T. (1998). The role of input and interaction in second language acqui-sition. Modern Language Journal, 82, 299–307. doi:10.1111/j.1540-4781.1998.tb01206.x

Gass, S., & Varonis, E. (1994). Input, interaction, and second language production. Studies in Second Language Acquisition, 16(3), 283–302. doi:10.1017/S0272263100013097

Goffman, E. (1967). Interaction rituals: Essays on face-to-face behavior. New-York, NY: Random House.

Hall, E. (1969). The hidden dimension. New York, NY: Doubleday.

Hampel, R. (2006). Rethinking task design in the digi-tal age: A framework for language teaching and learn-ing in a synchronous online environment. ReCALL, 18(1), 105–121. doi:10.1017/S0958344006000711

Hampel, R., Felix, U., Hauck, M., & Coleman, J. (2005). Complexities of learning and teaching lan-guages in a real-time audio-graphic environment. German as a Foreign Language Journal, 3, 1–30.

Hauck, M., & Hurd, S. (2005). Exploring the link be-tween language anxiety and learner self-management in open language learning contexts. European Journal of Open, Distance and E-Learning, 2. Retrieved Feb-ruary 5, 2011, from http://www.eurodl.org/materials/contrib/2005/Mirjam_Hauck.htm

Hauck, M., & Youngs, B. L. (2008). Telecollabora-tion in multimodal environments: The impact on task design and learner interaction. Computer Assisted Language Learning, 21(2), 87–124. doi:10.1080/09588220801943510

Helm, F., & Guth, S. (2010). The multifarious goals of telecollaboration 2.0 . In Guth, S., & Helm, F. (Eds.), Telecollaboration 2.0 languages, literacies and intercultural learning in the 21st century (pp. 69–106). Bern, Switzerland: Peter Lang.

Huang, A. W., & Chuang, T. R. (2009). Social tagging, online communication, and Peircean semiotics: A con-ceptual framework. Journal of Information Science, 35(3), 340–357. doi:10.1177/0165551508099606

Hutchby, I. (2001). Conversation and technology: From the telephone to the internet. Cambridge, UK: Polity.

Jewitt, C. (2004). Multimodality and new commu-nication technologies . In LeVine, P., & Scollon, R. (Eds.), Discourse and technology: Multimodal discourse analysis (pp. 184–195). Washington, DC: Georgetown University Press.

Jewitt, C., & Kress, G. (2003). Multimodal literacy. Bern, Switzerland: Peter Lang.

Karabulut, A., & Correia, A. (2008). Skype, Ellumi-nate, Adobe Connect, Ivisit: A comparison of web-based video conferencing systems for learning and teaching. In Proceedings of the Society for Informa-tion Technology and Teacher Education International Conference (pp. 481-484). Chesapeake, VA: AACE.

Kress, G. (2003). Literacy in the new media age. Lon-don, UK: Routledge. doi:10.4324/9780203164754

Kress, G., & Van Leeuwen, T. (2001). Multimodal discourse: The modes and media of contemporary communication. London, UK: Arnold.

Lamy, M.-N. (2004). Oral conversations online: Redefining oral competence in synchronous envi-ronments. ReCALL, 16(2), 520–538. doi:10.1017/S095834400400182X

Lamy, M.-N., & Flewitt, R. (in press). The multi-modal mediation of semiotic resources in an online conversation . In Develotte, C., Kern, R., & Lamy, M.-N. (Eds.), Décrire la communication en ligne: le face à face distanciel. Lyon, France: ENS Editions.

mnl2

Sticky Note

Lamy and Flewitt has now been published and should bear the date (2011).



Lamy, M.-N., & Hampel, R. (2007). Online communication in language learning and teach-ing. Basingstoke, UK: Palgrave Macmillan. doi:10.1057/9780230592681

Lemke, J. (2006). Towards critical multimedia literacy . In McKenna, M., Labbo, L., Kieffer, R., & Reinking, D. (Eds.), International handbook of literacy and technology (Vol. 2, pp. 7–14). Mahwah, NJ: Laurence Erlbaum.

Long, M. (1983). Linguistic and conversational ad-justments to non-native speakers. Studies in Second Language Acquisition, 5(2), 177–193. doi:10.1017/S0272263100004848

Mazur, J. M. (2004). Conversation analysis for educa-tional technologists: Theoretical and methodological issues for researching the structures, processes, and meaning of on-line talk . In Jonassen, D. H. (Ed.), Handbook of research on educational communica-tions and technology (pp. 1073–1098). Mahwah, NJ: Laurence Erlbaum.

Mondada, L. (2004). Téléchirurgie et nouvelles pra-tiques professionnelles: Les enjeux interactionnels d’opérations chirurgicales réalisées par visiocon-férence. Sciences Sociales et Sante, 22(1), 95–126. doi:10.3406/sosan.2004.1610

Negretti, R. (1999). Web-based activities and SLA: A conversation analysis research approach. Language Learning and Technology, 3(1), 75-87. Retrieved February 5, 2011, from http://llt.msu.edu/vol3num1/negretti/

O’Dowd, R., & Ritter, M. (2006). Understanding and working with ‘failed communication’ in telecollabo-rative exchanges. CALICO Journal, 23(3), 623–642.

Örnberg Berglund, T. (2006). Multimodality in a three-dimensional voice chat. In Proceedings of the 2nd Nordic Conference on Multimodal Communica-tion, Papers in Theoretical Linguistics, Göteborg, Sweden (pp. 303-316).

Örnberg Berglund, T. (2009). Multimodal student in-teraction online: An ecological perspective. ReCALL, 21(2), 186–205. doi:10.1017/S0958344009000184

Payne, J. S., & Ross, B. M. (2005). Synchronous CMC, working memory, and L2 oral proficiency development. Language Learning and Technology, 9(3), 35-54. Retrieved February 5, 2011, from http://llt.msu.edu/vol9num3/payne/default.html

Payne, J. S., & Whitney, P. J. (2002). Developing L2 oral proficiency through synchronous CMC: Output, working memory, and interlanguage development. CALICO Journal, 20(1), 7–32.

Reffay, C., & Chanier, T. (2003). How social network analysis can help to measure cohesion in collaborative distance learning. In Proceedings of the International Conference on Computer Support for Collaborative Learning (pp. 343-352). Dordrecht, The Netherlands: Kluwer Academic.

Rosell-Aguilar, F. (2005). Task design for audio-graphic conferencing: promoting beginner oral interaction in distance language learning. Com-puter Assisted Language Learning, 18(5), 417–442. doi:10.1080/09588220500442772

Sacks, H., Schegloff, E., & Jefferson, G. (1974). A simplex systematics for the organisation of turn-taking for conversation. Language, 50, 696–735. doi:10.2307/412243

Scollon, R., & Scollon, S. W. (2003). Discourses in place: Language in the material world. London, UK: Routledge. doi:10.4324/9780203422724

Smith, B. (2007). Theoretical and methodologi-cal considerations in applied linguistics research on computer-mediated communication. Studies in Languages and Language Teaching Journal, 14(1), 67–78.

Smith, B. (2008). Methodological hurdles in captur-ing CMC data: The case of the missing self-repair. Language Learning and Technology, 12(1), 85-103. Retrieved February 5, 2011, from http://llt.msu.edu/vol12num1/smith/default.html

Tudini, V. (2003). Conversational elements of online chatting: speaking practice for distance language learners? Apprentissage des Langues et Systèmes d’Information et de Communication, 6(2), 83–99.

Vetter, A., & Chanier, T. (2006). Supporting oral production for professional purposes in synchronous communication with heterogenous learners. ReCALL, 18(1), 5–24. doi:10.1017/S0958344006000218

Wooffitt, R. (2005). Conversation analysis and discourse analysis: A comparative and critical introduction. London, UK: Sage.

Xiao, M. (2007). An empirical study of using Internet-based desktop videoconferencing in an EFL setting. Unpublished doctoral dissertation, University of Ohio, Athens, OH.



Marie-Noelle Lamy is professor of distance language learning at the UK Open University. She has 15 years of experience in designing and implementing languages courses for interactive online study, involving extensive use of e-tutorials, text-based as well as voice-based. She has researched extensively in the field of computer-mediated communication for language learn-ing, with a particular interest in real-time conversations in multimodal settings. Her current interests are investigating methodologies for the description of such conversations, researching co-construction of cultures by groups of culturally diverse learners in multimodal online environ-ments, and investigating the potential of informal language learning via social networks online. Recent publications include a co-authored book, Online communication in language learning and teaching (2007) and two co-edited collections, Learning cultures in online education (2009) and Décrire la communication en ligne: le face-a-face distanciel (describing online communication: face to face at a distance, forthcoming).



Keywords: Blackboard, E-Learning, E-Portfolio, Effective Practice, Frameworks, Guidelines, Learning Management System (LMS), Personal Development Plans, Web 2.0

1. INTRODUCTION

Not all improvements in technology equate to significant advances in educational outcomes as was the case with the introduction of elec-tronic learning (e-learning) in relation to the implementation of educational policies and effective practice (Weaver, 2002). The insti-tutional infrastructure of tertiary educational establishments provide the support needed for

this new form of learning to function, as well as, access to technologies which can in many cases be beyond the control of the teacher and is dependent on a wider range of services than courses that do not use e-Learning (Deepwell, 2007). Further and higher educational institu-tions are now providing more support for this type of learning as well as more training for teaching staff who are required to engage with new teaching technologies.

With the government set to adopt the wide spread implementation of Personal Develop-

The Development of E-Portfolio Evaluation Criteria

and Application to the Blackboard LMS E-Portfolio

Gary F. McKenna, University of the West of Scotland, UK

Mark H. Stansfield, University of the West of Scotland, UK

ABSTRACTThe purpose of this paper is to develop e-portfolio evaluation criteria which will be used to review the Blackboard LMS e-portfolio being used at one Higher Education (HE) institution in the UK as evaluation criteria for reviewing e-portfolio provision does not exist in the literature. The approach taken was to initiate a wide literature search which involved reviewing over 600 articles by their abstract dating from 1995 to 2010. The findings show that little has been written about the development of e-portfolio effective practice frameworks. Therefore e-learning effective practice frameworks were used as a basis from which to design and develop an e-portfolio evaluation framework and then apply it to the university case which uses a Blackboard e-portfolio to support Personal Development Plans. The research provides a starting-point for further research into the development of robust e-portfolio evaluation models and frameworks.

DOI: 10.4018/jvple.2012010102



ment Plans (PDP) using e-Portfolios as a means to fulfill the 2010/11 aim of every student in the UK having a PDP e-portfolio as part of the criteria for the completion of a certificate, diploma, or higher degree and in preparation to replace the honours classification (Burgess, 2007, p. 9). It is crucial that educators, institu-tions, and other stakeholders have evidence in relation to the performance of e-portfolios as a learning medium. This paper will review the available literature of e-portfolio success within education to: (1) identify which aspects of e-portfolios have been evaluated as a learn-ing tool; (2) describe the evaluation strategies used; (3) synthesise the findings; (4) present an example of an e-portfolio project conducted in a higher education institution within the UK; and (5) discuss educational implications and future research directions.

The research has highlighted the need for an e-portfolio effective practice framework based on the findings evidenced in the evaluation of an e-portfolio project for a higher educational institution within the UK. As well as evaluat-ing the existing Learning Management System (LMS) from which the e-portfolio was situated the evaluation also involved building a Personal Development Planning (PDP) portfolio using the postgraduate templates provided by the institution, as well as linking the institutional PDP e-portfolio to external Web 2.0 social net-working sites such as Bebo, MySpace, and Wix (http://wix.codeplex.com/). This evaluation report identifies a need to develop an evalua-tion LMS e-portfolio framework based on the findings of other studies.

This study will be of interest to educational policy and programme developers, teaching staff, e-Learning and e-portfolio developers, faculty heads, Continuing Professional Devel-opment (CPD) and Lifelong Learning partners. The evaluation framework developed for this study was adapted from previous studies of e-learning best practices and virtual campus review frameworks. The adapted model was assembled for the purposes of improving moni-toring and development processes to enhance the overall quality assurance of e-portfolios.

2. BACKGROUND TO THE CASE STUDY

This review evaluated e-learning system criteria at a higher educational institution in the UK, their initiatives for implementing and facilitat-ing Personal Development Plans (PDP), and the Blackboard Learning Management System (LMS). A literature search was conducted to ascertain an evaluation framework that could be used to review the institution’s provision for LMS e-portfolio templates. This was because at the time of conducting the literature review there appeared to be a lack of evaluation cri-teria for assessing e-portfolios. As a result of this it was decided to review the e-learning literature as a basis for identifying e-portfolio assessment criteria that could be incorporated into the development of an e-portfolio evalua-tion framework. An e-learning framework was adapted from a previous study carried out by Stansfield and Connolly (2009) which has been recently developed and implemented in a multi HE campus environment and deemed suitable for our purposes.

3. HISTORY OF E-PORTFOLIO EVALUATION IN HIGHER EDUCATION

Electronic portfolios (e-portfolios) were first reported to assist in learner development in further and higher education in the late 1980s (Schwartz & Bridwell-Bowles, 1987). In rela-tion to the benefits of using portfolios Armitage (1988, p. 16) affirmed that digital-portfolios can help students learn more efficiently due to increasing levels of motivation and facilitating the publication of their work, which in turn has led to higher levels of self-confidence. Other supporters of e-Portfolios (Kimball, 2005; Hatzipanagos & Lygo-Baker, 2008) claim that these methods improve on traditional paper-based methods of reflection due to the learner having: (1) more control over content; (2) attained higher levels of reasoning and self-



efficacy; and (3) improved learner cognition in relation to the practice of reflection.

Recently interest in learning using e-Portfolios has risen across the sector within tertiary education in the UK driven by national policy and initiatives to improve intellectual growth and personal development (JISC, 2007). Advances in Web 2.0 provide new ways for learners to express themselves online using social networking applications which allow them to document their progress. Educational institutions are using PDP e-portfolios (Strivens, 2007, p. 3) to encourage students to record and plot their personal development over the course of their stay at university and to carry out this practice beyond graduation. Other methods used that are less prescriptive include, but are not restricted to, using social networking web applications such as a blog, wiki, personal website, podcast, videocast, animation, poll, quiz, and RSS feeds (McGee & Diaz, 2007; Richardson, 2008). These can be used to com-municate with other like-minded learners with similar interests and can include features to link and share e-portfolio information or to cross-reference resources. Zhang et al., (2007) claim that advances in the technological capability of Web 2.0 applications can help to improve and support both the practice of constructivist and reflective learning where the learner attempts to: (1) deconstruct didactic instruction from lectures and classroom tutorials; (2) reconstruct the information and build their own knowledge base; (3) evidence, reflect, and document learned theory to make connections with their own lived experience; and (4) make sense of theory and practice to create new knowledge which is unique to each learner.

4. THE DEVELOPMENT OF AN EVALUATION FRAMEWORK FOR REVIEWING E-PORTFOLIO AND LEARNING MANAGEMENT SYSTEMS

In order to develop an evaluative framework the authors reviewed the literature to gain an

understanding of the types of models and guide-lines that have been successfully implemented in other institutions to evaluate e-portfolio initiatives. As few studies were found the deci-sion was made to look at e-Learning evalua-tive frameworks and guidelines that could be used for synthesising a model for evaluating a blackboard LMS and e-portfolio system. The following two subsections discuss the research method and search criteria used for searching and categorising the frameworks and guidelines found in the literature.

4.1. Method Used for Literature-Based Investigation

As part of the initial work carried out on the evaluation of a higher educational institution’s PDP e-portfolio project, a literature-based in-vestigation was conducted into previous studies which made use of evaluation frameworks for evaluation purposes. The investigation involved examining papers, reports, and web-based content. From the evaluative frameworks that were reviewed during the literature search the most applicable were Wright (2003) and Britain and Liber (2004). We used literature review methodologies to categorise the ab-stracts from articles according to relevance and dependability – principally in relation to their effectiveness for informing decisions about developing an evaluation framework for use with the Blackboard LMS and PDP e-portfolio learning systems (Strauss & Corbin, 1998; Brit-ten, 2002; Gough, 2007).

Articles were selected by their abstracts and the two authors independently applied the criteria to categorise each article as descriptive or evaluative. If an abstract was not available or sufficient information could not be found to make a judgment the article was reviewed. If both authors were in disagreement a third opin-ion was provided by a colleague. The e-portfolio articles were categorised by evaluation domain. The literature was checked against our criteria for accessing the relevance of the evaluation methods being examined. Where applicable we divided studies into groups where the evaluative



features were specific to educational content and good design principles relevant to a Learning Management System (LMS).

4.2. The Literature Search

The literature search identified existing key frameworks of evaluation and the attributes that can be measured within an e-Learning environment. The adapted e-Learning evalu-ation framework was then used to identify what can potentially be evaluated in an LMS and e-portfolio system. The process involved searching 11 databases for evaluation criteria which are listed as follows:

(1) Emerald;(2) ACM;(3) Springerlink;(4) Science Direct;(5) EEBSCO;(6) Eric;(7) Taylor and Francis;(8) Expanded Academic ASP;(9) ISI Proceedings;(10) Sage;(11) Zetoc.

Relevant e-Learning evaluation frame-works from journal papers dating from the year 2000 were searched using a mixture of key search terms. The key search terms used consisted of keywords and phrases such as: “criteria” OR “evaluation” OR “framework” OR “guidelines”] AND [“e-Learning” OR “eLearning” OR “online courses” OR “digital learning” OR “online education”.

The search returned over 600 articles out of which 14 were found to be appropriate for our search criteria, that is e-Learning evalua-tion guidelines and evaluation frameworks for e-Learning. The literature search yielded results which we have used to inform our own criteria for an evaluation framework for evaluating a Blackboards LMS and e-portfolio system.

4.3. Previous Studies that Informed our Framework

From the literature search 10 areas were identi-fied as being important for measuring the ef-fectiveness of an LMS e-portfolio system. The 10 areas recognised as suiting our needs are: (1) the general information given to learners prior to commencement of a course (Wright, 2003; QAA, 2004); (2) accessibility of course materials (Ball & Sutherland, 2003; CATEA, 2006); (3) course organisation (Wright, 2003); (4) course language (ANSI, 2004); (5) course layout (ANSI, 2004); (6) course goals and objec-tives (Mager, 1975; Guilbert, 1984; Gallagher & Smith, 1989; Harrison, 1999); (7) course content (Wright, 2003); (8) learning strategies and opportunities for practice and transfer (Hill-esheim, 1998; Ragan, 1999; Marshall, 2005); (9) learning resources (Marshall & Mitchell, 2003; Wright, 2003); and (10) assessment (Iahad et al., 2004). The evaluation model we used was created and synthesised from the e-Learning evaluation criteria identified from 14 different sources.

In addition, three different evaluation types were identified from the literature in relation to evaluating various stages of the design and development process of an e-Learning system (Table 1).

For our purposes we used formative evaluation when testing the processes and us-ability of the curriculum, PDP materials, content system and e-portfolio interface because it al-lowed us to evaluate the processes and usabil-ity at the point of use (Wright, 2006).

5. THE EVALUATION CRITERIA

Our evaluation criteria for constructing an evaluation framework consisted of 10 different areas of e-Learning criteria and guiding prin-ciples identified from both the literature and the discussions held with staff members (Table 2).

The e-Learning evaluation criteria and guiding principles that make up our evaluation framework are considered important for assess-



ing the effectiveness of the Blackboard learning management system and learning e-portfolio. Within the context of developing evaluation criteria Wright (2003) published a set of crite-ria for evaluating the quality of online courses which we used as part of our effective practice model to assess the effectiveness of an LMS and e-portfolio system. The evaluation for the PDP e-portfolio templates and Blackboard e-Learning system were divided into 10 sections in accordance with the 10 areas of evaluation we identified form the literature which will be discussed within the following subsections.

5.1. General Information Given to Learners Prior to Commencement of Course

A number of criteria have been identified that are important to give to learners prior to the commencement of a course in terms of general information. The criteria relate to information provided at the beginning of a course that will assist the learner to complete the course and to understand its objectives and procedures. Wright (2003) identified a number of detailed criteria in relation to general information that should be made available to learners, examples of which are: Special technology requirements are identified relating to hardware and software specifications needed to take part online. The level of the course is clearly specified for example, using the QAA Scottish Credit and Qualifications Framework, which identifies 12 levels covering access, standard grade, Higher, HND, Degree, M.Sc. and Doctorate. Prereq-uisites are identified including the level of IT skills required to complete an online course. A brief description of the course includes goals

and learning objectives and learning outcomes is provided (QAA, 2004). A list of required and recommended resources including all textbooks, courseware, and online resources necessary to complete the course is made available. If learn-ers must access online databases, instructions are provided for locating and accessing these resources, including password information. If the materials are located in a library, learners are told whether the items are on reserve and, if so, for how long.

Guidelines for participating in online dis-cussions, (“netiquette”) are provided, as well as suggestions for handling incoming e-mail, e-mail attachments, viruses, and e-mail filters. Learners are informed about guidelines and grading criteria for assignments as well as their responsibilities as group members of a class. Learners are directed to a source for answers to “frequently asked questions” pertaining to the LMS and PDP/e-Portfolio. Learners are informed about their right to privacy and the conditions under which their names or online submissions may be shared with others.

5.2. Accessibility of Course Material

With reference to the importance of making course materials accessible online by imple-menting accessibility processes that ensure students can actually access the materials on an equal basis, the CATEA (2006) highlighted the importance of accessibility of course material within an online learning environment. They suggest reviewing course materials within the context of learning barriers so as to anticipate difficulties that learners may face when at-tempting to access online materials. Other

Table 1. Three evaluation types

No. EvaluatingStagesoftheDesign:3Types

1. Diagnostic evaluation is used to identify and analyse preconceptions and assumptions prior to use

2. Formative evaluation is used to appraise implementation processes and usability at the point of use

3. Summative evaluation used to assess outcomes in terms of changes in learning and teaching behav-iours and achievements (Wright, 2006)



factors that should be considered in relation to accessibility are the usability of the course materials in addition to broken web links as well as the rewording of confusing directions. Quality e-learning Web resources should be us-able and not just accessible (Ball & Sutherland, 2003). However, there may be instances where complying with accessibility guidelines might conflict with the usability of the web page or

require undesirable changes to the web pages’ appearance.

5.3. Course Organisation

A number of evaluative criteria and guiding principles concerning course organisation have been highlighted as essential if learners are to make connections between different parts of the course. Examples include the organisation

Table 2. Evaluation criteria

No. EvaluationCriteriaforCurriculum(PDP)Materials,andContentSystem

1. General Information Given to Learners Prior to Commencement of Course: Prior to the start course learners are provided with general information to assist them in completing the course and in under-standing its objectives and procedures. Any special technology requirements are identified such as software and hardware specifications needed to support online learning (Wright, 2003; QAA, 2004).

2. Accessibility of Course Material: Course materials are made accessible online to all learners on an equal basis, in terms of learning barriers, materials should be reviewed to anticipate difficulties that learners might face when attempting to access online materials (Ball & Sutherland, 2003; CATEA, 2006).

3. Course Organisation: The organisation in relation to the layout of materials is arranged appropriately for the subject matter and the intended audience (Wright, 2003).

4. Language: includes spelling, grammar, punctuation, format, reading level, cultural bias, technical terms and screen layout (ANSI, 2004).

5. Layout: Within the context of web design computer screen layout include characteristics such as: margins, white space, and physical readability should follow accepted conventions (ANSI, 2004).

6. Course Goals and Objectives: Outline what learners should expect to learn from a course and where appropriate, what they should expect to learn at the beginning of each module. They specify learn-ing outcomes related to knowledge, skills, competencies, behaviours and attitudes (Mager, 1975; Guilbert, 1984; Gallagher & Smith, 1989; Harrison, 1999).

7. Course Content: Should be designed so as to be assessable to learners form various backgrounds and abilities (Wright, 2003).

8. Learning Strategies and Opportunities for Practice and Transfer: Encapsulates the need to provide learning opportunities for learners of different learning styles. For many learners e-Learning tech-nologies and their associated pedagogies can be unfamiliar to them and the range of possibilities can be diverse. It is therefore important to provide information and opportunities for learners so as to familiarise them with what to expect (Hillesheim, 1998; Ragan, 1999; Marshall, 2005).

9. Learning Resources: provide learners with information or preparation in order that they can make meaningful context and effective use of it in their learning. Learning materials are appropriate for the learners and the subject matter and the learning resources are accessible, appropriate, accurate, and related to the course content. Lists of learning resources are divided into “required” and “optional” categories. Various learning resources are used to ensure compatibility with learners’ different inter-ests, abilities, and learning styles. Learning resources reflecting different points of view are provided when appropriate (Marshall & Mitchell, 2003; Wright, 2003).

10. Assessment: activities are feasible, relevant, accurate, and congruent with the objectives, content, and practical applications of the content. There are one or more components of assessment to demonstrate that the necessary learning outcomes have been achieved by the learner. Clear expectations are given with the criteria for assignments. Along with Examples of assignments that meet the criteria. The number of assignments and their due dates are reasonable. Grading policies are clear and explicit (Iahad et al., 2004).



of units of instruction as well as topics are di-vided into subunits of instruction and subtopics and organisation of components are consistent across the course. References to other parts of the material are correct including links to other parts of the course and external sources of in-formation are accurate (Wright, 2003; Britain & Liber, 2004).

5.4. Course Language

According to ANSI (2004) within the context of language there are several important factors that need to be considered when designing e-Learning materials. These design factors consist of criteria for checking correct spelling, grammar and punctuation of course materials in addition to the reading level being audience and content specific. Other factors involve defining technical terms correctly and consis-tently throughout the course content as well as avoiding cultural bias. Cultural bias may include assumptions made about attitudes, beliefs, and opinions that relate to a specific culture, therefore course content should be usable in cross-cultural contexts. Within the context of readability the reading level will vary and depend on the content, for instance, technical content will usually have a higher reading level due to the number of multi-syllable words used to describe something technical. Instructions, manuals and non-technical content will tend to have a lower reading level. In relation to consistency technical content should be at a single reading level, as well as test questions and instructions. Technical terms should only be used if they are relevant to the content and should include a definition that is understandable to a novice. An online glossary or hyperlinks to term definitions should be considered. Abbreviations or acronyms are specific to a field their initial use should contain an explanation.

5.5. Course Layout

Other important factors highlighted by ANSI (2004) concern guiding principles for design-ing the layout of the Graphical User Interface (GUI) for an e-Learning environment. They

suggest the following guidelines: that the layout for computer screens should be easy to follow with reference to how space and colour is used. Bad screen design can cause confusion for learners with regards to navigation. Spaces between paragraphs should be used to increase the readability of scrolled text. The space between content areas on the screen should contain sufficient space to set them apart and increase the readability of the text. The same spacing should be used throughout the product (Britain & Liber, 2004).

5.6. Course Goals and Objectives

According to Mager (1975) course goals and objectives describe a performance that learn-ers should be capable of demonstrating before they can be considered competent. Clearly written course goals and objectives indicate to learners what they should expect to get out of a course with regards to what they will learn from each module and the course as a whole. They should make explicit the learning outcomes that build new knowledge, skills, competen-cies, behaviours and attitudes (Harrison, 1999). They should be pertinent to the course content and be applicable in the “real world” where the content may be applied. In addition, each main section of a course should have specified “measurable” learning objectives. Guilbert (1984) identified six characteristics of course goals and learning objectives which he considers being important for learners if they are to gain new knowledge and are listed as: (1) Relevant; (2) unequivocal; (3) feasible; (4) logical; (5) observable; and (6) measurable. Others that have written about course goals and objectives are Gallagher and Smith (1989), who state that properly constructed education objectives represent relatively specific statements about what students should be able to do following instruction.

5.7. Course Content

In relation to evaluative criteria for course con-tent Wright (2003) proposes several guidelines relating to effective practices that should be



considered when evaluating course content such as: Learners from different backgrounds and varying abilities should be able to understand the course content as well as carryout instruc-tions for tasks. The content should be directly linked to the learning objects, complete in terms of providing all the content or learning experi-ences needed to achieve the learning objectives and be appropriate to the learners’ character-istics (ability and maturity level). The course content should be accurate, relevant, current, broken into small, incremental learning steps, presented in a logical sequence, and related to other material the learners may have studied or experiences they may have had. It should make use of illustrated examples and/or case studies when new information is presented and linked to other sources. Reading assignments are clearly specified and all quoted materials should be cited correctly, in addition to, permission for all copyrighted material used has been obtained.

5.8. Learning Strategies and Opportunities for Practice and Transfer

Within the context of providing opportunities for practice and transfer there are five levels of characteristic practices operating at each level of process capability identified by Marshall (2005, p. 37). They are: (1) ‘Initial’, the ability of learners to practice and prepare for the use of technologies in online courses is incidental to the provision of materials or systems; (2) ‘planned’, opportunities for learners to prac-tice and develop the necessary skills for using e-learning technology use are explicitly identi-fied in the course materials available prior to commencement of the course; (3) ‘defined’, there are standards for providing learners with preparation and practice opportunities for all standard technologies which are defined along with templates and examples for teaching staff to use in course materials. (4) ‘managed’, com-pliance with standards for providing learners with preparation and practice opportunities is measured, as is learner satisfaction and pre-paredness to use technologies; (5) ‘optimised’,

in relation to measurements of effectiveness of standards for providing learners with sufficient preparation and practice are used to maintain and update the standards and the requirements for courses generally (Hillesheim, 1998; Ragan, 1999; Britain & Liber, 2004; Marshall, 2005).

Learners’ preparedness should be taken into account prior to new technology introduction. In terms of being better prepared to get the most benefit out of e-Learning technologies many learners will need to make plans to ensure they have the necessary skills to be successful in taking part in online learning. Ensuring that learners have adequate information before a course commences will help to prevent early withdrawal or at a later stage (Hillesheim, 1998; Ragan, 1999).

5.9. Learning Resources

Writing about process indicators for evaluating learning resources in e-Learning Marshall and Mitchell (2003) highlight the following recom-mendations. Firstly, students are provided with sufficient background information or prepara-tion that they can place any resource within a meaningful context and make effective use of it in their learning. Select and apply pedagogical strategies which are based on current and rel-evant educational research and which are suited to the particular learning resources. Determine what technologies will be used to support the delivery of course resources and student inter-action. Ensure that the requirements of use of any technology are consistent with the student characteristics and that technology is selected for its educational benefits and not just for its own sake (Wright, 2003).

5.10. Assessment

The need to evaluate online assessment pro-cesses and the role of feedback in learner-centred e-learning has been identified and highlighted by Iahad et al. (2004). They state several key factors that are important to consider when evaluating online assessment criteria. They are that, the relationships between course learning outcomes and assessment strategies are evident



to the learner and between assignments as well as the final course grade being clearly specified. Detailed step-by-step instructions are provided for each component of assessment along with guidelines for submitting assignments. Learn-ers are informed about the criteria that will be used to evaluate their participation in online activities such as discussion groups. A variety of feasible and content-relevant assignments are provided. The assessments are relevant to the learners and the career or profession they may pursue. Learners are able to track and evaluate their own progress and self-tests are similar to the final assessments. If appropriate, learners are informed about the consequences of plagiarism and the failure to properly cite copyrighted material. Learners are told when they can expect to receive feedback from the instructor. Assessment procedures are congruent with the objectives and reflect any priorities that have been established for the objectives (Britain & Liber, 2004).

As a more general issue, e-learning materi-als are generally developed for particular learner populations. With some of the early discussions between the reviewer and the Continuing Aca-demic Personal Development (CAPD) staff, there seemed to be an aim to market the PDP materials together with faculty course content. Having PDP initiatives and differing faculty objectives could prove problematic for staff and students. A more integrated strategy might be needed in terms of a more ‘unified approach’ by faculties towards personal development, at the moment it seems to be left to the discretion of each faculty, as to how they go about embedding PDP into their own curriculum frameworks.

6. THE PROJECT: E-PORTFOLIO EXEMPLAR

After the e-portfolio evaluation criteria was identified from previous e-learning evaluation frameworks identified from the systematic literature search. An e-portfolio exemplar was built from the Blackboard LMS e-portfolio system used at a UK HE institution which was

then evaluated with the evaluation framework identified earlier in this study.

A report was produced for the (CAPD) staff at the HE institution which highlighted issues found and provided solutions and recommen-dations about the LMS e-portfolio exemplar. This was used as a pilot to test out the LMS e-portfolio prior to being used by students at the HE for recording their PDP. To achieve this one sample e-portfolio was constructed based on the PG Certificate in Education templates for evidencing PDP which was informed by teacher placement experience of the researcher. External social networking websites were tested to ascertain which sites are best suited for host-ing students’ personal development plans. The results of the review are as follows:

• The two social websites suited for hosting students’ PDP that we found were: Bebo and MySpace, although limitations were identified in terms of limitability of not be-ing able to create and customise templates and models for the purposes of setting per-sonal categories and topics of interest. We used Bebo to host an external PDP social website that focused on: ‘The Theories of Learning’ due to the availability of the uploading features for upload and videos. MySpace was used to host: ‘The Theories of modern education’.

• A website (Wix) was used as an alternative to using social networking websites as some students might not be comfortable with having a web presence on a social website. Another reason for using Wix is that it is a flash driven website which utilises pre-scripted templates which makes it simpler to create web pages. We used this to host a research page because it provided more control over naming the different categories and topics of the web pages.

• The problems and issues found with using these social Websites for PDP projects were: (1) RSS feeds could not be accom-modated in MySpace; (2) RSS feeds were accessible in Bebo, but were limited to what Bebo provided; and (3) Templates



and Modules could not be modified in both Bebo and MySpace in with reference to renaming modules to suit the purposes of the user.

• As an alternative to using social network-ing websites the flash driven website was found to be simpler for students’ to use with little or no experience in web design. This is a viable option for those who do not wish to have their PDP hosted on a social networking website.

7. OVERVIEW OF THE PDP E-PORTFOLIO TEMPLATE SYSTEM

7.1. Building an E-Portfolio within the Blackboard LMS

The PDP e-portfolio system was developed to be highly configurable using Blackboard LMS. The design tools allow templates to be created that students can choose from when creating a portfolio. The templates can be selected by undergraduate and postgraduate, examples include: Record of placement activities, my learning in context, critical reflections, year 1 initial skills audit, Trimester goals, feedback and employability related activities which are show in Figure 1. Students can build portfolios based on the context of the course, but teachers also have the ability to alter the generic templates to suit the needs of the course or produce their own framework in template form.

7.2. Template Modifying an Item Using Blackboard

The templates can be modified by using the ‘Modify Item’ selection which presents the student with a set of text tools that can be used to edit text much like a word document. The modifications to text can then be saved and altered or added to at a later date.

7.3. Adding a Web Link to a Template on Blackboard

Links to other web sites can be added by using the ‘Add Link’ feature allowing students to link to both internal and external web pages. In the example, the student has linked their e-portfolio to their Bebo web page by entering the web address in to the Uniformed Resource Locator (URL) under the ‘Specify Link’ section.

7.4. Sharing the PDP E-Portfolio with Other Students

The Blackboard framework allows for the sharing of e-Portfolios with other students and members of staff so that comments and feedback can be left about the content of their e-Portfolios. The e-portfolio also facilitates shar-ing with users who are external to blackboard such as prospective employers, staff from other institutions and careers representatives. This is achieved by the owner of the e-portfolio first adding the external user to their share list. Then an email is sent with a set of instructions from the student’s blackboard account which invites the external user(s) to access their portfolio via a web link and password.

7.5. Link to Bebo Web 2 Social Networking Site Created on Blackboard Template

Links to Bebo, MySpace and Wix were inte-grated as external links into the LMS e-portfolio system to allow students to share their personal interests with one another. This was done for the following two reasons. First, it was thought that this would allow students to include their activities out with the university into the LMS e-portfolio system providing a link to their outside interests. Second, it an attempt to link the PDP learning experience to student extra curricula activities bringing together all of their interests. This could be used by the students to provide employers with a more comprehensive



employability profile. To achieve this from the main e-portfolio page a users link was created that links to the Bebo social networking site. Figure 5 shows the Bebo page displayed within the e-portfolio after clicking the link shown in the left user’s panel. The Bebo site can be navigated from within the e-portfolio making visiting other websites and retrieving informa-tion simple without leaving the e-Portfolio.

7.6. Link to MySpace Page Created on Blackboard Template

The social web 2 website MySpace has been added and can be accessed from the main e-portfolio users’ page. Figure 6 shows MySpace web page being displayed from within the e-portfolio user page which can be accessed without navigating away from the e-Portfolio. The Myspace web site can be browsed and

Figure 1. PDP template selection available for students: PGCert Academic Writing Skills

Figure 2. Modifying text selection: example from postgraduate certificate in teaching



information can be downloaded from within the e-Portfolio.

7.7. Linking to My Research Page on Wix Created on Blackboard Template

The web site creator by WiX was used as a Research Log web site and is an alternative to having an external web presence without using social networking sites (as shown in Figure 7). Some students might prefer to build an external web site instead of a social web 2

web site. Building an external web site with WiX is made simple by using flash templates which the student can customise to suit their own purposes.

7.8. Managing My E-Portfolios

As shown in Figure 8 creating a new port-folio and receiving access to other students’ portfolio(s) is made simple with the Portfolio management tool. From this page a student can keep track of their own e-Portfolio(s) and make

Figure 3. Adding a link to other internal or external web pages and web sites

Figure 4. Sharing an e-portfolio with internal or external users



modifications or view received portfolios that other students have shared with them.

7.9. Downloading and Saving My E-Portfolio

The LMS e-portfolio system allows students to save their entire e-portfolio in a .zip file format (shown in Figure 9) which can be imported onto another learning management System (LMS) if a student decides to move to another institution or if after graduating progresses onto a different course at another institution. Students might also wish to keep a permanent record of their PDP progress or wish to continue with their progress file for professional development purposes when starting a job after graduation.

8. NEED TO REDEFINE THE LMS E-PORTFOLIO EVALUATION CRITERIA

LMS e-portfolio evaluation criteria were developed based upon e-learning evaluation frameworks for the purposes of reviewing the suitability of the LMS e-portfolio system used to record students’ PDP at a higher educational institution within the UK between 2009 and 2010. The e-portfolio exemplar was created as an example to demonstrate what can be achieved with a blackboard e-portfolio framework. Al-

though the e-portfolio evaluation framework developed for this study was successfully run, but since its initial implementation the LMS e-portfolio system has undergone changes as a result of the University’s Blackboard system being upgraded. This has resulted in certain as-pects of the LMS e-portfolio evaluation criteria having to be revised to take into account changes and usability upgrades to the system. Once the Blackboard upgrade has been completed across all of the institution’s different campuses the redefined evolution framework can then be applied to the LMS e-portfolio.

For educational purposes the study was undertaken to highlight any short comings of the frame work that was being use to implement PDP within a Higher Educational institution within the UK. In order to do this an ‘Evalu-ation criteria for Curriculum (PDP) Materi-als, and Content System’ was developed as a framework from which to review the framework being implemented across four campuses for 2009/2010 (Table 2). Prior evaluative models and frameworks have been developed or e-Learning initiatives but not much has been developed for e-Portfolios. More research is needed to provide alternate models and frame-works for educationalists and institutions to use when designing and evaluating the effective-ness of their own e-portfolio initiatives. Using the Blackboard e-portfolio templates a PDP

Figure 5. Linking to Bebo Web 2 social networking site



e-portfolio exemplar was created with links to external web 2 social networking sites and a web site. The features that can be used when creating a PDP e-portfolio were documented and are shown in Figures 1 through 9. Screen shots were used to illustrate the simplicity and ease of how an e-portfolio can be constructed using the templates provided by each institutions staff responsible for developing PDP guidelines and models of recording progress of personal development. Social networking web 2 web sites were linked to the institutional e-portfolio as an alternative to using the institutional templates and e-portfolio system, as some students had

made it clear that they feel more comfortable recording their personal development in an environment that they are already familiar with.

9. FUTURE RESEARCH: THE NEXT STAGE IN THE DEVELOPMENT AND USE OF THE PDP PLATFORMS

The next stage of the research in the develop-ment and use of the PDP platform for recording achievement will be to develop the evaluative framework and models as well as effective practice models further by testing them out

Figure 6. Linking to MySpace Web 2 social networking site

Figure 7. Linking to WiX flash driven web site



across other institutions and their schools or faculties. Schools and departments that wish to participate in the dissemination and piloting of the PDP e-portfolio framework will be sought for the purposes of identifying barriers that may inhibit or hinder the recording of their personal progress. Ways will be sought to overcome these barriers in the form of new solutions or enhanc-ers that will inform educational practice among staff and student. The progress of the research will be documented and the results of the study will be published along with the frameworks and effective practice models. They will be made available for educationalists and institutions to

use as guidelines when implementing their own initiatives and evaluations into the effectiveness of PDP e-Portfolios.

The revised LMS e-portfolio evaluation framework will be used to evaluate the updated version of Blackboard LMS e-portfolio system. At some point in the future the LMS e-portfolio evaluation framework will be applied to other LMSs to compare the different approaches and effectiveness by LMS. An important objective of future research surrounding this study will be to examine how Moodle LMS e-portfolio compares to Blackboard LMS e-portfolio. This would relate to a future research objec-

Figure 8. Management interface for my portfolio(s) and received portfolio(s)

Figure 9. Downloading an e-portfolio and saving to zip file format



tive which would be to provide a comparison of the features of the different types of LMS e-portfolio platforms that could be applied to different academic institutions.

10. CONCLUSION

10.1. Research Objectives

The overall objective was to evaluate the LMS e-portfolio system provision at a higher edu-cational institution in the UK, their initiatives for implementing and facilitating ‘Personal Development Plans’ (PDP), and ‘Blackboard Learning Management System’ (LMS). From this a literature search was conducted to as-certain an evaluation framework that could be used to review the institution’s provision for ‘Electronic Portfolio’ (e-Portfolio) templates.

In order to achieve this the paper reviewed the available literature of e-portfolio success within education to: (1) identify which aspects of e-portfolios have been evaluated as a learning tool; (2) describe the evaluation strategies used from e-learning which would be applicable to assessing e-portfolio; (3) present an example of an e-portfolio exemplar which was used to assess the recently developed e-portfolio evalu-ation framework; and (4) discuss educational implications and future research directions.

The adapted model included criteria for the evaluation of curriculum, PDP materials, and the content system. This was grouped into 10 subcategories:

(1) General Information Given to Learners Prior to Commencement of Course;

(2) Accessibility of Course Material;(3) Course Organisation;(4) Language;(5) Layout;(6) Course Goals and Objectives;(7) Course Content;(8) Learning Strategies and Opportunities for

Practice and Transfer;(9) Learning Resources;(10) Assessment.

10.2. The Importance and Benefits of this Paper

This study has highlighted the need for evalua-tive frameworks as well as the need for effective practice models to be developed that can be used to design and evaluate PDP e-portfolio initiatives within Higher Educational institu-tions. This paper is important because it is the first of several papers that will assert the need to develop evaluative and effective practice models for implementing ‘Personal Develop-ment Plans’ (PDP) using e-Portfolios as a means to fulfil the 2010/11 aim of every student in the UK having a PDP/e-portfolio as part of the criteria for the completion of a certificate, diploma, or higher degree and in preparation to replace the honours classification (Burgess, 2007, p. 9). The paper will be of importance to educators, institutions, and other stakeholders seeking evidence in relation to the performance of e-Portfolios as a recording and learning medium for PDP.

REFERENCES

ANSI Standards Committee on Dental Informat-ics. (2004). Working group educational software systems, ANSI/ADA specification 1001: Guidelines for the design of educational software. Retrieved December 2, 2009, from http://www.dental.pitt.edu/informatics/edswstd/

Armitage, C. (1998). The benefits of pause for thought (p. 16). The Australian.

Britain, S., & Liber, O. (2004). A framework for the pedagogical evaluation of eLearning environments: Evaluation of learning management system software part II of LMS evaluation, open source e-learning environment and community platform project. Retrieved October 8, 2009, from http://eduforge.org/docman/view.php/7/17/Evaluation%20of%20LMS%20-%20Part%20II.pdf

Britten, N., Campbell, R., Pope, C., Donovan, J., Mor-gan, M., & Pill, R. (2002). Using meta-ethnography to synthesis qualitative research: a worked example. Journal of Health Services Research & Policy, 7(4), 209–315. doi:10.1258/135581902320432732



Burgess, R. G. (2007). Beyond the honours degree classification: The Burgess Group final report. Retrieved November 1, 2009, from http://www.universitiesuk.ac.uk/Publications/Documents/Bur-gess_final.pdf

Cann, C., Ball, S., & Sutherland, A. (2003). Towards accessible virtual learning environments. TechDis. Retrieved November 1, 2009, from http://www.techdis.ac.uk/resources/VLE001.html

CATEA. (2006). Reviewing and evaluating course materials for accessibility. Retrieved December 18, 2009, from http://www.accesselearning.net/mod2/2_04.php

Gallagher, R. E., & Smith, D. U. (1989). Formula-tion of teaching/learning objectives useful for the development and assessment of lessons, courses, and programs. Journal of Cancer Education, 4(4), 231–234. doi:10.1080/08858198909528016

Gough, D. (2007). Weight of evidence: a framework for the appraisal of the quality and relevance of evidence. Applied and Practice-based Research, 22(2), 213–228.

Guilbert, J. J. (1984). How to devise educational objectives. Medical Education, 18(3), 134–141. doi:10.1111/j.1365-2923.1984.tb00992.x

Harrison, N. (1999). How to design self-directed and distance learning programs: A guide for creators of web-based training, computer-based training, self-study materials. New York, NY: McGraw-Hill.

Hatzipanagos, S., & Lygo-Baker, S. (2008, June 26-27). e-Portfolios for academic development: a career progression vehicle or a private tool of reflection. In Proceedings of the International Conference on e-Learning, Cape Town, South Africa.

Hillesheim, G. (1998). Distance learning: Barriers and strategies for students and faculty. The Internet and Higher Education, 1(1), 31–44. doi:10.1016/S1096-7516(99)80182-1

Hottos, S. (1993). CD-I designers guide. New York, NY: McGraw-Hill.

Iahad, N., Dafoulas, G. A., Kalaizakis, E., & Macau-lay, L. A. (2003). Evaluation of online assessment: The role of feedback in learner-centered e-learning. In Proceedings of the 37th Hawaii International Conference on Systems Science.

JISC. (2007). e-Portfolios: An overview of the portfolio activity area. Retrieved December 2, 2009, from http://www.elearning.ac.uk/subjects/pdpfold

Mager, R. (1975). Preparing instructional objectives (2nd ed.). Belmont, CA: Fearon-Pitman.

Mager, R. F. (1984). Preparing instructional objec-tives: A critical tool in the development of effective instruction. Belmont, CA: Pitman Management and Training.

Marshall, S. (2005). NZ e-Learning capability de-termination: Determination of New Zealand tertiary institution e-Learning capability: An application of an e-Learning maturity model. Report on the e-Learning maturity model evaluation of New Zealand tertiary sector, capability assessment methodology guide. Retrieved November 10, 2009, from http://www.utdc.vuw.ac.nz/research/emm/documents/MethodologyGuide.pdf

Marshall, S., & Mitchell, G. (2003). Potential indica-tors of e-learning process capability. In Proceedings of the EDUCAUSE Conference, Adelaide, Australia. Retrieved November 10, 2009, from http://www.cau-dit.edu.au/educauseaustralasia/2003/EDUCAUSE/PDF/AUTHOR/ED031009.PDF

McGee, P., & Diaz, V. (2007). Wikis and podcasts and blogs! Oh, my! What is a faculty member supposed to do? EDUCAUSE Review, 42(5), 1–11.

Miles, K. (2005). Database e-portfolio systems: A critical appraisal. Computers and Composition, 22(4), 434–458. doi:10.1016/j.compcom.2005.08.003

Petticrew, M., & Roberts, H. (2006). Systematic reviews in the social sciences: A practical guide. Ox-ford, UK: Blackwell. doi:10.1002/9780470754887

Quality Assurance Agency. (2004). Code of practice for the assurance of academic quality and standards in higher education: Section 2: Collaborative provi-sion and flexible and distributed learning (including e-learning). Mansfield, UK: Quality Assurance Agency for Higher Education. Retrieved December 22, 2009, from http://www.qaa.ac.uk/academicinfra-structure/codeofpractice/section2/default.asp

Ragan, L. C. (1999). Good teaching is good teaching: An emerging set of guiding principles and practices for the design and development of distance education. Cause/Effect, 22(1). Retrieved November 14, 2009, from http://www.educause.edu/ir/library/html/cem/cem99/cem9915.html

Richardson, W. (2008). Blogs, wikis, podcasts, and other powerful web tools for classrooms. Thousand Oaks, CA: Corwin.



Schwartz, H. J., & Bridwell-Bowles, L. S. (1987). A selected bibliography on computers in composition: An update. College Composition and Communica-tion, 38(4), 453–457.

Stansfield, M., & Connolly, T. (2009). Advancing e-Learning policies, guiding principles for identifying and promoting best practice in virtual campuses . In Stansfield, M., & Connolly, T. (Eds.), Institutional transformation through best practices in virtual campus development (pp. 147–162). Hershey, PA: IGI Global. doi:10.4018/978-1-60566-358-6.ch010

Strauss, A. L., & Corbin, J. (1998). Basics of qualitative research: Techniques and procedures for developing grounded theory. London, UK: Sage.

Strivens, J. (2007). A survey of e-pdp and e-portfolio practice in UK higher education. Lancashire, UK: Centre for Recording Achievement. Retrieved Oc-tober 16, 2010, from http://www.heacademy.ac.uk/assets/York/documents/ourwork/tla/personal_devel-opment_plan/survey_of_epdp_and_eportfolio_prac-tice_in_uk_higher_education.pdf

Weaver, P. (2002). Preventing e-learning failure: ten common pitfalls and how to avoid them. Retrieved December 9, 2009, from http://findarticles.com/p/articles/mi_m0MNT/is_8_56/ai_90512523/

Wright, C. R. (2003). Criteria for evaluating the quality of online courses, Alberta distance educa-tion and training association, instructional media and design. Retrieved December 9, 2009, from http://elearning.typepad.com/thelearnedman/ID/evaluatingcourses.pdf

Zhang, S., Olfman, L., & Ractham, P. (2007). Design-ing ePortfolio 2.0: Integrating and coordinating web 2.0 services with ePortfolio systems for enhancing users’ learning. Journal of Information Systems Education, 18(2), 203–214.

Gary F. McKenna is a research assistant in the School of Computing at the University of the West of Scotland. His research areas include: the application of e-portfolios for PDP in higher education as well as web based personal tutoring systems, elearning and multimedia.

Mark H. Stansfield is a reader in Learning Technologies in the School of Computing at the University of the West of Scotland. His research areas include: eLearning, virtual campuses and games-based eLearning. He also serves on the editorial boards for several international journals that include the International Journal of Information Management and the Journal of Information.



Keywords: Agile Software Development, Design-Based Research, Multi User Virtual Environments (MUVEs), Open Wonderland, Virtual World

INTRODUCTION

Interest in virtual worlds by educational orga-nizations is growing as they explore ways to gain advantage from these online environments. The potential for collaborative, dispersed but immersive learning has only just begun to be widely explored and there appear to be many opportunities to deliver interactive education in cost effective and innovative ways using

Multi User Virtual Environments (MUVEs) (Schultze et al., 2008).

This paper reports on a research project that seeks to translate an existing face-to-face workshop on agile software development into an online interactive learning experience based within a virtual world. Although the face to face workshop is designed to address the needs of both students and professionals, the virtual world described here is evaluated in the context of tertiary education. In this paper we chart the journey through two iterations of development.

A Virtual World Workshop Environment for

Learning Agile Software Development Techniques

David Parsons, Massey University - Auckland, New Zealand

Rosemary Stockdale, Swinburne University, Australia

ABSTRACTMulti-User Virtual Environments (MUVEs) are the subject of increasing interest for educators and trainers. This article reports on a longitudinal project that seeks to establish a virtual agile software development workshop hosted in the Open Wonderland MUVE, designed to help learners to understand the basic principles of some core agile software development techniques. The authors take a design-based research approach that allows for cycles of learning and reflection that has enabled following an iterative path of development through two major iterations. The authors trace the research path followed from the real world workshop to its virtual incarnation, describing the design philosophy and the constructed virtual artefacts. The authors conclude by reflecting on the insights into learner perceptions and practical implementations gained from building and evaluating the virtual workshop.

DOI: 10.4018/jvple.2012010103



The project takes a design-based research ap-proach that supports a theory based, iterative cycle of learning through the development and experimental phases of the workshop.

The workshop is used to explore the roles of different agile techniques in software devel-opment and enables us to explore the concept of utilizing a MUVE in a virtual learning situation. Using an established workshop as the focus of the experiment allows us to draw on the experiences and expertise of the team member involved in creating, developing and delivering the original material. This supports a more accurate distinction between challenges created by the delivery method and issues that might be attributed to the workshop content.

This paper reports on our experiences of building and evaluating the workshop en-vironment in the Open Wonderland MUVE. The contribution of the research lies in two areas. Firstly, we have evolved and carried out preliminary empirical testing of a conceptual framework that informs virtual world learning. Secondly, we offer findings from the evaluation of the learning that the developers achieved that will contribute to the growing literature on the development of MUVE learning initiatives and opportunities.

LEARNING IN A VIRTUAL WORLD

The use of IT for teaching and learning pur-poses has generated a vast range of literature as researchers and educators seek to understand how best to use software in an educational en-vironment. The use of technology is arguably well suited to active learning where the learn-ing involves the accumulation of knowledge through problem solving, understanding of the knowledge domain and discussion of the tasks achieved (Mayes & Fowler, 1999; Hadjerrouit, 2004). There has been an early recognition of the concept that if learning is to be achieved, software must be designed to make people think, with the emphasis on the task-based learning to be achieved rather than the technology (Mayes

& Fowler, 1999). Similarly, Dickey (2005) recognises that technology tools do not evoke the dynamics of the learning community but that these arise from the interplay of content, instructors and learners. This focus on the learn-ing is further highlighted in Dalgarno and Lee’s (2010) exploration of the potential learning benefits of 3-D virtual learning environments. They use the term affordances rather than ben-efits to underscore the argument that it is the ‘tasks, activities and underpinning pedagogical strategies’ that are supported by the technology and that it is not the technology that provides the learning (p. 18).

The use of virtual worlds as learning envi-ronments has given rise to a very broad range of literature from many disciplines and the need for the adaptation of frameworks to guide researchers in rethinking traditional learning approaches. Dalgarno and Lee (2010) offer a model of learning in 3-D virtual learning envi-ronments that presents two broad categories of representational fidelity and learner interaction. The former addresses the quality of the learning environment such as the need to provide quality visual displays, consistency of object behaviours and most importantly the user representation. User avatars allow for construction of an indi-vidual’s online identity, fostering confidence in their presence in the environment that con-tributes to their social interactions. Quality of representation also appears in de Freitas et al.’s (2010) study of immersive learning experi-ences where they include fidelity, immersion and interactivity as factors of representation in their four dimensional framework for exploring learning activities in virtual worlds.

In Dalgarno and Lee’s category of learn-ing interaction they highlight the ability of the avatars to support embodied action both visu-ally and verbally (gestures, facial expressions, text and voice). The 3-D learning environment also provides for more user input in terms of control of environmental attributes and behaviour and construction of objects. The focus on learner control is also identified by de Freitas et al. (2010) who argue that learning gains are enhanced for learners who have an



element of control over their learning within an environment that encourages engagement, learner-generated content and peer supported communities.

These emerging models and frameworks are required to underpin the extensive range of learning activities evident in virtual world learn-ing environments where many initiatives remain in the early stages of development. Types of educational delivery vary from simple meeting rooms for distance learners (Doesburg, 2008) through to more immersive experiences such as navigating through foreign or historical cities or studying the ecology of wetlands (Kuo & Levis, 2002). More complex situations are created through the development of gaming techniques to create learning spaces where complex tasks simulating real life activities such as medical procedures can be experienced (Levine, 2006). The contextualisation of learning to improve the transfer of knowledge and skills to real situa-tions is found to be a major affordance of the virtual world environment (Dalgarno & Lee, 2010). In designing virtual educational delivery there is some consensus that the completion of tasks is a key learning outcome where the virtual environment provides communication activities that support collaborative assignments with a practical output (Dalgarno & Lee, 2010; de Freitas et al., 2010; Dickey, 2005)

Despite the level of activity in developing virtual world learning there remain concerns that there is little concrete evidence of the advantages that are being gained (Dalgarno & Lee, 2010) and it is difficult to ascertain the significance of current training efforts in this virtual space. Salmon and Hawkridge (2009) note that re-searchers may either be near the beginning of a major development in learning technology or moving towards the end of developing its potential. Researchers have reported consider-able difficulty in creating learning programmes that can be deemed successful with challenges being reported in economic, technical and so-cial spheres (Bainbridge, 2007; Schultze et al., 2008; Eschenbrenner et al., 2008). Atkins and Caukill (2009) sound a further note of caution that the difficulties in developing programmes

to meet complex knowledge requirements will create significant problems for educa-tors and trainers. Nevertheless, there is strong agreement that virtual worlds offer innovative ways to exploit immersive environments that can engage learners in a world that replicates their physical environment or can offer new experiences (Jäkälä & Pekkola, 2007; Phang & Kankanhalli, 2009; Eschenbrenner, Nah, & Siau, 2008).

IDENTIFYING THE ISSUES

The increasing amount of literature on MUVEs supports the identification of some of the current issues facing the development of learning initia-tives. While there are a wide range of such issues including aesthetics, culture and legal concerns (Hadjerrouit, 2004) and concentration, social presence, 3D realism and enjoyment (Phang & Kankanhalli, 2009) this section uses an eco-nomic, social and technical challenges structure to discuss them (Bainbridge, 2007; Schultze et al., 2008; Eschenbrenner et al., 2008).

From an economic perspective a serious drawback is that MUVEs are often created for economic reasons in a sector that has yet to stabilise. MUVEs are, in some cases, com-mercial operations subject to both the laws of the country in which they reside and to the regulations, whims and infrastructure imposed by the owner of the virtual world (Guest, 2007; Sidel, 2008). This makes for an unstable envi-ronment for investment although Mennecke et al., (2008) argue that creating a MUVE for a specific purpose within the corporate environ-ment would be a potential solution to control issues. Unfortunately those tools available for such deployments, typically open source tools such as Open Wonderland, are generally less well developed than the commercial alterna-tives. This increases the cost of development, which can be very high (Schultze et al., 2008; Jacobsen as cited in Dalgarno & Lee, 2010). Virtual worlds are also susceptible to disruption, fraud and other security issues where standards or governance procedures are the responsibility



of the MUVE owner (Mennecke et al., 2008; Guest, 2007).

The social aspect of virtual world learning is seen as a benefit in that it enhances the sense of presence that is so often lacking for traditional distance learners (deFreitas et al., 2010). The ability for learners to communicate and col-laborate is enhanced when text interaction is enhanced by voice and by social cues expressed through the body language of avatars (Dalgarno & Lee, 2010; de Freitas et al., 2010). However there is a downside to the virtual social context. Beyond the control of instructors or learners, social concerns arise in commercially owned MUVEs where dissatisfied customers have noted the high level of regulations that inhibit activities such the case of Linden Labs, which has been likened to a dictatorship by a number of bloggers and wiki contributors. Anti-social behaviour has also been widely reported and rogue users or ‘griefers’ can cause serious dis-ruption within a MUVE. For example, Guest (2007) has documented the specific case of a virtual mafia group led by a character known as Marcellus Wallace. While the challenge to a learning environment appears small, the lack of control by participants remains a concern for educators.

In the technical domain, challenges to establishing effective learning initiatives are extensive with steep learning curves for users and developers (Berge, 2008). Dispersed users require access to the computing power necessary to engage in a virtual world and need to develop at least basic level skills in order to manoeu-vre and operate within the environment, often without physical help. For developers it may ‘take dozens or hundreds of hours … to gain the skills in scripting and the time for creating or building anything that is substantial, creative or innovative’ (Berge, 2008, p. 29). For learners there may be real barriers to adapting to a virtual world where some experience little affinity with the virtual environment, finding it non-intuitive and intrusive. Others have been seen to experi-ence a steep learning curve in adapting to the demands of the virtual world such as creating and operating avatars and communicating via

text (de Freitas et al., 2010). In the same study de Freitas et al. (2010) found that connectivity and the capabilities of the hardware contributed to learner frustration as Internet speeds, firewalls and graphics impeded the representational fidel-ity of the experience. Such concerns have been identified in other studies where for example Mennecke et al. (2008) reported problems with scaling and significant time lags as more avatars joined an online meeting. Further technical is-sues raised by Mennecke et al. (2008) include questions regarding the performance, design and technical capacity of the applications and the need for discussion around the issue of technical standards.

Nonetheless, the perception of substantial potential benefits encourages ongoing develop-ment in this area. The richly interactive environ-ment supports the use of avatars to represent a physical presence that enhances the learning experience and dispenses with the need for physical co-location (Suzuki & Huang, 2004). Perhaps most importantly, MUVEs offer the ability to simulate conditions that would be unrealistic in real life; to create in software ‘things that never were nor could be’ (Brooks, 1995). Bainbridge (2007) suggests that avatars can replace humans in experiments on modeling the spread of virulent diseases or manipulating experiments that require large numbers of par-ticipants. Collaborative tools also support a high level of social interaction where the avatars can represent the users by walking, talking and mak-ing friends, thereby replicating the socializing advantages of face-to-face learning situations (Suzuki & Huang, 2004).

VIRTUAL WORLDS IN THE COMPUTER SCIENCES

There are examples of highly innovative learn-ing activities in scientific disciplines including computer science (Ritzema & Harris, 2008) software engineering (Ye, Liu, & Polack-Wahl, 2007), and artificial intelligence and artificial life (Au, 2006). For example, in the sphere of software engineering education, Ye et al.



(2007) use a game-based approach to encour-age students’ learning. The use of games in the virtual space is often seen as not ‘respectable’ although there is evidence that people learn more effectively when they are immersed within the enjoyment of the learning environ-ment (Rieber, 1996, p. 43). Rieber reflects on a long history of research, arguing that play is a powerful mediator for learning and Phang and Kankanhalli’s (2009) exploration of flow theory found that concentration and enjoyment were key constructs of learning within a virtual world. Ye et al.’s. (2007) experiments with vir-tual world games resulted in positive feedback from the majority of students who believed that their learning experience had improved their learning of the fundamentals of software specification activities and the principles of software development processes. The virtual games-based approach was seen as enjoyable, to have enhanced team communications, and encouraged interactivity. Adverse comments from student feedback related almost entirely to the representational fidelity (Dalgarno & Lee, 2010) of the environment with comments on time lag, connectivity and buggy software and bad graphics.

A further study into the delivery of a computer programming course via Second Life reflected similar feedback from students on the element of enjoyment and the benefits of communication (Esteves et al., 2009). This course aimed to overcome difficulties students find in learning computer programming and aimed to teach them how to design a solution to a problem and to motivate students in the learning of abstract concepts. Esteves et al. (2009) found that where the learning project was found to use the characteristics of the virtual learning environment to the full (e.g., interactivity and movement) the students were more engaged. The strong visual impact was interesting to the students and allowed for the right level of complexity to be judged through the displaying of actions as students progressed through their tasks. Students were then able to instantly correct their errors as they moved

forward which stimulated critical thinking and encouraged collaborative programming.

These two studies confirm findings that designing the learning outcomes to reflect a combination of communication activities that support collaboration leading to a practical output enhances the advantages of a virtual world learning environment (Dalgarno & Lee, 2010; de Freitas et al., 2010; Dickey, 2005). Esteves et al. (2009) also confirm Atkins and Caukill’s (2009) view that developing virtual world learning environments is very demand-ing of educators requiring skilled planning and design as well as intensive preparation to support students through the learning process.

THE AGILE WORKSHOP

One of the key success factors for a virtual world activity is good task design (Vallance et al., 2010.) One approach to ensuring good task design is to take a successful real world task and host it inside a virtual world. In this paper we describe the implementation of a virtual world workshop activity for teaching about agile software development techniques. This activity meets many of the commonly stated criteria for potential success as a virtual world activity that have been previously discussed, including communication, collaboration and practical output (Dalgarno & Lee, 2010; de Freitas et al., 2010; Dickey, 2005.)

In order for this project to be understood it is necessary for us first to describe the real world activity that has been moved into a virtual world. The ‘Agile Technique Hour’ workshop is a classroom based face to face activity that is designed to provide participants with the opportunity to reflect on the nature of agile software development through experiencing a number of agile techniques. It is based on Cockburn’s (2002) concept of the process min-iature, whereby the key features of a software development process can be explored in a very short time period. There are a number of such activities that have been widely used in the past, including the Extreme Hour (Cunningham



& Cunningham, Inc., 2005) and the XP Hour (Peeters & Van Cauwenberghe, 2001.) However these other activities tend to focus on project management aspects of agile methods, whereas research has shown that it is the individual tech-niques chosen that can have the most impact (Parsons, Ryu, & Lal, 2007). The intention of the Agile Technique Hour workshop is to explore a set of core techniques that are used in agile software development. These are; stakeholder participation, co-location, pair programming, test driven development, continuous integration, common coding guidelines and refactoring. The game works over three iterations within which the teams have to design a human powered ve-hicle. In the first iteration, no agile techniques can be used. In the second, some techniques are allocated and some can be chosen from a set of options. In the final iteration, further allocated and optional techniques are introduced. The intention is for the participants to appreciate not only the value of individual techniques but also to see how they combine together to support a software development process.

In the physical version of the workshop, teams are given sets of pre-written user stories for each iteration which they must prioritise based on business value and estimated effort. Each story relates to a required feature of the vehicle, and each feature must be drawn on a single overhead transparency sheet. The overall design of the vehicle is created by laying all the various features on top of one another.

The workshop in its real world form has proved very successful, not only with students but also with academics and professionals, who have participated at conferences and on training courses. However there are some issues that led us to consider the potential benefits of hosting the workshop in a virtual world. The first issue is one of resources. Each workshop requires a suitable room with an overhead projector, which are becoming less common than they used to be in university classrooms. Although the trans-parencies can be laid on white paper on a table top if necessary, this limits their visibility when working with larger groups. Then there is the issue of the transparencies themselves. Again

these are becoming less common as a standard stationary item, and each team can use up to 30 transparencies in one workshop. Since a single workshop can include several teams, this uses a large number of transparencies. A suitable number of overhead transparency pens and story cards also need to be prepared for each work-shop, and there is also some attrition of these. The second major issue is that the workshop and only be run when people are physically co-located, but we would like to be able to offer this workshop outside the constraints of local classes and occasional conference workshops. Virtual worlds offer advantages in addressing both resource constraints and physical loca-tion that would both be beneficial for running virtual workshops.

TAKING A DESIGN-BASED APPROACH

Our methodology was grounded in design-based research, with long iterative cycles. Within these cycles, we have addressed the components of the knowledge base from different perspectives, and the evolution of the various constructed components has also meant that the evaluation stages have had different concerns at different stages of the overall development cycle.

Petter (as cited in Vaishnavi & Kuechler, 2007) maps the process steps of the general design cycle to the broader phases of a particular research project. This is not designed to be a generic view of all research, but it neverthe-less helps to contextualise our work. We have adapted this concept to apply the primary concerns of our own research phases to this model (Figure 1).

The following sections summarise our research process through Petter’s stages of tool evaluation, theories and frameworks, design and assembly, experimentation (with conclu-sions) and future research.



TOOL EVALUATION

Initially, we undertook a tool evaluation phase to identity the most appropriate tools for imple-menting our project. Our analysis was based on the economic, social and technical agenda outlined in the introduction. We evaluated three virtual worlds as possible platforms, finally selecting Wonderland, a free, cross-platform, open source Java project as the most suitable for our purposes (the others were Second Life and Open Simulator). Originally developed by Sun Microsystems as Project Wonderland, it has sophisticated communication tools and the ability to share applications and documents. Sun’s subsequent acquisition by Oracle led to the release of the project into the open source community, and it has been renamed Open Wonderland. A key advantage of Wonderland is it is an extensible open source Java applica-tion, rather than a commercial venture with its own scripting language. This offers many programming resources and Wonderland can be used on a local network within a firewall of an organization, without the cost of rent-ing virtual space and material on a third party server. Another important feature is that it does not have the potential distractions for learn-ers of public virtual spaces, nor the danger of external ‘griefers’.

Gardner et al. (2008) have identified key issues regarding Wonderland as a virtual world platform for teaching. On the plus side, Wonderland is open source and extensible, and more platform agnostic than many open source alternatives due to its Java codebase. It also

enables greater control over resource access, privacy and security than commercial MUVEs. The primary intent of the Wonderland platform is that it can be tailored and integrated by organizations within their own infrastructures (Gardner et al., 2008). A potential issue is that the system works well within an organisational firewall but has problems providing equal ac-cess for remote users. For example in order for Wonderland to function correctly a large number of ports must be opened on the server, which may be regarded as creating potential vulnerabilities. Nevertheless our analysis was that Wonderland was an appropriate platform for our work.

THEORIES AND FRAMEWORKS

The work of Gardner, Scott, and Horan (2008) on the evaluation of MiRTLE at the University of Essex draws on earlier work regarding theo-retical frameworks in computer based learning. Gardner et al.’s (2008) extension of Mayes’ framework identifies three levels or modes of learning. The first mode of conceptualisation involves considering the ideas or concepts of others, which leads to construction and ‘the building and testing of one’s knowledge through the performance of meaningful tasks’ (Gardner et al., 2008, p. 138). This in turn leads to dia-logue between learners, their peers and their teacher that results in new concepts emerging thereby leading back to conceptualisation and an iterative cycle of learning.

Figure 1. Research phases (below) mapped to the general design cycle (above) (based on Petter as cited in Vaishnavi & Kuechler, 2007)



In a virtual space these modes of learning can be mapped to the concepts of immersion; psychological, physical and social (Gardner et al., 2008). Mayes’ original framework is based on the categorisation of courseware, which is divided into primary (subject matter), secondary (environment, tools and tasks) and tertiary (produced by other learners) (Mayes & Fowler, 1999). In Table 1 we have integrated these various perspectives and mapped them to our virtual world workshop activity. This has helped us to understand the nature of the work that we are undertaking in ways that go beyond simply delivering subject matter (conceptualisa-tion). We can recognise that our work already addresses some core concepts of construction and dialogue, but are also able to identify the key themes that should continue to be the focus of future work. We are also aware of further perspectives that may help to contextualise our aims and objectives.

DESIGN AND ASSEMBLY

The main requirements of the design of the virtual world for the agile workshop were that it should provide virtual equivalents of the

tools used in the face to face workshop; the transparencies, the pens, the story cards and the overhead projector. When we investigated the virtual tools at our disposal, it seemed that we could adapt the Wonderland whiteboard viewer to adopt the same role as the overhead transparencies and pens in the physical work-shop. The basic functionality was available to enable a feature to be drawn on a whiteboard in various colors using freehand drawing, text or shapes chosen from the toolbar. The challenges were in emulating the overlaying of multiple transparencies on an overhead projector. In the physical workshop, the fact that the projector is in its own space, apart from the team areas is important as it simulates the deployment of a complete set of features to an integration system. We decided to again use the whiteboard tool to replace the projector, but still place it separately from the team areas to ensure that the step from development to deployment was non-trivial. Therefore a separate customized whiteboard was created to act as this integration space.

The Wonderland whiteboards do not by default support all the functions required in the workshop. Therefore they had to be customized in different ways to support both the required developer tools and the integration system. In

Table 1. A conceptual framework for virtual world learning mapped to the agile in wonderland workshop (adapted from Gardner et al., 2008; Mayes & Fowler, 1999)

FrameworkConcepts TypesofImmersion MappingtoAgileWorkshop

ConceptualisationPrimary courseware: subject matter Basic resources

Psychologicalimmersion(abstractspace)Deliberately abstract; explorative; self-directed; experimental; multiple representations/visualisations

PreworkshopactivitiesNeed to be designed to cater for different backgrounds and experi-ences, access materials and experi-ment with tools

ConstructionSecondary courseware: environ-ment, tools and tasks Interactive resources

Physicalimmersion(physicalspace)Deliberately concrete; realistic be-haviours; manipulative, role playing; multiple viewpoints; tutor directed; expected outcomes

WorkshopcontextandprocessMultiple realistic software develop-ment roles, organised by modera-tor, assessed products and learning outcomes

DialogueTertiary courseware: produced by previous learners Creational resources

Socialimmersion(socialspace)Deliberately situated; localised con-versations; identity; reactive avatars; meeting rooms

WorkshopenvironmentsCustom built context, localised team and developer rooms, co-located avatar conversations required



the real world workshop, when a developer has finished drawing a feature on a transparency, that transparency is added to the team’s collection of complete features, ready for the next integration session at the end of the iteration. We needed some way of simulating this practice with the whiteboards, once a feature has been created. To do this, we redesigned the whiteboard toolbar to include additional team buttons to allow an image to be sent to a team repository via a button labeled with the team’s letter (A, B, C or D), as can be seen on the modified toolbar shown in Figure 2. When the appropriate team button is pressed, the current drawing is stored as a scaled vector graphics (SVG) image ready for integration, and the whiteboard is cleared ready for the next feature to be drawn.

The implementation behind this whiteboard toolbar enables a set of SVG images to be col-lected together for each team and then overlaid as a single image on the integration version of the whiteboard. This whiteboard also has a custom toolbar (Figure 3). In this case, editing is not required (or wanted); the whiteboard is intended for ‘reading’ rather than ‘writing’. The role of this whiteboard is to enable all the fea-tures from a single team to be displayed inte-grated together. At this point in the workshop, the person in the quality assurance role performs

a series of ‘acceptance tests’, looking at the features and accepting or rejecting them ac-cording to the test criteria. Because the integra-tion whiteboard just needs to be able to display the results from different teams, the toolbar does not have any drawing tools. The team buttons, instead of clearing the image and send-ing it to the team repository, as they do on the programmer whiteboard, retrieve the full set of images for that team and display them to-gether.

The other key component was the story cards. We implemented these using PDF view-ers that enabled a story board to be placed near a developer whiteboard. The tools provided with the PDF viewer enable the stories to be cycled through by the user.

DESIGNING THE VIRTUAL SPACE

One of the major limitations of the real world workshop is that it always takes place in an ad hoc environment based on the availability of rooms. These are never ideal and do not always contain the required equipment or a suitable ar-rangement of space and furniture for teamwork. One major advantage of the virtual workshop is that we were able to design our own custom

Figure 2. The toolbar on the customized developer whiteboard, including team buttons for stor-ing features

Figure 3. The toolbar on the integration whiteboard, including team buttons for displaying integrated features



space to host the workshop, designed to support the workshop activities. We built a large virtual building with four separate team workspaces by default, though they can easily be reconfigured with different numbers of workspaces. Figure 4 shows the typical view of an avatar when the client application is first launched. The user be-gins at the front of a four winged building, each wing containing a separate project team room.

Inside the building, each team workspace comprises a number of developer rooms, each containing a developer whiteboard and a story card viewer. Figure 5 shows an avatar inside one of the team areas. Separate developer areas are visible containing the whiteboards and story card viewers.

When an iteration is in progress, develop-ers will be drawing allocated user stories as individual features on whiteboards (one story = one feature = one drawing). Figure 6 shows an example where the user story ‘The driver must be protected from attack by wild animals’ is being implemented. The developer has drawn a cage-like structure that can be applied to the

vehicle. It is important to note that such an activity enables the user to participate in a creational environment (ref. the framework in Table 1).

Each time a developer completes the imple-mentation of a specific user story, s/he commits that feature to the team repository by clicking the appropriate team button (A, B, C or D) on the whiteboard tool bar, storing the feature drawing and clearing the whiteboard ready for the next feature. Figure 7 shows the same de-veloper working on the next user story; ‘The vehicle must be able to travel over rough and uneven ground’. Again, on completion of the drawing, the feature will be committed to the team repository.

In a complete iteration, there will be four developers simultaneously working on a num-ber of user stories. To simplify the figures, the example here shows only the two story imple-mentations from Figures 6 and 7. At the end of the iteration, the teams will gather round the integration whiteboard. Each team’s overall design, consisting of all the individual features

Figure 4. The exterior of the workshop building, showing the entrance to the four main team areas



layered together, can be viewed by selecting the team’s identifier from the integration white-board toolbar. Figure 8 shows the two features from our examples being combined for the team. This is a further aspect of a creational environment; the components created by indi-viduals are combined to create on overall arte-fact created by the team as a whole. At this point, construction leads on to dialogue (Table 1).

EXPERIMENTATION

The practical tests and evaluation carried out thus far have focused on technical performance and qualitative measures of system usability. There were two sessions of experimental test-ing after each iteration, using postgraduate students. The first set of tests and evaluations was carried out in two stages, the first within the university before a second test was run overseas. Some results of these tests led to a number of technical guidelines being produced to enable the workshop to run smoothly, including en-

vironmental settings, platform configuration and login procedures, as well as changes to the actual workshop environment and tools (more detailed results from the first iteration can be found in Parsons and Stockdale, 2010). In the evaluation session that concluded the second technical iteration, a group of nine postgraduate students were asked to qualitatively assess the virtual world workshop as part of their normal classroom activities in a course exploring agile software development methods. Table 2 sum-marises our research questions, variables, levels of analysis, methods and instruments for this evaluation using the format for design-based research suggested by Santos (2010).

As part of the evaluation process the stu-dents first participated in the real world work-shop. Having completed this, but before par-ticipating in the workshop activity in the virtual world, they were asked to reflect on what they thought the potential advantages or disadvantages there might be in delivering the workshop in a virtual world. The intention of this reflection process prior to experiencing the

Figure 5. Outside the developer rooms, showing whiteboards and story card viewers



virtual workshop was to gain unbiased feedback prior to the participants having experience of the particular design approach we had taken. Of course we had our own ideas about why we thought implementing the workshop as a vir-tual activity might be of value, but we wanted to get other perspectives. The feedback re-ported here is summarised from notes taken during wide ranging discussions that often went beyond the original trigger questions. The main advantages identified by the group included:

• The ability to deliver the workshop inde-pendently of spatial constraints

• The ability to deliver the workshop in-dependent of time constraints, including across time zones

• The levelling nature of being in a virtual world, where it was less likely that certain individuals would be able to dominate the proceedings

• The equalising nature of being an avatar, whereby disabilities or other limita-

tions would not be so obvious to other participants

• The ability to create artefacts that could not be created in reality

• The saving of various kinds of resources, including the overheads of travel and materials

The main disadvantages were considered to be

• Participants not taking the workshop as seriously as they would in real life. Discus-sion centred on the way that some people tend to behave on line in contexts such as chat rooms and social networking sites

• Technical limitations. This discussion centred on the idea that in theory, a virtual world could do anything (e.g., the Holodeck from Star Trek, or the Matrix), but that in practice there would be major limitations on what would be possible with current technology, particularly in terms of inter-action modes.

Figure 6. Implementing a user story on a developer whiteboard



The main findings from this session were that the students paid much more attention to the possible social benefits and drawbacks of using a virtual world than we had done as designers.

To ensure that we had taken proper ac-count of the conceptualisation stage of the adopted framework, the students were then given a supervised orientation session in the virtual world, one week prior to doing the virtual workshop activity. This orientation session consisted of configuring the client so that it would work correctly in the network environment of the computer lab, navigating around the virtual environment and learning to use the whiteboard tools and story viewers. The participants were all in the same computer lab during the orientation session so that they could be assisted where necessary. They were also invited to spend further time in the virtual world whenever they wished to between the two sessions, so they could explore the world truly as a virtual experience, and become more familiar with the tools and environment. Since the client runs via Java Web Start, the only requirements

for their own computers were a web browser and the Java Runtime Environment. The fol-lowing week, the same group performed the workshop activities in the virtual world and were asked to reflect on their experience, and comment on the way the workshop had been implemented in the virtual world. This session again took place in a single lab, to enable the students’ interactions with the virtual activities to be observed for evaluation purposes.

In the post hoc reflection session, the general responses were positive in terms of the design decisions regarding the use of white-boards in place of overhead transparencies. The participants suggested that the whiteboard tools were better than using the transparencies because it was easier to arrange the drawings, and in particular it was easier to move features that were rejected by the quality assurance role, or were being replaced as part of a refactoring process. In the real world workshop the manage-ment of the overhead transparencies had proved increasingly difficult for this group through the various iterations, particularly when they had

Figure 7. Implementing a second user story after the first has been committed



chosen ‘refactoring’ as one of the optional tech-niques. The main problem with the whiteboards appeared to be the difficulty in telling apart the developer and integration whiteboards, because

the customised toolbars only appear when con-trol of the whiteboard has been taken. Prior to that point, all the whiteboards look much the same, apart from their size.

Figure 8. Combining user stories on the integration whiteboard

Table 2. Research questions, variables, levels of analysis, methods and instruments for evalu-ation (Santos, 2010)

ResearchQuestions Variable LevelofAnalysis

Methods Instruments

What do users perceive as the potential advantages and disadvantages of a virtual world workshop?

Contextual Group Brainstorming Trigger questions

What is the users’ level of engage-ment during learning?

Learning Cognitive Observation Observation guide

How were the collaborative learning activities performed?

Climate Group Observation Observation guide

How do the virtual world resources scaffold learning

Contextual Resource Observation, reflection

Observation Guide, Trigger ques-tions

How well does the virtual world support the intentions of the activity

Learning Group Reflection Trigger questions



A number of comments were made about the slow loading of the client, and it was sug-gested that the world was too large, and spent too long loading in components that were never used, including a whole set of ‘cones of silence’ that were not used because the voice tools were disabled by a lack of open ports. However our previous evaluations suggested that making the world smaller and simpler did not seem to help much in terms of performance. However the size of the environment was also criticised for making it harder to navigate around.

Perhaps the most disappointing aspect of the evaluation session was that the students seemed quite happy with the design of the vir-tual world and its tools, whereas we had hoped that we might have given them the opportunity to provide some more imaginative feedback.

CONCLUSION

Despite Berge’s (2008) assertion that there are few activities in Second Life that cannot be taught via websites, there are many features of a MUVE that can provide a unique value proposition. These include initiatives that would be difficult if not impossible to deliver by other means such as Yellowlees’s experiment in psy-chiatry (Berge, 2008) or the British Literature Classroom (McDunnough, 2007). While this type of unique value does not apply to our virtual workshop, which is already presented in a face-to-face situation, we believe there are benefits to be gained from experimenting with learning initiatives in MUVEs. These benefits can be realized through the hedonic element of social interaction within the workshop that contributes to the ability of participants to gain the required skills. Agile methods require focus on context and relationships to achieve the best outcomes from applying the techniques. We have endeavoured to capture this idea in the workshop environment within the dialogue con-cept of the framework. The virtual environment is well placed to enhance this social dimension if the design provides the creational resources for social immersion.

In terms of resources there are several forces to be addressed. While it is possible to begin using virtual worlds at a very low cost, these costs quickly escalate both in terms of the direct costs of supporting a facility (e.g., renting virtual space) and in development requirements (i.e., time and skills). As Hiltze remarked when asked if there is a future for business game simulations in a MUVE ‘yes….if I had a grant and a smart graduate student who could do all the programming for me’ (in Schultze et al., 2008, p. 366). There is also the hidden cost of access; it became clear in our tests that a powerful computer with a good graphics card was required to run the Wonderland cli-ent, which was more demanding on hardware resources than, for example, the Second Life client. We had decided to use an Open Source virtual world largely due to the economic and social issues outlines in the introduction; we had not fully absorbed the implications of the technical costs from an end user’s perspective.

The implications for practice that arise from our project are, as yet, to be well defined. Initia-tives of this type are certainly in their infancy as far as formal learning is concerned, though educational institutions have made significant contributions. Priority and visibility are given to initiatives that emphasize the unique value proposition that MUVEs offer and require substantial resources to develop. However, the potential for learning initiatives are extensive and projects such as this indicate the path for small, low resourced research teams to develop the skills that will keep them in the vanguard of MUVE developments.

In terms of implications for theory, we leveraged and further evolved a framework that facilities our work and drawn on a design-based approach to inform our cycles of iteration and reflection.

Overall, we anticipate that our project will contribute to the development of more sophistication in the further development of learning opportunities in virtual workshop based activities.



FUTURE RESEARCH

A characteristic of the condition of design sci-ence research effort is that ‘it is the result of satisficing’ (Vaishnavi & Kuechler, 2007, p. 21). Thus whilst our final artefacts may not yet be in perfect harmony with our original intentions, nevertheless we can judge that outcomes are the source of valuable knowledge, and the root of further research efforts.

Further work lies in the need to develop the workshop to a level that can be tested and evaluated in everyday rather than experimental contexts, in other words to feel as confident about running the workshop in virtual world as we currently are in the classroom. There is also more work to be done on testing the virtual world toolsets to gain a clearer understanding of the pros and cons of adopting a particular platform. A valuable contribution of future work would be to make an in depth comparison of a range of virtual world platforms and their ability to support specific types of virtual world learn-ing, bearing in mind the types of hardware and internet connectivity that the average student is likely to have available.

With the amount of growing activity in virtual worlds, there is vast scope for future research, both in creating artefacts and as Bainbridge (2007, p. 427) notes MUVEs in offering ‘great potential as sites for research in the social, behavioural, and economic sciences, as well as in human-centred computer science.’

REFERENCES

Atkins, C., & Caukill, M. (2009). Serious fun and serious learning . In Molka-Danielsen, J., & Deutschmann, M. (Eds.), Learning and teaching in the virtual world of Second Life (pp. 78–89). Trondheim, Norway: Tapir Academic.

Au, W. (2006). God game. New World Notes. Re-trieved May 5, 2011, from http://nwn.blogs.com/nwn/2006/05/god_game.html

Bainbridge, W. (2007). The scientific research poten-tial of virtual worlds. Science, 317, 472. doi:10.1126/science.1146930

Berge, Z. (2008). Multi-user virtual environments for education and training? A critical review of Second Life. Educational Technology, ▪▪▪, 27–31.

Brooks, F. (1995). The mythical man-month, essays on software engineering (2nd ed.). Reading, MA: Addison-Wesley.

Cockburn, A. (2002). Agile software development. Reading, MA: Addison-Wesley.

Cunningham & Cunningham, Inc. (2005). Extreme Hour wiki. Retrieved May 5, 2011, from http://c2.com/xp/ExtremeHour.html

Dalgarno, B., & Lee, M. J. W. (2010). What are the learning affordances of 3-D virtual environments? British Journal of Educational Technology, 41(1), 10–32. doi:10.1111/j.1467-8535.2009.01038.x

de Freitas, S., Rebolledo-Mendez, G., Liarokapis, F., Magoulas, G., & Poulovassilis, A. (2010). Learning as immersive experiences: Using the four-dimensional framework for designing and evaluating immersive learning experiences in a virtual world. British Journal of Educational Technology, 41(1), 69–85. doi:10.1111/j.1467-8535.2009.01024.x

Dickey, M. D. (2005). Brave new (interactive) worlds: A review of the design affordances and constraints of two 3D virtual worlds as interactive learning environments. Interactive Learning Environments, 13(1-2), 121–137. doi:10.1080/10494820500173714

Doesburg, A. (2008, June 16). Way too cool for school? Try virtual learning. New Zealand Herald.

Eschenbrenner, B., Nah, F. F.-H., & Siau, K. (2008). 3-D virtual worlds in education: Applications, benefits, issues, and opportunities. Journal of Da-tabase Management, 19(4), 91–110. doi:10.4018/jdm.2008100106

Esteves, M., Fonseca, B., & Martins P. (2009). Using Second Life for problem based learning in computer science programming. Virtual Worlds Research, 2(1).

Gardner, M., Scott, J., & Horan, B. (2008). Reflec-tions on the use of Project Wonderland as a mixed-reality environment for teaching and learning. In Proceedings of the Conference on Researching Learning in Virtual Environments (pp. 130-141).

Guest, T. (2007). Second lives: A journey through virtual worlds. London, UK: Hutchinson.



Hadjerrouit, S. (2004). Implementing web-based Learning in higher education: An evolutionary software engineering approach. In L. Cantoni & C. McLoughlin (Eds.), In Proceedings of the World Conference on Educational Multimedia, Hypermedia and Telecommunications (pp. 483-490). Chesapeake, VA: AACE.

Jäkälä, M., & Pekkola, S. (2007). From technology engineering to social engineering: 15 years of re-search on virtual worlds. The Data Base for Advances in Information Systems, 38(4), 11–16.

Kuo, E., & Levis, M. (2002). A new Roman world: Using virtual reality technology as a critical teaching tool. In Proceedings of the Annual Meeting of the American Educational Research Association, New Orleans, LA.

Levine, A. (2006). Heart murmur sim, assessment of learning in SL - Interview with a man in a sur-gical mask. NMC Campus Observer. Retrieved May 5, 2011, from http://sl.nmc.org/2006/09/25/jeremy-kemp/

Mayes, J., & Fowler, C. (1999). Learning technology and usability: A framework for understanding course-ware. Interacting with Computers, 11, 485–497. doi:10.1016/S0953-5438(98)00065-4

McDunnough, S. (2007). Playing the piano in the British Literature classroom. Retrieved May 5, 2011, from http://bloghud.com/id/8799/

Mennecke, B., McNeill, D., Roche, E., Bray, D., Townsend, A., & Lester, J. (2008). Second Life and other virtual worlds: A roadmap for research. Communications of the Association for Information Systems, 22(20), 371–388.

Parsons, D., Ryu, H., & Lal, R. (2007). The impact of methods and techniques on outcomes from agile soft-ware development projects . In McMaster, T., Wastell, D., Ferneley, E., & DeGross, J. (Eds.), Organisational dynamics of technology-based innovation: Diversify-ing the research agenda (pp. 235–249). New York, NY: Springer. doi:10.1007/978-0-387-72804-9_16

Parsons, D., & Stockdale, R. (2010). Cloud as context: Virtual world learning with open wonderland. In Proceedings of the 9th World Conference on Mobile and Contextual Learning, Malta (pp. 123-130).

Peeters, V., & Van Cauwenberghe, P. (2001). The XP game. Retrieved from http://www.xp.be/xpgame.html/

Phang, C. W., & Kankanhalli, A. (2009). How do per-ceptions of virtual worlds lead to enhanced learning? An empirical investigation. In Proceedings of the 30th International Conference on Information Systems.

Rieber, L. P. (1996). Seriously consider play: Design-ing interactive learning environments based on the blending of microworlds, simulations and games. Educational Technology Research and Development, 44(2), 43–58. doi:10.1007/BF02300540

Ritzema, T., & Harris, B. (2008). The use of Second Life for distance education. Journal of Computing Sciences in Colleges, 23(6), 110–116.

Salmon, J., & Hawkridge, D. (2009). Editorial: Out of this world. British Journal of Educational Technology, 40(3), 401–413. doi:10.1111/j.1467-8535.2009.00957.x

Santos, A. (2010). Using design-based research for virtual worlds research projects. Journal of Virtual Worlds Research, 3(1).

Schultze, U., Hiltz, R., Nardi, B., Rennecker, J., & Stucky, S. (2008). Using synthetic worlds for work and learning. Communications of the Association for Information Systems, 22(19), 351–370.

Sidel, R. (2008, January 23). Cheer up, Ben: Your economy isn’t as bad as this one. Wall Street Journal. Retrieved May 5, 2011, from http://online.wsj.com/article/SB120104351064608025.html?mod=todays_us_page_one

Suzuki, H., & Huang, R. (2004). Virtual real-time 3D object sharing for supporting distance education and training. In Proceedings of the 18th International Conference on Advanced Information Networking and Applications, Fukuoka, Japan.

Vaishnavi, V., & Kuechler, W. (2007). Design science research methods and patterns: Innovating informa-tion and communication technology. Boston, MA: Auerbach. doi:10.1201/9781420059335

Vallance, M., Martin, S., Wiz, C., & van Schaik, P. (2010). Designing effective spaces, tasks and met-rics for communication in Second Life within the context of programming LEGO NXT Mindstorms™ Robots. International Journal of Virtual and Personal Learning Environments, 1(1), 20–37. doi:10.4018/jvple.2010091703

Ye, E., Liu, C., & Polack-Wahl, J. (2007). Enhancing software engineering education using teaching aids in 3-D online virtual worlds. In Proceedings of the 37th ASEE/IEEE Frontiers in Education Conference, Milwaukee, WI (pp. 8-13).



David Parsons is a senior lecturer in the Institute of Information and Mathematical Sciences at Massey University, Auckland, New Zealand. He has a PhD in information technology from Nottingham Trent University and has wide experience in both academia and industry. In addi-tion to his university role he acts as a knowledge engineer for Software Education Associates, specialising in Java technologies and agile software development methods. He is the author of a number of texts on computer programming, web application development and mobile learning, and is the founding editor in chief of the International Journal of Mobile and Blended Learning. He is a member of the International Association for Mobile Learning and a professional member of the British Computer Society.

Rosemary Stockdale is an associate professor in the faculty of information and communication technology at Swinburne University of Technology in Melbourne, Australia. She has a PhD in information systems from Edith Cowan University, Perth and has worked in universities in Aus-tralia, New Zealand, Scotland and Austria. Her research interests include mobile and e-learning, online communities and the use of social media in organizations. Dr Stockdale has published in a range of Information Systems journals and conferences including the European Journal of Operational Research, Information and Organization and the European and Australasian Confer-ences on Information Systems.



Keywords: Category, E-Learning, Feature, Intelligent Software Agent, Review

INTRODUCTION

Computer technologies are significantly chang-ing the content and practice of education. The consequent application of multimedia and simulation technologies, computer-mediated communication and communities, and Internet-

based support for individual and distance learning have the potential for revolutionary improvements in Education. Online learning (i.e., e-learning) offers new possibilities in learn-ing. In a simple definition e-learning is defined as the use of network technology, namely the Internet, to design, deliver, select, administer and extend learning. Important features of this form of learning are the separation of learner and

Utilization of Intelligent Software Agent Features for

Improving E-Learning Efforts:A Comprehensive Investigation

Mandana Farzaneh, University of Tehran, Iran

Iman Raeesi Vanani, University of Tehran, Iran

Babak Sohrabi, University of Tehran, Iran

ABSTRACTE-learning is one of the most important learning approaches within which intelligent software agents can be efficiently used so as to automate and facilitate the process of learning. The aim of this paper is to illustrate a comprehensive categorization of intelligent software agent features, which is valuable for being deployed in the virtual world of learning and training. The characteristics of agents make them worthwhile for being used in the implementation and improvement of e-learning information systems, especially the web-based or web-enabled e-learning mechanisms. An extensive exploration of agent-related international papers has been conducted so as to cover a wide range of internationally accepted features as well as the classification of agents and their correspondent references. These agent characteristics enable the online learning mechanisms to act on behalf of the user in electronic learning and to respond to specific requirements in a more efficient and effective manner than the traditional learning systems. The implementation of agents according to the most useful features helps teachers transfer the related knowledge easier, faster, and more directed towards the targeted audience of the e-learning system for a better and deeper learning quality

DOI: 10.4018/jvple.2012010104



teacher that can take place anywhere (workplace or at home), at any time and at any pace. Thus, through e-learning a student can get immediate feedback on solutions to problems and learning paths can be individualized.

Online learning is, as a result, a growing business. Thus, the number of organizations that are working on online learning and the number of courses available on the Internet are grow-ing rapidly. In addition, personalized services have received considerable attention recently because the information needs are different among users. A significant consideration in web-based educational systems is the structure of the domain and the content, which are usually presented in a static way without taking into account the learners’ goals, their experiences, their existing knowledge and their abilities. These systems do not have sufficient flexibility and interactivity. Consequently, there is less opportunity for receiving instant responses and feedback from the instructor when online learners need support. Considering the ability of software agents, which could provide expert advice and take on the role of someone who can listen to an explanation and interact with users, even if only in a limited manner, hence leads us to exploit them for designing learning platforms online. Therefore, adding interactivity and intelligence to Web educational applications is considered an important direction for future research (Baylari & Montazer, 2009; Gladun, García-Sánchez, Martínez-Béjar, & Fernández-Breis, 2009; Holmes, 2007).

E-learning is one of the main areas within which intelligent software agents propose great benefits. Indeed, one of the most useful tools with various functionalities and usages in e-learning is the Intelligent Software Agent. The main purpose of this paper is to identify the most important features and classification of agents through a comprehensive investigation of international research on the subject, covering a wide range of internationally accepted types of agents and their correspondent features in order to apply them to the electronic learning context as required. Such a classification is applicable to the domain of e-learning.

LITERATURE REVIEW

Intelligent agents have received considerable attention by researchers over the last few years. This field emerged as a result of the benefits of having applications with a technology that allows applications to decide for themselves what they need to do in order to satisfy their design objectives. One of the most commonly accepted definitions defines an agent as a computer system situated in an environment that is capable of autonomous action in order to meet its design objectives (García-Sánchez, Valencia-García, Martínez-Béjar, & Fernández-Breis, 2009; Xu & Wang, 2006). Researchers define the term as having the ability to provide simple autonomy, sociability, reactivity or pro-activeness (Tweedale, Ichalkaranje, Sioutis, Jarvis, Consoli, & Phillips-Wren, 2007).

The intelligent agent provides pro-active resource discovery, and offers value-added information services and products (Chou & Seng, 2009). Agent technology is appropriate for the development of complex and distributed applications in environments that incorporate numerous components with different expertise and conflicting interests (Garcıa-Sanchez et al., 2009). Plenty of investigations have been reported, such as information filtering and gathering, telecommunication systems, process control and electronic learning (Kuo, 2007). Researchers have proposed agents as computing systems that inhabit some complex dynamic environments and sense and act autonomously in the environment (Xu & Wang, 2006). Intel-ligent agents make use of a knowledge base and algorithms to carry out their responsibilities. A conceptualized or implemented understanding of them typically includes domain and problem solving knowledge specific to the agent’s focal tasks, control knowledge to bridge this domain and problem-solving knowledge, and software components to realize the system’s desired behaviours (Haynes, Cohen, & Ritter, 2009). After a user’s need has been identified (need identification), the agent acting on behalf of the user is involved (Louta, Roussaki, & Pechliva-nos, 2008). The agent automatically determines



the ideal behaviour within a specific context that maximizes teaching performance with respect to pre-defined measures (Haynes et al., 2009).

In recent years, electronic learning has become widespread, especially since standard-izing initiatives for learning technologies have begun. In fact, the time spent by teachers and moderators in e-learning courses is critical and costly in terms of resources. Thus, the augmentation of the number of students per teacher reduces the time that each teacher can dedicate to each student. This makes the learning process impersonal and not many teachers can keep track of their students’ progress. Informa-tion and Communication Technologies offer possibilities in learning assessment, because they allow the management of the information to be used in the evaluation process (Gladun et al., 2009). Other scholars believe that agent architecture is capable of linking aspects of perception, interpretation of natural language, learning and decision- making (Huang, Liang, Lai, & Lin, 2010). Finally, agents being sub-merged into a virtual urban environments can interact with each other and their environment in order to predict and evaluate the effects of different policy scenarios and plan alternatives (Sikora & Sachdev, 2008).

Supported development environments are available and design methodologies, reference architectures and standards are beginning to ap-pear in the agent zone (Tweedale et al., 2007). These all lead to personalised services for each learner in their e-learning environment, and such developments can be worthwhile to both sides. Personalization can ensure that the system will take into account the particular strengths and weaknesses of each individual who is using the program. Intelligent agent technologies facilitate interaction between students and the system and generate artificial intelligence models of learning, pattern recognition, learning styles and simulation, such as the student model, task model, pedagogical model, and repository technology. These models can work together in a productive way to support students’ learning activities and to adapt to them (Bergadano, Puliafito, Riccobene, Ruffo, & Vita, 1999; Schi-

affino, Garcia, & Amandi, 2008; Xu & Wang, 2006). In addition to providing expert advice, software agents can also take on the role of someone who can listen to an explanation and provide different kinds of instructional support such as reflection or elaboration feedback, even if only in a limited manner. The personas exhib-ited by software agents can have an important influence on student motivation. Agents that have the ability to develop or grow intellectually along with their student partner may provide one way to foster intrinsic motivation and help students take control of their own learning. This encourages developers to take advantage of the capability of a virtual world environment to provide software agents that are programmed with specific characteristics and personalities (Heidig & Clarebout, 2010; Holmes, 2007).

Intelligent agents combine both conflict detection and assistance to teachers through alerts. They are often integrated into web-based distance-learning platforms in order to support collaborative work in distance learning education. The agent uses an extensible set of rules to detect collaboration conflicts among students and notify teachers about them through on-screen alerts. Thus, the agent lightens the often difficult and time-consuming tasks that are carried out by teachers (Casamayor, Amandi, & Campo, 2009). The intelligent agents may possibly recognize learners’ needs by means of interacting with each one and making a profile or knowledge base of each learner. They respond to learners’ requisitions based on their circumstances and then provide the appropri-ate course, examination, help and tuition. In addition, they support personalized students’ learning activities by providing automated and personalized learning instruction to online learners. Explicitly, some of their properties, i.e., autonomy, pre-activity, pro-activity and cooperativity, support systems in recognizing online learners’ learning stage and in reacting with tailored instruction including personalized learning materials, tests and instant interactions (Baylari & Montazer, 2009, Xu & Wang, 2006). Intentionality and orienting the activity around



a problem-based teaching exercise is also an important factor to consider (Holmes, 2007).

Therefore, the properties of intelligent agent support systems which recognize online learners’ learning stages can react with tailored instruction including personalized learning ma-terials, tests, and instant interactions. Intelligent agent technologies are one of the best tools to facilitate the interaction between students and the systems which conduct adapted services. Consequently, researchers are clearly interested in developing systems, which are suitable for handling real online courses by using an adap-tive and personalized approach that is tailored to individual online learner needs by means of intelligent software agents and their features. It enables the electronic learning developers to provide online learners with an adaptive learn-ing plan and personalized learning instruction based on each individual’s background and ability (Xu & Wang, 2006).

The advantage of such an approach to implementation via an agent-based format in e-learning would be the reduction of repeti-tion in learning processes and also the better management of time for concentrating on the core processes of learning. The agents guide learners through online learning or they teach them about certain subjects. They enable users and learners to concentrate more on what they want regardless of how they should process it. So they could facilitate personalized learning services such as the appropriate explaining, monitoring, instruction, help seeking and learn-ing resources, and act as a learning partner and support for learners. This is why the intelligent agent has each learners profile in its knowledge base. In addition, the agent seeks the resources, provides the right learning resources to learn-ers in accordance with the condition and level of the learners, and makes a plan to conduct them through the learning process. The main features of agent in electronic learning can be summarized as follows:

(1) Motivation: Arouse interest, highlight the relevance of the topic, and strengthen the learner’s confidence;

(2) Information: Draw the learner’s attention to the learning content, activate prior knowl-edge, and enable the learner to integrate the new information and prior knowledge in to a new knowledge structure;

(3) Information processing: Provide explicit in-formation about prerequisites, conditions, relationships or outcomes of the learning content; enable learners to decompose new information in to smaller units, and to syn-thesize them in order to extract similarities and differences, and make instruction based on the resources and learners’ profile;

(4) Storing and retrieving: Guide learners to compare new and already stored informa-tion by reviewing similarities and differ-ences, as well as by integrating the new information in to the existing cognitive structure and learner profile;

(5) Transfer of information: Apply the new knowledge and transfer it to other topics and new problems, explain the progress to each of the learners and help them to promote themselves by presenting tailor-made advice;

(6) Monitoring and directing: Monitor the learners’ activities and guide them (Heidig & Clarebout, 2010).

E-LEARNING AGENT’S FEATURES

Intelligent agents are known as one of the data mining techniques in the field of e-learning as well as one of the major e-learning tools. Us-ing intelligent agents with the aim of providing e-learning would help us to create an adaptive and personalized learning environment for the individual learner. Intelligent software agents are capable of extracting the information about the learner at the time of his/her interaction with the system. Therefore, they could find out individual learning goals, experiences, previ-ous knowledge and skills in the field of his/her subject through continuous communication with the individual learner. They increase interaction with the individual learner in order to detect



users and learner’s behaviour and explore preferences, pattern recognition and learning models for each individual. They can make a profile of individuals and propose a variety of services based on each learner’s profile.

Intelligent agents could help learners in many aspects such as creating notification about the presence of new information regarding the classroom; periodically and continuously evaluating learners; being aware of their prog-ress; scheduling learners’ training curriculum and facilitating the interaction and contact of learners with experts. Therefore, individuals can manage their own learning process based on their specific training requirements. The agents can also act as an e-coach based on the individual’s educational level, provide guid-ance about the appropriate learning time of the learners and thereby accomplish large parts of the difficult tasks associated with individual instructor.

Conducting such functions entails each agent having certain characteristics. The lack of a wide and precise categorization of intelligent agent features in e-learning environments has been revealed after surveying a broad range of relevant research literature. Since the features have been provided in the literature, there has been an attempt to comprehensively review the research literature and collect the most common and applicable features of e-learning agents in a concise format. Table 1 presents a comprehensive set of major e-learning agent features as well as the references pertaining to each. The method for choosing each feature has been the highest frequency of usage within the literature:

These features enable the e-learning agents to act on behalf of the user and hence success-fully carry out their assigned tasks in an e-learning system in order to respond to specific requirements of the learners and tutors.

CLASSIFICATION OF E-LEARNING AGENTS

Although all e-learning agents must possess the above-mentioned characteristics and features, other characteristics such as their assigned tasks on a platform and their related functions in an e-learning system, encourage the authors to investigate the intelligent agents’ classifica-tion and categorization in electronic learning environments. In order to create the classifica-tion of the e-learning agents, there has been an attempt to comprehensively review the literature with the aim of extracting the most applicable agents from the most relevant published papers to facilitate e-learning efforts through virtual environments. By studying the broad literature, seven types of agents have been identified in the broader context of electronic learning. The process of identification has been conducted as follows:

• The most relevant and recent papers have been identified and extracted.

• Through a comprehensive review, the shared features and characteristics of agents have been grouped together. Since similar agents would resemble each other’s tasks in processing behind the scene for the us-ers of the e-learning system, they can be analyzed and grouped together according to their similar features.

• The process of grouping and naming of the agents has been based upon the frequency of the features in the literature. Therefore, the agents’ names have been chosen accord-ing to the important feature being covered by the agents.

Table 2 illustrates the most applicable e-learning agents with their detailed features and references according to their highest frequency of appearance in the research literature.



continued on following page

Table 1. Mostly applied e-learning agent features

Features Reference

Adaptability(Tweedale et al., 2007; Chang, 2008; Biswas, 2008; Montano, Yoon, Drummey, & Li-ebowitz, 2008; Pan, Leung, Moon, & Yeung, 2009; Lau, Li, Song, & Kwok, 2008; Guo & Zhang, 2010).

Asynchronous (Qureshi & Dawood, 2008).

Automatic (Saarloos, Arentze, Borgers, & Timmermans, 2008; Guo & Zhang, 2010).

Autonomy

(Bergadano et al., 1999; Karacapilidis & Moraitis, 2001; Klusch, 2001; Wang, 2001; Anumba, Ugwu, Newnham, & Thorpe, 2002; Chen & Wang, 2007; Shirazi & Soroor, 2007; Tweedale et al., 2007; Qureshi & Dawood, 2008; Chang, 2008; Biswas, 2008; Hall, Guo, Davis, & Cegielski, 2009; Keegan, O’Hare, & O’Grady, 2008; Beydoun, Low, Mouratidis, & Henderson-Sellers, 2009; Gao & Xu, 2009; Garcıa-Sanchez et al., 2009; Haynes et al., 2009; Pan et al., 2009; Phillips-Wren, Mora, Forgionne, & Gupta, 2009; Wang, Gwebu, Shanker, & Troutt, 2009; Guo & Zhang, 2010; Huang et al., 2010; Li & Li, 2010).

Bandwidth (He et al., 2009).

Continuous Operation (Wang, 2001; Chang, 2008; Biswas, 2008).

Flexibility (Gao & Xu, 2009; He et al., 2009; Yang, 2009; Zunino & Campo, 2009).

Goal-Oriented (Shirazi & Soroor, 2007; Chang, 2008; Biswas, 2008; Gao & Xu, 2009; García-Sánchez et al., 2009; Aarona & Admoni, 2010).

Having Knowledge Base (Guo & Zhang, 2010; Sueyoshi & Tadiparthi, 2008).

Intelligent and Make Reason

(Bergadano et al., 1999; Klusch, 2001; Goel, Zobel, & Jones, 2005; Kuo, 2007; Twee-dale et al., 2007; Chang, 2008; Biswas, 2008; Sokolova & Fernández-Caballero, 2009; Wang et al., 2009; Huang et al., 2010).

Co-Operation

(Bergadano et al., 1999; Klusch, 2001; Anumba et al., 2002; Shakshuki & Gong, 2005; Shirazi & Soroor, 2007; Kuo, 2007; Tweedale et al., 2007; Chang, 2008; Saarloos et al., 2008; Biswas, 2008; Keegan, O’Hare, & O’Grady, 2008; Beydoun et al., 2009; Gao & Xu, 2009; Hall et al., 2005; He et al., 2009; Kwon, 2009; Wang et al., 2009).

Learning(Bergadano et al., 1999; Anumba et al., 2002; Tweedale et al., 2007; Chang, 2008; Biswas, 2008; Keegan, O’Hare, & O’Grady, 2008; Saarloos et al., 2008; Montano et al., 2008; Gao, Shang, & Kokossis, 2009; Zunino & Campo, 2009; Guo & Zhang, 2010).

Mobility(Klusch, 2001; Wang, 2001; Kuo, 2007; Tweedale et al., 2007; Biswas, 2008; Keegan, O’Hare, & O’Grady, 2008; Beydoun et al., 2009; García-Sánchez et al., 2009; Pan et al., 2009; Phillips-Wren et al., 2009).

Portability (Qureshi & Dawood, 2008).

Pro-Activeness

(Karacapilidis & Moraitis, 2001; Klusch, 2001; Chen & Wang, 2007; Shirazi & Soroor, 2007; Biswas, 2008; Keegan, O’Hare, & O’Grady, 2008; Beydoun et al., 2009; Gao & Xu, 2009; Garcıa-Sanchez et al., 2009; Hall et al., 2009; Phillips-Wren et al., 2009; Wang et al., 2009; Guo & Zhang, 2010; Li & Li, 2010).

Rationality (García-Sánchez et al., 2009).



DISCUSSION

In spite of the importance of applying e-learning agents in making personalized learning services, there has been a lack of comprehensive catego-rization of e-learning agents and their related features. By broadly reviewing the available literature, there has been an attempt to impart an informative framework of the e-learning agent’s features. The agents collaborate with each other in order to provide intelligence to electronic educational applications for the purpose of facilitating the usage of systems. This has led to enough flexibility for providing personalized learning mechanisms and helping learners to continuously learn and progress without much problem in configuring their routine or repetitive needs and wants within the virtual environment of learning. The classification helps in finding the most relevant agent for the most suitable job within the e-learning environment. Such a classification of agents and their related features also helps academic scholars and practitioners have a better understanding of the structure and the variety of available agents according to agents’ characteristics. This classification also provides the system designers with a firm ground and insight for better analyzing and designing e-learning systems while consider-ing the most useful agents for each part of the these systems. Consequently, the possibility of

establishing well-written e-learning software would increase significantly.

Exploring through such a pervasive set of features and their corresponding agents, help the practitioners differentiate the educational system usages by providing the users with a more flexible, fast and encompassing system which is less required to be configured by each of the students. The agents will also automati-cally provide the students with many new items to consider. For example, they can provide the students with a wide range of new scientific information on the interested fields or inform them of the most relevant conceptual or empiri-cal advances in their field of study. They also assist the scholars in building deeper and more pervasive ontology of agents according to their respective features.

Another important role of agents has been identified in terms of the security provision role. Such agents might accompany the automatic self-learning mechanisms with which they become capable of addressing the security is-sues in an improving manner. Without such an important capability, the system administrators should constantly watch for the inconsistencies or network problems and risks of infection with viruses in e-learning systems. Another important role of these agents can be charac-terized in granting or forbidding students from a specific feature of the e-learning system or from accessing critical data (for example, for

Reactivity

(Klusch, 2001; Barthes & Tacla, 2002; Chen & Wang, 2007; Shirazi & Soroor, 2007; Qureshi & Dawood, 2008; Keegan, O’Hare, & O’Grady, 2008; Beydoun et al., 2009; Gao & Xu, 2009; García-Sánchez et al., 2009; Hall et al., 2009; Wang et al., 2009; Guo & Zhang, 2010).

Reliability (Bergadano et al., 1999).

Remote Access to Sources (Qureshi & Dawood, 2008).

Sensitive to its Environment (Sahba, Tizhoosh, & Salama, 2008; Beydoun et al., 2009; Gao & Xu, 2009).

Social Ability (Klusch, 2001; Chen & Wang, 2007; Shirazi & Soroor, 2007; García-Sánchez et al., 2009; Guo & Zhang, 2010; Hall et al., 2009).

Temporal Continuity (Shirazi & Soroor, 2007; García-Sánchez et al., 2009).

Usefulness (Ease of Use) (Chang, 2008).




Table 2. Mostly applied e-learning agent classification

Agent FeaturesandtheirReferences

Interface Agent

- Interface with learners (Bergadano et al., 1999) - Interact directly with the learners, facilitates designer/agent interaction (Anumba et al., 2002) - Learners can login the theme-based learning system through a user interface agent to partici-pate in the learning activities (Huang, Liang, Lai, & Lin, 2007) - Maintains inputs and outputs of the system, interacting with the participant (Chen & Wang, 2007) - Provide a communication interface to the learners (Chen, Jeng, Lee, & Chuang, 2008) - Translate web services’ requests to agent communication language encodings, enable connec-tion between agents (Chen & Chen, 2008) - learner assistance, providing alternative views, context-sensitive suggestions and critics, predict the needs of the learners (Saarloos et al., 2008) - Provide interaction between learners and the system via keyword searches or natural language queries, Interface with learners (Montano et al., 2008) - Personalizable interfaces between a learners and a system, enjoyment, personal innovativeness (Serenko, 2008) - Records learners’ activities, duration of a particular task, documents load/unload, etc. and stores them in the learners’ profile (Baylari & Montazer, 2009) - Enable learners to view the current state of the system, enable the corresponding learners to issue requests to the other agents in the system (Gao & Xu, 2009) - Receive tasks from the learners and return results (Gao et al., 2009) - Explanation capabilities (Haynes et al., 2009) - Provide a personalization interaction, assistance, records interaction information and related feedback in the learner model and flexible interaction interface (Yang, 2009)

DB Agent

- Locate files, search in the local database, locate web sites address where to conduct the search (Bergadano et al., 1999) - Inform the learners about the arrival of data, data availability, applying algorithms for deciding which data, when and how to send them, produce consistency (Qureshi & Dawood, 2008) - Collect all learners’ information from the e-learning portal, encode learners information into the XML format (Chou & Seng, 2009) - Collect and analyze the raw process operation data, make the result available by other agents (Gao et al., 2009) - Perform data pre-processing procedures, outliers and anomalies detection in data, deal with missing values, smooth normalization, oriented to data fusion, initial information reception and meta-data creation (Sokolova & Ferna´ndez-Caballero, 2009) - Manage the database, ensure the consistency of the data, inform concerned agents of the data change (Wang et al., 2009)

Facilitator Agent

- Provide pertinent information desired by agents registered with the system, send contract mes-sages to involved parties (Goel et al., 2005) - Track the progress of each real learner and tries to find learners with similar learning interests and experience (Yang, Wang, Shen, & Han, 2007) - Broadcast the draft plan to interested learners over the e-learning platform, collects the prefer-ences/constraints of each individual participant (Chen et al., 2008) - Interpret queries from requester, matches capabilities provided by other agents (Chen & Chen, 2008) - Forward learners queries directly to the Machine Learning (Montano et al., 2008) - Task decomposition and coordinating of other agents (Gao et al., 2009) - Monitor and ensure a correct functioning of the e-learning platform, control and balance the workload (García-Sánchez et al., 2009) - Visualize data and deliver to the learners (Sokolova & Fernández-Caballero, 2009) - Support the localization of agents and the services they provide in the area of a domain or the whole environment (Yang, 2009)



Customer/ Client Agent

- Abstracts the learners’ learning model, based on the his/ her pattern and profile (Xu & Wang, 2006) - Make customized report or training for the learners (Shirazi & Soroor, 2007) - Compute the learner’s potentiality index according to the learner’s profile data, regularly adjust parameters according to the pattern of learner characteristic (Chan, 2007) - Generates the learning profiles of learners (Huang et al., 2007) - Provides personalized recommendations to learners depending on their profile and on their performance with a certain Web-based course (Schiaffino et al., 2008) - Maintain learners preferences, indicate them for the option items be considered in their learn-ing plan (Chen et al., 2008) - Find and execute preference requests and services and deliver results to the learners (Chen & Chen, 2008) - Awareness of the communities to which learners belong, use this awareness to fulfil their learners’ search requests efficiently and effectively (Beydoun et al., 2009) - Selects the suitable course from database and presents it to the learners based on learner ability from his/her profile (Baylari & Montazer, 2009) - Assesses a wide variety of detailed information relating to the learner’s account, provide a single view of the learner profile (Gao & Xu, 2009) - Receive tasks from the learners and return results (Gao et al., 2009) - Have learners’ preferences (Kwon, 2009) - Possess a number of learners’ specified preferences, explain its actions to the learner (Zunino & Campo, 2009)

Security Agent

- Detect intrusive accesses, identify the actual sender, check message integrity, signature the message sender, control for instruction, authentication, no repudiation, access control, confiden-tiality, authentication (Bergadano et al., 1999) - Listen to inter-agent communication, refusing to access to private data of an agent or learner (Klusch, 2001) - Responsible for broad but not exhaustive safety principles, returns suggestions to meet mini-mum standards (Anumba et al., 2002) - Accurately verify the identity of a learners, apply authorization, encryption, decryption, digital signature and veri□cation (Shakshuki & Gong, 2005) - Monitor & analyze the inspection agent’s information attack reports, evaluate the threat value, take corresponding actions according to the network system’s security policy, adapt to unpre-dicted complex evolution, report attack information/warning to other agents (Kuo, 2007) - Check confidentiality, integrity, authentication and availability in the platform (Beydoun et al., 2009) - Follow the interaction processes of the multi-agent mechanism, supervise the flow of informa-tion (Chou & Seng, 2009) - Monitor the activities on a risk-based assessment, risk-assessment report will be automatically produced, identify transactions with greatest risk, present a suspicious activity report to the ap-propriate compliance learner to investigation and action (Gao & Xu, 2009) - Monitor, detect, and notify operation situation or abnormal events that need to be responded to (Gao et al., 2009) - Explores unknown relationships & trends in data (Sokolova & Ferna´ndez-Caballero, 2009)

Information Agent

Information Retrieval (Bergadano et al., 1999) - Handle incomplete, conflicting information during a transaction (Karacapilidis & Moraitis, 2001) - Information acquiescing, managing, analyzing, manipulating, fusing heterogeneous informa-tion, guiding the learners through the available, individual information space (Klusch, 2001) - Extract relevant information, update knowledge from a knowledge repository, distribute knowledge, monitor changes that occurred in knowledge repository of each learner (Gao et al., 2009)




the mid-term or final exams and the related answers). There are also software agents that take the responsibility of management or coor-dination of other agents in order to optimize the performance of the systems for the users. They possess an authoritarian role for orchestrating the efforts of agents toward the specified goals as assigned by the system administrator. In this way, the teacher’s efforts in the e-learning domain will be reduced to a minimum for more important managerial and coordinator roles and the agents in charge will do a much larger part of the work. Other important roles of agents can be identified in the interface and database activities. The interface might be personalized according to the needs of users. In addition, the requirements of connecting and fetching data from databases might dramatically differ in essence due to differences in the levels of training and learning.

Studying such features help e-learning system analysts, designers, quality control per-sonnel and developers to gain a better insight into which agents to choose and how best to take advantage of them in terms of successful deploy-ment and utilization. The final point to highlight is that e-learning agents ought to be streamlined toward the needs of students and requirements of learning in virtual learning environments in which the control and management of students, provision of useful resources, security issues, access grants for the selected students, and other specific issues of teaching and learning, are of importance to the teachers, students, analysts and developers of such systems. There are also

many other agents which conduct various tasks in other types of information systems which are certainly of less advantage to this study. Thus, they have not been considered as relevant to this specific attempt. An overall classification of all the agents without proper attention to the final usage of agents creates the risk of less ac-curacy and precision in providing the interested researchers with the most helpful and influential requirements of developing and utilizing an effective system of virtual learning.

CONCLUSION

Software agents have been considered a major technique for solving network and business problems and for enhancing the systems performance with respect to computational ef-ficiency, flexibility and reusability. They have been widely applied to many network-based applications. One of the major applications of intelligent agents has been recognized to be in electronic learning since it can provide personalized services by containing specific features as has been comprehensively illustrated in the paper.

Since there have been few attempts to provide a broad categorization of e-learning agents in terms of well-established systems of classification this paper has sought to provide a map of the area that may be beneficial to re-searchers. Thus we have considered the features that make e-learning agents more effective in providing personalized learning services to

Agent Manager

- Manage dependencies between activities of one or more agent/actor, avoid conflict through transparency (Klusch, 2001) - Provide the supervisory control on related inputs and outputs of the activity associated with the agents and assure consistency and performance of the system (Chen & Wang, 2007) - Utilize process models to seek learning process, utility system optimization and scheduling task (Gao et al., 2009) - Avoid system resources overloading, monitor the status of all the interactions, solving interop-erability issues (García-Sánchez et al., 2009) - Act as a human secretary, organizes calendar, schedules meetings, negotiates dates, places and times of each learner with others (Zunino & Campo, 2009) - Responsible for managing the process of distributed learning and the feature selection coordi-nation (Czarnowski & Jedrzejowicz, 2011) - Manage operations on the agent platform, managing the agents themselves (Yang, Sung, Wu, & Chen, 2010)



learners and teachers while keeping in mind the high standards of quality required for learning mechanisms to be effective and efficient.

In addition, the most applicable e-learning agents have been identified by this significant review of available research. Intelligent soft-ware technology features and functionalities make them interesting to be applied in different implementations of e-learning systems. The agents help in many aspects of electronic learn-ing such as automatic recognition of students’ actions and by enabling the teachers to focus deeply on more important aspects of learning while also delegating the ordinary activities to agents. They also guide learners toward the best practices of learning; simulate the process of col-laboration and mutual understanding of learners; work on behalf of the teacher and ease some of the routine responsibilities; automatically and even confidentially gather information for the purpose of assessment while the students are involved in solving the problems, and so on. In addition, they themselves have the ability to learn and adjust; they can gracefully degrade themselves and continue to work if they do not manage to access the required resources for full functionality. These characteristics make them worthwhile in terms of being implemented and used in e-learning systems, especially in the web-based versions.

Virtual learning students might also take advantage of the classification in that they be-come familiar with the usage of such agents and the features they can provide for the students. Hence, there are various attempts to utilize the agents within the new systems of e-learning. The familiarity of students with the characteristics and advantages of interface, information, secu-rity, and other interacting agents would enable them to best exploit the given opportunities toward a more effective process of learning.

Further works might include studies on more complex interactions among the e-learning agents in a multi-agent system within which the agents collaboratively interact and process the needs toward the same goals for the best pos-sible functionality. They can constitute a firm ground for using and developing a multi-agent

application followed by field experiments. In particular, further investigation of the com-munication and interaction of various agents would be an appropriate field of study so as to acquire a deeper understanding of a suitable and applicable architecture of relevant agents for e-learning systems. Implementation of a sound architecture would create the opportunity to take advantage of this beneficial technology in the virtual world of e-learning. Another sug-gestion of the authors would be the creation of a comprehensive ontology of agents in the e-learning field. It is hoped that the system of classification outlined in this paper could pro-vide the basis for this prospective ontology of e-learning agents.

REFERENCES

Aarona, E., & Admoni, H. (2010). Action selection and task sequence learning for hybrid dynamical cognitive agents. Robotics and Autonomous Systems, 58(9), 1049–1056. doi:10.1016/j.robot.2010.05.006

Anumba, C. J., Ugwu, O. O., Newnham, L., & Thorpe, A. (2002). Collaborative design of structures using intelligent agents. Automation in Construction, 11(1), 89–103. doi:10.1016/S0926-5805(01)00055-3

Barthes, J. P. A., & Tacla, C. A. (2002). Agent- sup-ported portals and KM in complex R&D projects. Computers in Industry, 48(1), 3–16. doi:10.1016/S0166-3615(02)00006-4

Baylari, A., & Montazer, Gh. A. (2009). Design a personalized e-learning system based on item re-sponse theory and artificial neural network approach. Expert Systems with Applications, 36(4), 8013–8021. doi:10.1016/j.eswa.2008.10.080

Bergadano, F., Puliafito, A., Riccobene, S., Ruffo, G., & Vita, L. (1999). Java-based and secure learning agents for information retrieval in distributed system. Information Sciences, 113(1-2), 55–84. doi:10.1016/S0020-0255(98)10041-5

Beydoun, G., Low, G., Mouratidis, H., & Henderson-Sellers, B. (2009). A security-aware meta model for multi-agent systems (MAS). Information and Software Technology, 51(5), 832–845. doi:10.1016/j.infsof.2008.05.003



Biswas, P. K. (2008). Towards an agent-oriented approach to conceptualization. Applied Soft Comput-ing, 8(1), 127–139. doi:10.1016/j.asoc.2006.11.009

Casamayor, A., Amandi, A., & Campo, M. (2009). Intelligent assistance for teachers in col-laborative e-learning environments. Computers & Education, 53(4), 1147–1154. doi:10.1016/j.compedu.2009.05.025

Chang, H. (2008). Intelligent agent’s technology char-acteristics applied to online auctions’ task: A com-bined model of TTF and TAM. Technovation, 28(9), 564–577. doi:10.1016/j.technovation.2008.03.006

Chen, D. N., Jeng, B., Lee, W. P., & Chuang, C.-H. (2008). An agent-based model for consumer-to-business electronic commerce. Expert Systems with Applications, 34(1), 469–481. doi:10.1016/j.eswa.2006.09.020

Chen, R. S., & Chen, D. K. (2008). Apply ontology and agent technology to construct virtual observatory. Expert Systems with Applications, 34(3), 2019–2028. doi:10.1016/j.eswa.2007.02.018

Chen, Y. M., & Wang, S. C. (2007). Framework of agent-based intelligence system with two-stage decision-making process for distributed dynamic scheduling. Applied Soft Computing, 7(1), 229–245. doi:10.1016/j.asoc.2005.04.003

Chou, T. H., & Seng, J. L. (2009). An intelligent multi-agent e-services method—An international telecom-munication example. Information & Management, 46(6), 342–350. doi:10.1016/j.im.2009.05.006

Czarnowski, I., & Jedrzejowicz, P. (2011). An agent-based framework for distributed learning. Engineering Applications of Artificial Intelligence, 24(1), 93–102. doi:10.1016/j.engappai.2010.07.003

Gao, S., & Xu, D. (2009). Conceptual modeling and development of an intelligent agent-assisted decision support system for anti-money laundering. Expert Systems with Applications, 36(2), 1493–1504. doi:10.1016/j.eswa.2007.11.059

Gao, Y., Shang, Z., & Kokossis, A. (2009). Agent-based intelligent system development for deci-sion support in chemical process industry. Expert Systems with Applications, 36(8), 11099–11107. doi:10.1016/j.eswa.2009.02.078

García-Sánchez, F., Valencia-García, R., Martínez-Béjar, R., & Fernández-Breis, J. T. (2009). An ontology, intelligent agent-based framework for the provision of semantic web services. Expert Systems with Applications, 36(2), 3167–3187. doi:10.1016/j.eswa.2008.01.037

Gladun, J. R., García-Sánchez, F., Martínez-Béjar, R., & Tomás Fernández-Breis, J. (2009). An application of intelligent techniques and semantic web technolo-gies in e-learning environments. Expert Systems with Applications, 36(2), 1922–1931. doi:10.1016/j.eswa.2007.12.019

Goel, A. C., Zobel, W., & Jones, E. C. (2005). A multi-agent system for supporting the electronic contracting of food grains. Computers and Electron-ics in Agriculture, 48(2), 123–137. doi:10.1016/j.compag.2005.02.016

Guo, Q., & Zhang, M. (2010). An agent-oriented approach to resolve scheduling optimization in intel-ligent manufacturing. Robotics and Computer-inte-grated Manufacturing, 26(1), 39–45. doi:10.1016/j.rcim.2009.02.003

Hall, D., Guo, Y., Davis, R. A., & Cegielski, C. (2005). Extending unbounded systems thinking with agent-oriented modeling: conceptualizing a multiple perspective decision-making support system. Deci-sion Support Systems, 41(1), 279–295. doi:10.1016/j.dss.2004.06.009

Haynes, S. R., Cohen, M. A., & Ritter, F. E. (2009). Designs for explaining intelligent agents. Interna-tional Journal of Human-Computer Studies, 67(1), 90–110. doi:10.1016/j.ijhcs.2008.09.008

He, J., Lai, H., & Wang, H. (2009). A commonsense knowledge base supported multi-agent architecture. Expert Systems with Applications, 36(3), 5051–5057. doi:10.1016/j.eswa.2008.06.008

Heidig, S., & Clarebout, G. (2011). Do pedagogical agents make a difference to student motivation and learning? A review of empirical research. Educa-tional Research Review, 6(1), 27–54. doi:10.1016/j.edurev.2010.07.004

Holmes, J. (2007). Designing agents to support learning by explaining. Computers & Education, 48(4), 523–547. doi:10.1016/j.compedu.2005.02.007

Huang, C. C., Liang, W. Y., Lai, Y. H., & Lin, Y. C. (2010). The agent-based negotiation process for B2C e-commerce. Expert Systems with Applications, 37(1), 348–359. doi:10.1016/j.eswa.2009.05.065

Huang, C. J., Liu, M. C., Chu, S. S., & Cheng, C. L. (2007). An intelligent learning diagnosis system for Web-based thematic learning platform. Comput-ers & Education, 48(4), 658–679. doi:10.1016/j.compedu.2005.04.016



Qureshi, K. A., & Dawood, M. Z. (2008). Intelligent ubiquitous computing and e-Business solution. Inter-national Journal of Information Management, 28(2), 128–135. doi:10.1016/j.ijinfomgt.2007.06.003

Saarloos, J. M., Arentze, T. A., Borgers, A. W. J., & Timmecrmans, H. J. P. (2008). A multi-agent paradigm as structuring principle for planning sup-port systems. Computers, Environment and Urban Systems, 32(1), 29–40.

Sahba, F., Tizhoosh, H. R., & Salama, M. M. M. A. (2008). A reinforcement agent for object seg-mentation in ultrasound images. Expert Systems with Applications, 35(3), 772–780. doi:10.1016/j.eswa.2007.07.057

Schiaffino, S., Garcia, P., & Amandi, A. (2008). eTeacher: Providing personalized assistance to e-learning students. Computers & Education, 51(4), 1744–1754. doi:10.1016/j.compedu.2008.05.008

Serenko, A. (2008). A model of user adoption of interface agents for email notification. Interacting with Computers, 20(4-5), 461–472. doi:10.1016/j.intcom.2008.04.004

Shakshuki, Z. L., & Gong, J. (2005). An agent-based approach to security service. Journal of Network and Computer Applications, 28(3), 183–208. doi:10.1016/j.jnca.2004.06.004

Shirazi, M. A., & Soroor, J. (2007). An intelligent agent-based architecture for strategic information system applications. Knowledge-Based Systems, 20(8), 726–735. doi:10.1016/j.knosys.2006.10.004

Sikora, R. T., & Sachdev, V. (2008). Learning bidding strategies with autonomous agents in environments with unstable equilibrium. Decision Support Systems, 46(1), 101–114. doi:10.1016/j.dss.2008.05.005

Sokolova, M. V., & Fernández-Caballero, A. (2009). Modeling and implementing an agent-based envi-ronmental health impact decision support system. Expert Systems with Applications, 36(2), 2603–2614. doi:10.1016/j.eswa.2008.01.041

Sueyoshi, T., & Tadiparthi, G. R. (2008). An agent-based decision support system for wholesale electricity market. Decision Support Systems, 44(2), 425–446. doi:10.1016/j.dss.2007.05.007

Tweedale, J., Ichalkaranje, N., Sioutis, C., Jarvis, B., Consoli, A., & Phillips-Wren, G. (2007). Inno-vations in multi-agent systems. Journal of Network and Computer Applications, 30(3), 1089–1115. doi:10.1016/j.jnca.2006.04.005

Karacapilidis, N., & Moräıtis, P. (2001). Building an agent-mediated electronic commerce system with decision analysis features. Decision Support Systems, 32(1), 53–69. doi:10.1016/S0167-9236(01)00100-2

Keegan, S., O’Hare, G. M. P., & O’Grady, M. J. (2008). Easishop: Ambient intelligence assists everyday shopping. Information Sciences, 178(3), 588–611. doi:10.1016/j.ins.2007.08.027

Klusch, M. (2001). Information agent technol-ogy for the Internet: A survey. Data & Knowledge Engineering, 36(3), 337–372. doi:10.1016/S0169-023X(00)00049-5

Kuo, M. H. (2007). An intelligent agent-based collaborative information security framework. Expert Systems with Applications, 32(2), 585–598. doi:10.1016/j.eswa.2006.01.053

Kwon, O. (2009). A two-step approach to build-ing bilateral consensus between agents based on relationship learning theory. Expert Systems with Applications, 36(9), 11957–11965. doi:10.1016/j.eswa.2009.03.070

Lau, R. Y. K., Li, Y., Song, D., & Kwok, R. C. W. (2008). Knowledge discovery for adaptive negotia-tion agents in e-marketplaces. Decision Support Sys-tems, 45(2), 310–323. doi:10.1016/j.dss.2007.12.018

Li, S., & Li, J. Z. (2010). Agents international: In-tegration of multiple agents, simulation, knowledge bases and fuzzy logic for international marketing decision-making. Expert Systems with Applications, 37(3), 2580–2587. doi:10.1016/j.eswa.2009.08.022

Louta, M., Roussaki, I., & Pechlivanos, L. (2008). An intelligent agent negotiation strategy in the elec-tronic marketplace environment. European Journal of Operational Research, 187(3), 1327–1345. doi:10.1016/j.ejor.2006.09.016

Montano, R., Yoon, V., Drummey, K., & Liebowitz, J. (2008). Agent learning in the multi-agent contracting system [MACS]. Decision Support Systems, 45(1), 140–149. doi:10.1016/j.dss.2007.12.013

Pan, A., Leung, S. Y. S., Moon, K. L., & Yeung, K. W. (2009). Optimal reorder decision-making in the agent-based apparel supply chain. Expert Systems with Applications, 36(4), 8571–8581. doi:10.1016/j.eswa.2008.10.081

Phillips-Wren, G., Mora, M., Forgionne, G. A., & Gupta, J. N. D. (2009). An integrative evaluation framework for intelligent decision support systems. European Journal of Operational Research, 195(3), 642–652. doi:10.1016/j.ejor.2007.11.001



Wang, J., Gwebu, K., Shanker, M., & Troutt, M. D. (2009). An application of agent-based simulation to knowledge sharing. Decision Support Systems, 46(20), 532–541. doi:10.1016/j.dss.2008.09.006

Wang, S. (2001). Toward a general model for web-based information systems. International Journal of Information Management, 21(5), 385–396. doi:10.1016/S0268-4012(01)00029-9

Xu, D., & Wang, H. (2006). Intelligent agent sup-ported personalization for virtual learning environ-ments. Decision Support Systems, 42(2), 825–843. doi:10.1016/j.dss.2005.05.033

Yang, F., Wang, M., Shen, R., & Han, P. (2007). Community-organizing agent: An artificial intelligent system for building learning communities among large numbers of learners. Computers & Education, 49(2), 131–147. doi:10.1016/j.compedu.2005.04.019

Yang, S. Y. (2009). Developing of an ontological interface agent with template-based linguistic pro-cessing technique for FAQ services. Expert Systems with Applications, 36(2), 4049–4060. doi:10.1016/j.eswa.2008.03.011

Yang, Y. J., Sung, T. W., Wu, C., & Chen, H. Y. (2010). An agent-based workflow system for enter-prise based on FIPA-OS framework. Expert Systems with Applications, 37(1), 393–400. doi:10.1016/j.eswa.2009.05.049

Zunino, A., & Campo, M. (2009). Chronos: A multi-agent system for distributed automatic meet-ing scheduling. Expert Systems with Applications, 36(3), 7011–7018. doi:10.1016/j.eswa.2008.08.024

Mandana Farzaneh is currently a postgraduate student in the department of information technol-ogy management, School of Management, University of Tehran, Iran. She will receive her PhD degree in July 2011. She received her bachelor’s degree in industrial management from the Uni-versity of Tehran. She was accepted as one of the 100 top students in Tehran University in 2009.

Iman Raeesi Vanani is a PhD student in Information Technology Management in the College of Management, University of Tehran, Iran. He received his MS in IT Management from the Col-lege of Management, University of Tehran and he received his BA in public administration from Allame Tabatabaie University. His research interests include Knowledge Management, Decision Support Systems, Enterprise Resource Planning, Software Development, Business Intelligence, and Information Systems Management.

Babak Sohrabi is an associate professor in the Department of information technology manage-ment, faculty of management at University of Tehran, Iran. He received his PhD in management science, from Lancaster University, England in 2000. He is an Editor-in-Chief of Information Technology Management Journal at the University of Tehran and Head of the E-learning center of the University. His research interests include artificial intelligence, knowledge management, e-commerce, KM, decision support systems and the organizational impact of information technol-ogy, and he has published in a number of leading international journals in these fields.



Second Language Distance Learning and Teaching: Theoretical Perspectives and Didactic ErgonomicsJean-Claude Bertin, Patrick Gravé, and Jean-Paul Narcy Combes© 2010 IGI Global391 pp.$180.00ISBN: 9781615207077

Although Molina et al. (2007) lament the “lack of a methodological framework” for the model-ling and design of computer-mediated tools, the last decade in particular has seen an outpouring of information on theories for learning and instructional design for distance and/or online courses, and advice for the instructor who wants to dip his or her toes into the ocean that I call ICT, or Information and Communication Tech-nology. Molina et al. (2007) present their own methodology, which they call “Collaborative

Interactions Application Notations” (or CIAN). Salmon (2003) has the “key” to e-learning. Park and Bracewell (2008) refer to “CHAT.” Beldarrain (2006) refers to “learner-centered constructivist” environments as a perspective in distance learning. Jacques and Salmon (2007) look towards “team-based learning.” We thus find ourselves immersed in buzz words like “meaningful learning,” “paradigm shifts” and of course, particularly for distance learning, “flexibility.” Having looked at distance learn-ing models from as far back as 1972, Moore in 2007, reassured us that “there is room for more than one theory.” As he says, “The whole point and purpose of distance education theory is to summarize the different relationships and strength of relationship among and between ... variables.” In addition to the concept of em-bracing complex, multiple theories for distance learning, Park and Bracewell also postulate the idea that, as theories are changing, we should adopt a Cultural-Historical Activity Theory, which they call CHAT (2008). They explain

Second Language Distance Learning and Teaching:Theoretical Perspectives and

Didactic ErgonomicsIffaf Kahn, University of Essex, UK

DOI: 10.4018/jvple.2012010105



this as follows: “We believe that to cultivate best practices of ICTs-supported learning en-vironments, researchers should actively engage in undergoing transformation processes of the actual activity system in question and construct new models of activity with the local partici-pants” (2008, p. 109)

In Second Language Distance Learning and Teaching: Theoretical Perspectives and Didactic Ergonomics (2011) Bertin, Gravé, and Narcy Combes elicit their point of view about these theories by asking us: “How can the complex reality of distance language learning be accounted for?” They direct the reader to a model that they call “didactic ergonomics,” where educators observe and interpret forms of instruction and also look at effective design. Since ergonomics may be defined as “the ap-plication of scientific information concerning humans ... to the design of objects, systems and environments,” Bertin et al. (2011) are essentially relating human factors to choices about and the design of technologies for second language distance learning. This is what makes this an invaluable book. The writers approach theories with a critical eye, analyse the factors and forms of learning, and then, more impor-tantly in my opinion, provide a framework for how to operate distance learning.

The authors analyse second language acqui-sition and their conclusions from the analyses support the distance learning designer or teacher, whether or not they are teaching English as a second language. As an example, the reader may look at Narcy Combes’s statements that the “construction of knowledge is an individual, situated phenomenon as well as a social one” and that “Meaningful interaction will trigger learning processes (p. 96).” They may seem like set phrases but Narcy Combes goes on to propose that a social construct does not have to be completely authentic, as long as the tasks in themselves are realistic and interactive. The “the actual real-world tasks” are, what Combes calls “macro tasks (p. 98).”

Two important considerations in the con-struction of distance learning are synchronous and asynchronous tasks and learning because

of the way they affect the design of the course, the content, the type of input and the way the teacher provides either input or feedback. Bertin and Gravé remind us (p. 23) that “asynchronous interaction makes the materials design phase more complex as no immediate human retroac-tion and feedback can take place and significant differences may then appear between didactic intention and actual learner practice.” This is not to say there is a sense of isolation (Wegerif, 1998; Rovai, 2002) but the social dimensions of asynchronous learning are in flux, as are the emotions of the students and the teachers. As Combes says, there may be anger because of “what is perceived and what was meant by the interlocutor.” Having found students react more strongly than they would in synchronous and/or face-to-face learning, I would iterate Combes’s observation, with one of my students who observed that “The world is flat!”

Inevitably, with asynchronous learning comes autonomy because the learner has to navigate their way round the technology and the material without the immediate help of the teacher, even although they may have the sup-port of the teacher. The authors’ approach to this issue is pertinent - through the introduction of the concept of “interdependence” (p. 115). They are referring to how the learner has to make his/her own decisions about the direction to take but how the learner may also need to have, what Narcy Combes calls “metacogni-tive and cognitive awareness” or an ability to understand what they are reading or doing. The addition of the support from the teacher provides the interdependence. This translates into the teacher letting the learners know how to work their way through the course and the learners themselves becoming more aware of the possibilities. The learner can develop as a learner, using critical thinking skills to negoti-ate their way round content that they are not familiar with and then process it. Narcy Combes suggests that: “This may initially trigger off more or less conscious resistance which may be countered by some form of learner training in the form of reflective interaction.”



In the authors’ opinion, distance learn-ing is more than just an addition to learning in the classroom – it is a multi-dimensional and complex process in itself. The teacher and the learner are not interacting in a traditional manner, but, instead “new roles” are emerging “and new requirements (need) to be defined in order to ensure the integration and adoption of innovation and distance in and by the various specific contexts.” These become learning and teaching models, even if there is “no unifying paradigm” (Combes, p. 252). In turn, the dis-tance course designer may be able to reinterpret this new role for the teacher sitting in front of their students in the traditional classroom. In their own words: “distance as an analyzer clearly points to blended learning as a positive alterna-tive to traditional face-to-face environments.”

REFERENCES

Beldarrain, Y. (2006). Distance education trends: Integrating new technologies to foster student interac-tion and collaboration. Alumni - ProQuest Education Journals, 27(2), 139-153.

Hung, D., & Khine, M. S. (Eds.). (2006). Engaged learning with emerging technologies. Dordrecht, The Netherlands: Springer-Verlag. doi:10.1007/1-4020-3669-8

International Organisation for Standardisation Glossary of Terms. (n. d.). Consumers standards. Retrieved March 20, 2011, from http://www.iso.org/sites/ConsumersStandards/en/5-glossary-terms.htm

Jacques, D., & Salmon, G. (2007). Learning in groups: A handbook for face-to-face and online environments. London, UK: Routledge.

Molina, A. I., Giraldo, W. J., Redondo, M. A., & Ortega, M. (2007). A proposal of integration GUI development of groupware applications into the software development process. Computer Science, 4715, 111–126.

Moore, M. G. (Ed.). (2007). Handbook of distance education. Mahwah, NJ: Lawrence Erlbaum.

Rovai, A. P. (2002). Sense of community, perceived cognitive learning, and persistence in asynchro-nous learning networks. The Internet and Higher Education, 5(3), 197–211. doi:10.1016/S1096-7516(02)00102-1

Salmon, G. (2003). E-moderating: The key to teach-ing and learning online. New York, NY: Routledge Falmer.

Wegerif, R. (1998). The social dimension of asynchro-nous learning networks. Journal of Asynchronous Learning Networks, 2(1), 34–49.

Zumbach, J., Schwartz, N., Seufert, T., & Kester, L. (Eds.). (2008). Beyond knowledge the legacy of competence: Meaningful computer-based learning environments. New York, NY: Springer.

Iffaf Kahn currently works for a department in the University of Essex – the International Acad-emy. Primarily, she works on best practice in the use of learning technologies and has been designing online modules for her department. She is lead tutor on the pre-sessional e-module and project leader for a new teacher training module, that she designed herself, that focuses on learning technologies for 30 credits towards a post-graduate qualification. She also spends time in virtual reality – by teaching through Second Life. At iffaf.com, she blogs on computer-mediated learning and teaching. She has a masters in english and professional teaching qualifications.

MissionThe mission of the International Journal of Virtual and Personal Learning Environments (IJVPLE) is to study and disseminate research about the design, development, and evaluation of online learning environ-ments. IJVPLE is committed to encouraging the best teaching and learning practices by examining the role of technology enhanced learning in the emerging area of virtual and personal learning environments.

SubscriptionInformationIJVPLE is published quarterly: January-March; April-June; July-September; October-December by IGI Global. Full subscription information may be found at www.igi-global.com/ijvple. The journal is available in print and electronic formats.

Institutions may also purchase a site license providing access to the full IGI Global journal collection fea-turing more than 100 topical journals in information/computer science and technology applied to business & public administration, engineering, education, medical & healthcare, and social science. For informa-tion visit www.igi-global.com/ISJ or contact IGI at [email protected].

CopyrightThe International Journal of Virtual and Personal Learning Environments (ISSN 1947-8518; eISSN 1947-8526). Copyright © 2012 IGI Global. All rights, including translation into other languages reserved by the publisher. No part of this journal may be reproduced or used in any form or by any means without written permission from the publisher, except for noncommercial, educational use including classroom teaching purposes. Product or company names used in this journal are for identification purposes only. Inclusion of the names of the products or companies does not indicate a claim of ownership by IGI Global of the trademark or registered trademark. The views expressed in this journal are those of the authors but not necessarily of IGI Global.

International Journal of Virtual and Personal Learning Environments

An official publication of the Information Resources Management Association

Editorial: Michael Thomas, PhD Editor-in-Chief IJVPLE E-mail: [email protected]

SubscriberInfo: IGI Global Customer Service 701 E Chocolate Avenue Hershey PA 17033-1240, USA Tel: 717/533-8845 x100 E-mail: [email protected]

Correspondenceandquestions:

The International Journal of Virtual and Personal Learning Environments is currently listed or indexed in: Bacon’s Media Directory; Cabell’s Directory; DBLP; Google Scholar; Library and Information Science Abstracts (LISA); MediaFinder; The Standard Periodical Directory; Ulrich’s Periodicals Directory

IGI P Raith, U. of Education Heidelberg, Germany Ken Reeder, U. of British Columbia, Canada Thomas...

Documents

Transcript of IGI P Raith, U. of Education Heidelberg, Germany Ken Reeder, U. of British Columbia, Canada Thomas...