Abstract

Annotations also provide third party, subjective metadata about the contents of a web page that can beanalysed to provide additional information for use in web searching and dynamic link generation. This raises

* Corresponding author. Tel.: +44 0 1707 284473; fax: +44 0 1707 284666.E-mail address: i.r.glover@herts.ac.uk (I. Glover).

Computers & Education 49 (2007) 13081320

www.elsevier.com/locate/compedu0360-1315/$ - see front matter 2006 Elsevier Ltd. All rights reserved.Annotation is a useful practice for learners as it allows the in-context addition of the learners own knowl-edge with the information under review. This has made the technique popular where the reviewed informationis of a specialist nature, such as technical specications, religious texts and language translation, and is storedon paper. However, large amounts of material are now available as electronic documents hosted upon webservers where readers cannot generally make direct annotations. This increase in web-based learning materialsmeans that there is a need for a mechanism to allow the insertion of annotations directly into web pages. Thetwo key advantages of inserting annotations into the web page are the ability to share these notes with otherlearners and being able to access the annotations from any web enabled computer. These facilities allow learn-ers to collaborate by sharing their ideas about the learning material with other learners who are accessing thesame material.Annotation can be a valuable exercise when trying to understand new information. The technique can be used to createa condensed version of the original information for later review and to add additional information into the existing doc-ument. The growth in web-based learning materials and information sources has created requirement for systems thatallow annotations to be attached to these new sources and, potentially, shared with other learners. This paper discussesannotation in an educational context and introduces some of the web annotation systems currently available. It also pro-vides an overview of the development of a new system, eLAWS, by the authors, based upon the Web Service architecture.Finally, the paper provides suggestions for the future development of e-Learning Annotation tools. 2006 Elsevier Ltd. All rights reserved.

Keywords: Interactive learning environments; Multimedia/hypermedia systems; Cooperative/collaborative learning

1. IntroductionOnline annotation Research and practices

Ian Glover a,*, Zhijie Xu b, Glenn Hardaker c

a Learning and Teaching Development Unit, University of Hertfordshire, College Lane, Hateld, Herts, UKb School of Computing and Engineering, University of Hudderseld, Queensgate, Hudderseld, UK

c Hudderseld University Business School (HUBS), University of Hudderseld, Queensgate Hudderseld, UK

Received 18 October 2005; received in revised form 7 February 2006; accepted 8 February 2006doi:10.1016/j.compedu.2006.02.006

I. Glover et al. / Computers & Education 49 (2007) 13081320 1309the possibility of using annotation contents, for example, as part of the ranking criteria in search engines anddynamically generated web tours.

This paper will describe the development of such a tool by the authors which was funded by the UK JointInformation Systems Committee (JISC), as part of the Distributed e-Learning Programme, for inclusion in theJISC e-Learning Framework (ELF). This new development was based upon a similar application, Annoty,that had been previously developed by the authors and the ndings from which are also reviewed.

2. Literature review

Annotation is a useful technique for adding information to existing documents, which is frequently iden-tied as a valuable part of the process of learning. It is a long established process when working withpaper-based materials. A study of annotations made by US college students to personal copies of academictextbooks highlighted the variety of forms of annotation made by the students to assist in their learning(Marshall, 1997). These annotations ranged from simple highlighting of key words and phrases to conceptmap-type structures seeking to place core concepts within the greater context of the subject. However, repe-tition of some forms of annotations occurred in a signicant proportion of the textbooks and could suggestsome commonality of thought processes. In interviews with students, Marshall (1997) discovered that somestudents prefer to purchase used textbooks that contain a large amount of annotation, whereas other studentspreferred to purchase the cleanest copies available, therefore highlighting potential dierences between learn-ers regarding third-party materials.

A study of the annotations university students and academics made to texts revealed that the most commonstyle and purpose of annotation was highlighting to identify key parts of the document for later review(Ovsiannikov, Arbib, & McNeill, 1999). These highlights are a generic form of annotation with a commonlyascribed meaning, whereas other types of annotation, such as notes and symbols, may only have explicitmeaning for the original annotator (Marshall, 1998). This suggests the need for any proposed annotation sys-tems to allow the annotator to specify the access level of individual annotations. This would increase the use-fulness of the system by removing those annotations where there is very little public relevance or usefulnessfrom the common pool, whilst retaining the ability to create such annotations.

The main advantage of annotating paper-based documents stems from attaching annotations to the contextof the surrounding information and having them remain within the original context. This close tie to theunderlying context allows the annotations to contain the minimum amount of information required for under-standing, which in turn allows the reader to interpret the information at a glance. However, the rise of elec-tronic learning and the increasing use of the Internet means more educational materials are being published asweb-based resources and the ability to make in-context annotations to these new materials would allow thebenets of paper-based annotation to be transferred to the e-learning domain. Annotations provide a way forlearners to make a record of their thoughts or observations within the context of the original document andallow the informal sharing of knowledge related to an artefact or concept (Hicks, 2003; Onas-Kukkonen,1997). Sharing these annotations can provide the stimulus required to start discussions about a topic in a col-laborative learning environment (Kappe & Maurer, 1994; Koivunen & Swick, 2001). However, this informalsharing has also been cited as one of the reason that people do not make annotations to digital documents asthey do not wish to appear to be overly critical (Cadiz, Gupta, & Grudin, 2000). This further supports the needfor learners to be able to make annotations that are only available for personal reference rather than publicconsumption, whilst retaining the public annotation mechanism, as this would enable users to make notes fortheir own use that they may not wish to share with others. A study of the use of annotations for the provisionof translations to language-learners conducted by Chunl and Payne (2004) showed that those students withlower working-memory capacity were able to compensate for this limitation through the use of the annota-tions. This suggests that focussed annotations can be of great assistance to students who encounter problemswith unfamiliar vocabulary and jargon.

Traditionally, the Internet, like the publishing industry, has been centred on the ability to push informa-tion to readers and this has limited the ability to publicly interact with that information. This style of infor-mation delivery has advantages as it allows simultaneous access to the same material by a large number of

readers. However, a mechanism to allow the reader to add annotations to a resource, for public consumption,

1310 I. Glover et al. / Computers & Education 49 (2007) 13081320would allow that resource to grow collaboratively from the accumulated knowledge of all readers. A well-known example of this collaborative pooling of information is Wikipedia [http://www.wikipedia.org], a freeonline encyclopaedia in which the users contribute the articles. A more common method for encouraging read-ers to interact around the content of a web page is the use of discussion boards and reader-supplied footnotes.The use of discussion boards can be eective; however, without strong reference to the material under discus-sion and tutor moderation (Hardaker & Smith, 2000), the dialogue can quickly drift away from relevancy andinto argument and trivialities. Placing the new content directly within the existing information could help tomaintain the focus on the relevancy to the context of the document (Rau, Chen, & Chin, 2004). However, ahybrid mechanism that allows annotations to be made to pages, and responses to those annotations in a dis-cussion board, may provide the benets of standard annotation whilst allowing greater collaboration by theincreased interaction between students (Palme, 1999).

There have been a number of implementations of web annotation software, some of which are highlightedin the following section; however, several limitations in the implementation of current web annotation systemshave been identied (Denoue & Vignollet, 2000). The most signicant of these is the lack of a standardisedrepresentation for web annotations, which means that current systems are divergent and proprietary, thereforelimiting the possibility for third parties to create clients. Recently, this problem has been addressed in a num-ber of ways; for example, Annotea (highlighted below) uses an open architecture to allow the development ofclients by other parties, whilst others propose systems where the content of annotations can be containedwithin the address of the page (Denoue & Vignollet, 2002; Yee, 2002). The rst method is the most desirabledue to its potential to create a rich feature set that can be utilised in various innovative ways, whereas the sec-ond method has the problem of creating web page addresses that are extremely complex and unmemorableand would therefore require a centralised storage mechanism.

A major limitation of current solutions is encountered when attempting to annotate dynamic web pages.This is the nature of pages where the requirement is to adapt to new information or to display existing infor-mation to multiple users in dierent contexts. The problem is very dicult to resolve and no directly compa-rable situation exists when annotating hard copy information. The solutions available are currently limited,with the one most observed being to store a full copy of the page with embedded annotations. However, thismethod requires very large amounts of storage space for the annotated pages and may breach copyright lawsin terms of storage and manipulation. Another option, described by Fiala and Meiner (2003), is to create anindex for annotations detailing the context of the page when the annotation was made and using this indexto determine whether the annotation is relevant in the current context. However, this method requires the con-stituent parts of the page to be constructed from special components and therefore would be unsuitable for themajority of web pages.

A study of the eectiveness of web annotation with regard to student learning was conducted by Hwang,Wang, and Sharples (2007). This study combined student feedback questionnaires with pre- and post-tests andfound that the majority of students felt that the use of the annotation tool enhanced their learning and the testresults for those students with access to the annotation system were signicantly increased compared to thecontrol group. However, results of a nal examination were so similar between the experimental and controlgroups to suggest that there was a ceiling eect due to high motivation to study for nal exams, although it isnoted that it may be that the control group had to study more intensely to obtain this result in the nal exam-ination that the group with access to the annotation system. The amount of increase in a students achieve-ment in the post-test showed a direct correlation with the level of annotation activity with the studentswho annotated more frequently achieving the higher marks; however, students with higher motivation arethe most likely to make frequent annotations and this may, at least in part, account for the observed dier-ence. Nokelainen, Miettinen, Kurhila, Floreen, and Tirri (2005) describes an empirical test of an online anno-tation system, EDUCOSM. The results obtained from the test subjects were favourable with all of the subjectseither agreeing or strongly agreeing with the notion that viewing comments made by other learners assistedthem in their own learning. The authors acknowledge that the sample size of 50 is insucient to draw anybroad ranging conclusions, however, these results suggest that there is at the least a benecial psychologicaleect and a study into the performance eects stemming from the process is needed.

Annotation can provide benets to students learning both as a process and an artefact. As a process, anno-

tation oers a technique to promote deep reading of a textual resource (Marshall, 1998), that is, the act of

I. Glover et al. / Computers & Education 49 (2007) 13081320 1311annotation causes a reader to think about the content that they are about to annotate in order to ensure boththe relevance and the merit of their thoughts before committing them to paper. As an artefact, annotations canprovide alternative interpretations of a piece of content (Agosti, Ferro, Frommholz, & Thiel, 2004) that mayprovide the impetus for a reader to re-contextualise their knowledge of a subject with either the content of theannotation or the underlying content of the resource.

A web annotation system with a well-dened annotation structure would make possible the use of auto-matic web adaptation technologies to alter the presentation of the annotation to learners. Hori, Kondoh,Ono, Hirose, and Singhal (2000) describe a generic web-based system that allows annotations regarding sec-tions of a page to be used to determine the display characteristics of that section. Aspects of such a system,while not directly comparable to an annotation system intended to foster learning, could provide advantageswithin an e-learning annotation tool. For example, if the level of knowledge possessed by individual students isknown, a set of routines could be developed that allow both annotations and content to be hidden from stu-dents of a particular level according to how it has been assessed by their peers. This assessment could allow thestudents to rate content according to relevance to their own level of understanding and this could be used tohide advanced level information from novices and breakdowns of ratings by level made available to the con-tent author. However, for e-learning annotation systems, this area requires further investigation into both theconceptual aspects and the technical challenges involved.

Grnbk, Sloth, and Bouvin (2000) suggest that the annotations made by readers of a web page can beused as a form of metadata for web resources. By analysing the annotations made to a specic resource, itwould be possible to infer classication information related to the resource (Greenhill & Venkatesh, 1999).This classication information could then assist in making the resource machine-understandable and a partof the Semantic Web, which could be used, for example, to provide an increase in the signicance of informa-tion presented as search results. Annotation systems that support user-generated linking mechanisms may alsobe used to create contextual pathways and topographies between web resources where no such connectionspreviously existed. This would also assist in the classication of non-Semantic web pages by creating new rela-tionships between resources.

3. Current systems

A number of commercial and non-commercial web annotation and collaboration tools currently exist.These systems have taken a number of dierent approaches to the complexity of allowing users to annotateweb pages, with varying degrees of success and user support. This section outlines seven of these systems thathave been made available for public use.

3.1. Annotea

The World Wide Web Consortium (W3C) has produced Annotea [http://www.w3.org/2001/Annotea/] tohighlight some of the technologies comprising the Semantic Web development programme. Annotea uses tech-nologies, such as the resource description format (RDF) and XPointer to demonstrate the capabilities of thecomponents intended to be the basis of the next-generation World Wide Web. Annotea currently requires theinstallation of either a new browser with integrated support, or the installation of a plug-in for existing brows-ers. The technologies underpinning Annotea are open standards and the constituent specications are avail-able for developers to implement. As a W3C project, Annotea is in a strong position to produce and promote afuture standard schema for the representation and storage of web annotations. Further information availablefrom Kahan, Koivunen, PrudHommeaux, and Swick (2002).

3.2. Annoty

Developed as a proof-of-concept by the authors as a test-bed for both addressing the technical challenges ofand evaluating student attitudes towards online annotation. The system was created with a standard Javaapplication providing a centralised server and was accessed though a Java Applet running in a web browser.

The system was evaluated by a group of students and a summary of the ndings is presented in Section 5.

1312 I. Glover et al. / Computers & Education 49 (2007) 130813203.3. Crit

Crit [http://zesty.ca/crit/] was developed by Ka-Ping Lee and is based around a central annotation serverthat deals with the storage and retrieval of annotations made by users. The software makes no distinctionbetween public and private annotations with all annotations made to a page being available to all users ofthe same Crit server. The system did not provide any mechanism to make annotations of types other than text.Crit is described in further detail in Yee (2002).

3.4. e-Marked

e-Marked [http://www.e-marked.com] is an annotation tool developed to assist in language learning byallowing students to make annotations to web pages. The key feature of e-Marked in terms of language assis-tance is the facility to retrieve dictionary denitions for highlighted words automatically in order to form theannotation. The e-Marked system is implemented using PHP, which allows the functionality to be used with-out the need for additional software; however, it currently supports only private annotations with no collab-orative features available.

3.5. Gibeo

Gibeo [http://www.gibeo.net] is a commercial system that utilises a proxy server to perform the integrationof annotations with page data and therefore requires no additional software to use. The system provides anumber of useful features, such as guided tours, RSS feeds of annotations and customised user groups; how-ever, some of the more advanced features are only available to paying customers and there is only limited sup-port for the creation of private annotations. Gibeo provides a very eective and streamlined user interface;however, this interface is entirely mouse-driven and may present accessibility problems for learners with visualor motor impairments.

3.6. Third Voice

Third Voice was a commercial system introduced in May 1999 which allowed users to create annotationsand discussions around words within a web page. The software evolved into ThirdVoice 2000 which was anambitious attempt to bring a number of additional features to web users, such as automated conversion ofkeywords into links; discussion communities and annotation in the form of sticky notes. The softwarewas initially hindered by only being available to users of Microsoft Internet Explorer using the Windows oper-ating system as a browser plug-in that was, in some cases, problematic to install and use. The software receiveda mixed response from users and web page developers with some users praising the features but with somedevelopers being oended by the layer of commercialisation being placed over their work. The system was dis-continued in April 2001.

3.7. YAWAS

YAWAS [http://www.fxpal.com/people/denoue/yawas/], an acronym derived from Yet Another WebAnnotation System, was developed by Laurent Denoue. The software is developed in JavaScript and inte-grates with Microsoft Internet Explorer to allow easy access to the functionality. The software uses a localannotation store and does not directly support shared access, although there are a number of workaroundspossible to allow annotations to be sent to other users and a recent update allows annotations to be uploadedto a server. The software has been developed using technologies that mean it is only available for use withinMicrosoft Internet Explorer on Windows and is mouse-driven.

Fig. 1 shows a high-level representation of the process ow used by the majority of annotation systems. Theserver responds to client requests for annotated pages by accessing the data store to retrieve any annotationsrelevant to the page selected by the student. The server downloads the web page data from the relevant web

server; matches annotation reference points with their counterparts within the page; inserts the annotation

I. Glover et al. / Computers & Education 49 (2007) 13081320 1313data at that location; and the server sends the data to the client for display. Modication rights to the originalpage are not required by the annotation system; this means that the data retained in the data store is only afragment of the original page rather than a full copy where the le size increases with every annotation. How-ever, there is a signicant problem that has yet to be resolved by any of these systems and it is caused bychanges to the underlying webpage causing the annotation reference points to become invalid (i.e., no longerpresent within the page) making annotations orphans (see Fig. 2).

Fig. 1. Process overview.

Fig. 2. eLAWS in use.

1314 I. Glover et al. / Computers & Education 49 (2007) 130813204. General requirements for educational web annotation systems

As a result of investigations into the literature surrounding web annotation and assessment of publiclyavailable systems it is possible to draw together a number of requirements and desirable factors for such sys-tems, both as educational and general research tools. The factors can be further broken down into conceptualand technical, with conceptual factors being those that are intrinsic to annotation and the technical factorsbeing those related to the implementation of the conceptual factors. The essential factors are:

Conceptual:

(a) Text annotations. The ability to add textual annotations to the webpage is fundamental in order to ben-et learning by the integration of further context with the original content.

(b) Private annotations. Learners must feel assured that there can be no adverse reaction from the commentsthat they choose to make otherwise they may choose not to make any annotations.

Technical:

(c) No additional software necessary. In order to limit potential barriers to usage it should not be necessaryfor the student to install additional software in order to use the tool.

(d) Accessible. The system should be compatible with assistive technologies, such as screen readers and beusable with only a keyboard.

(e) Browser independent. The core functionality should be usable in all of the major browsers in order to beavailable to the largest number of learners.

(f) Maintains page integrity. The underlying page must remain essentially the same when viewing with theannotation system as it would otherwise. This is to ensure that the learners are not being adverselyaected by the use of the system.

The desirable factors are:Conceptual:

(a) Graphical annotations. The ability to add graphical annotations provides greater exibility to the stu-dents and allows the use of resources that may present interesting concepts in novel forms.

(b) Linking annotations. A signicant benet of annotations is the ability to reference materials that are notprovided in the main text. By creating hyperlinks learners can develop new information webs and revi-talise older content by providing links to more recent arguments and ideas.

(c) Shared annotations. In order to obtain the benets of Collaborative Learning it is necessary to providethe ability to share annotations amongst students. This can increase the likelihood of peer-learningoccurring.

(d) Annotation reports. The ability to collate annotations into structured reports would allow the students tocreate their own learning materials for future reference and for use when studying for examinations.

Technical:

(e) Open architecture. An architecture that allows other developers to create systems that utilise the corefunctionality and add new features and process is desirable in order to focus upon the dierent needof learners in dierent subjects and locations.

(f) Non-commercial. Although the available commercial systems are highly competent, the advantage ofnon-commercial systems is apparent in both the initial cost but also they are more likely to be basedupon an Open architecture.

The requirements in the essential category are deemed to be fundamental when online annotations are tobe used to support an educational process. These requirements address the minimum level of functionality

needed to provide a benecial experience to the students. The desirable category contains a higher proportion

I. Glover et al. / Computers & Education 49 (2007) 13081320 1315of conceptual, rather than technical, requirements and provides for features that will make an annotation sys-tem more exible and interesting to use. By allowing dierent annotation types to be made the student is freeto select those most appropriate to the material, the learning process or their learning style. These factors areneither exhaustive nor absolute as slight dierences in the problem domain would most likely alter the require-ments signicantly; however, the listed requirements are intended to provide a generic overview of the consid-erations likely to be encountered when developing or selecting a system for implementation.

Each of the systems in Section 3 has a slightly dierent focus from the others and therefore addresses thegeneral requirements accordingly. The requirements that are addressed depend upon the original intentions ofthe developers and the target audience/sector for the system and Table 1 shows a matrix of the requirementsoutlined above and whether each system addresses the individual requirements.

Table 1 shows that the majority of the systems listed support the features in the essential category; how-ever, the level of support for those features in the desirable category is more irregular. The fundamental rea-son for this is that the systems are generally targeting dierent domains and therefore the user requirementsare likely to be dierent in each of these domains. This matrix gives an overview of the systems, however, it isnecessary to state that the majority of these systems are still being developed and the feature sets are likely tochange over time. Each of these systems has aspects that would be desirable to be brought into a single tool,such as the automated dictionary, thesaurus and translation lookup features in e-Marked, the streamlined userinterface of Gibeo and the annotation export of YAWAS.

There are potential legal implications that arise from the creation of annotations, especially when the infor-

Table 1Implementation/requirements matrix

Essential Desirable

Conceptual Technical Conceptual Technical

a b c d e f a b c d e f

Annotea X X P P P X X X X X XAnnoty X X X X X X X X X X X XCrit X X X X X X X Xe-Marked X X X X X X X X Gibeo X X X P X X X X P Third Voice X X P X X YAWAS X X X P X XeLAWS X X X X X X X X X X X X

Key: X, supported; , unsupported; P, partial support.mation is to be publicly available, mainly focussed around issues of copyright and libel that need to be resolvedbefore annotation systems can become a mainstream resource. There is also the issue of false authority andmisrepresentation where the annotations can change the underlying information in such a way that there is con-fusion as to both the message being conveyed and the origin of that message. These issues have been a problemin other collaborative information systems, such as Wikipedia, however, in many of these systems the intrinsi-cally evolutionary nature has dealt with the eradication of dubious information.

5. Summary of results from student evaluation of Annoty

The Annoty system was tested by a group of 105 Interactive Multimedia students in the context of gath-ering information for a report on aspects of consideration when developing web sites. The students on thismodule are derived from eight dierent pathways with students from both arts and technology backgrounds.The students were given an initial brieng regarding the use of the software and allowed to familiarise them-selves during a practice session. Following the training session, the students were directed to a web-page andtold to make annotations and generate an annotation report containing their entries. This report was submit-ted along with a completed questionnaire regarding their impressions of the software and its perceived impacton their learning.

1316 I. Glover et al. / Computers & Education 49 (2007) 13081320Table 2 shows some of the comments made by the test students, which are indicative of the observationsexpressed by the entire group. The need for a user guide is a comment mentioned numerous times duringthe early phases of testing, however, once the student has been using the software for a short period, they havelittle problem with the functionality available. The majority (87%) of the test users stated that they would beinterested in using the system for other university work, both for formal assignments and for backgroundresearch, with almost half of the students (52/105) stating that they would also use the system for personalresearch on the internet. The majority of the students (91%) either agreed or strongly agreed to the state-ment using the Annoty tool has improved my learning in this subject. This suggests that there is a consensusamong the students regarding the potential of the software, despite having dierent educational backgroundsand diering levels of experience with technology.

The overall results of the student test show that there was a positive attitude to the Annoty software and theinformal observations that were made of the students during the sessions suggested that, provided the softwarecould be initially presented as a useful learning tool, students were receptive to the underlying concepts. Theresults from this testing encouraged the development of a new system, the e-Learning Annotation Web Service(eLAWS), which was intended to address some of the limitations of Annoty and to allow integration withother e-learning systems. The eLAWS is detailed in the next section.

6. eLAWS development

The eLAWS system was intended to address the main problems encountered by some of the students whotested the Annoty system. The two main problems were the need to install the Java Runtime Environment(JRE) software on the students PC in order to use Annoty and the functionality being unavailable outsidethe testing environment. These problems were related to the use of a Java Applet for the client, as the JRE

Table 2Sample user comments on Annoty system

Reason for need for tool Other features desired General comments

Easy to use. Cuts down written work. Makes it easier to create reports. All the information is saved. Alterations can easily be made. Information can be easily updated See what others think. Saves time on report writing. Easier to use than writing a report. Users able to compare their answerswith other students.

Clear report structure already provided.

Idiots guide to Annoty to make iteasier to understand on rst visit.

Not having to delete the informationfrom the side box by yourself.

User Guide on How to use Annoty.

Other people should not be able to seewhat you have wrote [sic].

Better Navigation structure needed. Needs to be made easier to use toenter annotations.

Ability to access Annoty anywhere onthe Internet (currently not possible).

Easier to edit/delete annotations.is required to run the software within the browser and the connection between the client and the server canbe blocked by security systems on either the server machine or the client. The eLAWS is part of the JointInformation Systems Committee (JISC) E-Learning Framework (ELF) [http://www.elframework.org] andas a result the code for the software is freely available for implementation and modication by any individualor organisation.

Altering the basis for the eLAWS into a fully server-side system takes the burden of code execution awayfrom the students PC and places it on the server. This means that the software can be accessed with only aweb browser and no additional software is required to be installed. The browser-based nature of the new soft-ware means that as long as a PC can access web pages the security systems will allow it to access the eLAWS,thereby removing the need to alter security policies in order to provide access to students. A signicant benetof the move to a server-side system is that changes are available to all users when they next access the systemmeaning that if new features, such as additional annotation reports, are added, students will be able to beginusing them almost immediately.

Recoding the Annoty into the eLAWS meant that large parts of the existing, tested coded could be used toform the basis of the new system and performance enhancements could be made where appropriate. However,

The eLAWS is controlled by using the web from in the left-hand frame and the page appears in the largerright-hand frame. The annotation in Fig. 1 is visible in the main frame as text with a darker background; how-

I. Glover et al. / Computers & Education 49 (2007) 13081320 1317ever, in actual use the student can set the colour scheme to whatever best suits their own requirements.

7. Conclusions

Annotation is a valuable learning practice that can support reinforcement of existing knowledge and thecontextual integration of new knowledge. The growth in online learning increases the desirability for a mech-anism for web annotation and the provision of collaborative features has the potential to increase the benetsobtained from such a web annotation system by allowing users to learn from their peers. Empirical study ofboth the use and benets of such annotation systems needs to be performed in order to identify the validity ofthe technology when applied to dierent types of learners, particularly dierent levels of understanding, suchas schoolchildren, university students and lifelong learners. However, those studies that have been undertaken,generally based upon adult learners, suggest that there is a correlation between the availability of annotationsand an increase in learner understanding.

The number of currently available systems shows that there is interest in the application of annotations toweb pages. However, those systems and tools currently available, with the exception of e-Marked, areintended as general tools rather than providing a focus on learning and the requirements of learners. Theintention of the eLAWS system is to form the basis of an annotation system with a focus towards collabora-tive e-learning. eLAWS achieves this by supporting a number of dierent types of annotations as well as allow-ing annotations to be visible to all learners (public) or only for the annotation author (private). The workundertaken by the W3C on the Annotea system may eventually lead to the creation of a standard ontologyfor the representation of web annotations, especially in the Semantic Web Domain and any move towardsstandardisation is to be welcomed.

The use of the web service architecture for the eLAWS gives increased exibility to developers wishing toutilise the system as they can create both new clients for the system that work with existing Services and new oraltered Service software that works with existing clients. Therefore, the web service basis of the system allowsas the location of the client was being altered, the communication mechanism between the client and the serverwas reassessed and, in order to provide the greatest exibility in future development, it was decided that a WebService interface between the two parts was desirable. A Web Service is a piece of software hosted on a web-accessible machine that uses standardised, lowest-common-denominator mechanisms to communicate withclients, which means that developers can develop client software to access a Web Service and perform custo-mised processing on the results obtained. For the eLAWS, this would mean that other developers could takethe core functionality and expand upon it or could directly integrate the functionality into existing software,perhaps making the functionality a core part of a larger virtual learning environment.

The eLAWS provides a set of reports that can be used by students to get an overview of the annotationsthat have been made through the system. These reports are intended for use as a source of reference for revi-sion purposes, as well as one which shows the username of those students who have made an annotation to aparticular resource. The reports types are as follows:

All public annotations for the current page. All annotations made to the current page by the current student. All annotations made to any page by the current student. List of annotators.

Students and annotations are not linked within these reports as the privacy aspect is necessary to get somestudents to contribute. However, new report types can be created by those with only a small amount of Javaknowledge and the new reports can be made available to students very quickly and without the need to restartthe eLAWS system.

Fig. 1 shows the eLAWS software being used to annotate the homepage of the W3C [http://www.w3c.org].developers to create and implement exactly the system that is required.

1318 I. Glover et al. / Computers & Education 49 (2007) 13081320Wider adoption of web annotation technology could eventually benet all users of the web, not just thosedirectly using web based learning resources. The resulting annotations could be utilised by search engines asadditional data to rank and lter pages according to relevance, both by explicit key words and by implicitmeans though existing links, created either by the original page author or by other readers.

There are potential legal implications that arise from the creation of annotations, especially when the infor-mation is to be publicly available, mainly focussed around issues of copyright and libel that need to be resolvedbefore annotation systems can become amainstream resource. There is also the issue of false authority andmis-representationwhere the annotations can change the underlying information in such away that there is confusionas to both the message being conveyed and the origin of that message. These issues have been a problem in othercollaborative information systems, such as Wikipedia; however, in many of these systems the intrinsically evo-lutionary nature has dealt with the eradication of dubious information, although this issue has recently causedWikipedia to require users to register before they are able to edit content in order to decrease the accountabilityfor content of the Wikimedia Foundation [see http://news.bbc.co.uk/1/hi/technology/4520678.stm for details].The approach that would need to be takenwith an annotating tool in order to reduce institutional liability for thecontent would need to be multi-faceted. Firstly, acceptance of an appropriate set of terms and conditions wouldbe require for students register; secondly, a mechanism to allow super-users to remove or demote to privatestatus any oending or misleading annotations may be desirable; thirdly, lters to identify potentially infringingor oensive annotation content and prevent storage would need to be investigated (although these lters mayconstitute unfair censorship; and nally, a mechanism to prevent annotations being made to specic resourcesneeds to be investigated). The last point has a number of dierent options for implementation, either opt-in oropt-out where authors would explicitly sanction annotation or prevent it, and institutionally maintainedwhite-lists of sites that can not be annotated could be created.

The issue of misinformation is particularly relevant in e-learning annotation as the purpose is to enhancestudents acquisition and assimilation of knowledge and misinformation, either deliberate or accidental, wouldgreatly hinder this purpose. The issue is likely to be of greatest importance in subject areas where there areopposing viewpoints and controversy as annotations could be used to discredit a particular stance by support-ers of another stance. A mechanism to partially address this situation needs to be identied, although a ratingsystem for the validity of annotations may be appropriate with the votes being weighted according to the levelof knowledge of the rating user. Such a mechanism would also be subject to abuses, but would require moreeort to aect the result.

8. Future developments in e-learning annotation

As the volume of information and specic learning resources accessible through the World Wide Webincreases, so does the desirability of online annotation tools; however, as the format for learning resourcesbecomes more standardised, there is currently no emerging standard or specication for the format of anno-tations. Without a formalised structure it is unlikely that organisations will commit to the implementation ofan annotation system as they would be unable to migrate the annotations they have already captured to a newsystem. Therefore, it is essential that a standard ontology be devised that provides the basis for annotationsand gives a level of interoperability between the dierent systems.

With a standardised ontology for annotations it would be possible to utilise annotations in new ways, suchas overlays for content in reusable learning objects (RLOs). For example, an RLO could be supplied with aset of annotations by the author or a third party that would allow additional information and links to be dis-played at the learners request adding a new layer of contextualisation to the original resources dependentupon the needs of the learner. Annotation sets could also be produced by an organisation to address the prob-lems encountered by their own students without the need to update the underlying RLO. This may assist stu-dents in developing new insights into the materials with which they are working and help reinforce existingknowledge by allowing access to dierent contexts. Developing an adaptation mechanism that utilises free-form, yet structured, annotation data to inuence the display of content would present a signicant challenge,and yet may produce rewarding results. For example, one learner may state a preference for annotations beinglisted in a tabular format, whereas another user may desire to see those same annotations in a concept map

format, both of which could be generated automatically by the system. However, this would require either the

Greenhill, S., & Venkatesh, S. (1999). Constructing and navigating personalised views of the Web. Information Processing and

I. Glover et al. / Computers & Education 49 (2007) 13081320 1319Management, 35(5), 679689.Hardaker, G., & Smith, D. (2000). E-learning innovation through the implementation of an Internet supported learning environment.

Educational Technology and Society Journal, 3(3), 422432.Hicks, D. (2003). Supporting personalization and customization in a collaborative setting. Computers in Industry, 52(1), 7179.Hori, M., Kondoh, G., Ono, K., Hirose, S.-i., & Singhal, S. (2000). Annotation-based web content transcoding. Computer Networks, 33,

197211.Hwang, W.-Y., Wang, C.-Y., & Sharples, M. (2007). A study of multimedia annotation of web-based materials. Computers & Education,

48(4), 680699.Kahan, J., Koivunen, M.-R., PrudHommeaux, E., & Swick, R. R. (2002). Annotea: an open RDF infrastructure for shared web

annotations. Computer Networks, 39, 589608.Kappe, F., & Maurer, H. (1994). From hypertext to active communication/information systems. Journal of Microcomputer Applications,

17, 333344.Koivunen, M.-R., & Swick, R. R. (2001). Metadata based annotation infrastructure oers exibility and extensibility for collaborative

applications and beyond. In: Proceedings of K-CAP2001 workshop knowledge markup and semantic annotation. http://seman-not2001.aifb.uni-karlsruhe.de/papers/1_annotea.pdf Accessed 08.06.05.

Marshall, C. (1997). Annotation: from paper books to the digital library. In: Proceedings of ACM digital libraries 1997. http://www.csdl.tamu.edu/~marshall/pubs.html Accessed 02.06.05.

Marshall, C. (1998). Toward an ecology of hypertext annotation. In: Proceedings of ACM hypertext 1998, 4049.Nokelainen, P., Miettinen, M., Kurhila, J., Floreen, P., & Tirri, H. (2005). A shared document-based annotation tool to support learner-

centred collaborative learning. British Journal of Educational Technology, 36(5), 757770.Onas-Kukkonen, H. (1997). Towards greater exibility in software design systems through hypermedia functionality. Information andcreation of explicit links between annotations or the use of an algorithm to identify the relevance of an anno-tation to the desired context according to its content.

Integration with other online systems may improve the learning experience by creating a richer environmentfor those students who desire it. For example, integration with a discussion board system would allow anno-tations to lead to complex discussions related to the content of the materials with which they are working,therefore helping to bolster the collaborative learning aspects of the systems. At the moment this is possiblein many of the existing systems by using annotations as links, however, this does not allow the discussionboard system to directly access the details of the annotation and which could be useful for automatically pro-viding a set of linked resources within the discussion. Another example where integration may be benecialwould be with a content management system (CMS) that would allow students to make graphical annotationsand have the image automatically uploaded into the CMS so that it is available from any location.

Acknowledgements

This work was supported by the Joint Information Systems Committee (JISC) (http://www.jisc.ac.uk) aspart of the Distributed e-Learning Programme and the University of Hudderseld. The source code and doc-umentation for the system is available from Sourceforge (http://sourceforge.net/projects/annoty).

References

Agosti, M., Ferro, N., Frommholz, I., & Thiel, U. (2004). Annotations in digital libraries and collaboratories facets, models and usage.In R. Heery & L. Lyon (Eds.), Research and advanced technology for digital libraries. Proceedings european conference on digitallibraries (ECDL 2004). 3-540-23013-0 (pp. 244255). Springer.

Cadiz, J., Gupta, A., & Grudin, J. (2000). Using web annotations for asynchronous collaboration around documents. In: Proceedings ofthe 2000 ACM conference on computer supported cooperative work.

Chunl, D. M., & Payne, J. S. (2004). What makes students click: working memory and look-up behavior. System, 32, 481503.Denoue, L., & Vignollet, L. (2000). An annotation tool for web browsers and its applications to information retrieval. In: Proceedings of

RIAO2000. http://citeseer.ist.psu.edu/denoue00annotation.html Accessed 08.06.05.Denoue, L., & Vignollet, L. (2002). Annotations in the wild. In: Proceedings of the ECAI2002 workshop on semantic authoring, annotation

and knowledge markup. http://sunsite.informatik.rwth-aachen.de/Publications/CEUR-WS//Vol-100/Laurent_Denoue-et-al.pdfAccessed 08.06.05.

Fiala, Z., & Meiner, K. (2003). Annotating virtual web documents with dynamicmarks. Workshop XML Technologien fur das semanticweb (XSW 2003). http://www-mmt.inf.tu-dresden.de/ala/pub/0304.pdf Accessed 08.06.05.

Grnbk, K., Sloth, L., & Bouvin, N. O. (2000). Open hypermedia as user controlled meta data for the Web. Computer Networks, 33,553566.Software Technology, 39, 391397.

Ovsiannikov, I. A., Arbib, M. A., & McNeill, T. H. (1999). Annotation technology. International Journal of Human-Computer Studies, 50,329362.

Palme, J. (1999). Talking back to the WWW. Computer Networks, 31(21), 22812286.Rau, P.-L. P., Chen, S.-H., & Chin, Y.-T. (2004). Developing web annotation tools for learners and instructors. Interacting with

Computers, 16(2), 163181.Yee, K.-P. (2002). CritLink: Advanced hyperlinks enable public annotation on the web. Demonstration abstract. In: ACM conference on

computer-supported co-operative work. http://zesty.ca/pubs/cscw-2002-crit.pdf Accessed 08.06.05.

1320 I. Glover et al. / Computers & Education 49 (2007) 13081320

Online annotation - Research and practicesIntroductionLiterature reviewCurrent systemsAnnoteaAnnotyCrite-MarkedGibeoThird VoiceYAWAS

General requirements for educational web annotation systemsSummary of results from student evaluation of AnnotyeLAWS developmentConclusionsFuture developments in e-learning annotationAcknowledgementsReferences