An E-assessment Model for Supporting Collaborative Knowledge Building in a Social Software

Ole C. BrudvikAnders Nome

NKS Nettstudierwww.nks.no

NKS Nettstudier

• 4000 distance education students• 100 years in 2014• Total of 3 million students so far• Located in Norway• Focus on university/university college degrees• Collaborate with many universities/university

colleges• Main subject areas:

– Business– Media & Communication– Pedagogy & Society– Health & Social Studies

Motivation for Study

“Remix means to take cultural artifacts and combine and manipulate them into new kinds of

creative blends” (Knobel & Lankshear, 2008)

• Develop an e-assessment model for evaluating students engagement in remixing new creative knowledge products using social software.

• Employ this e-assessment model to evaluate students engagement with knowledge building using a social content network – www.diigo.com

Purpose of study

• Develop an e-assessment model for supporting and assessing learning by knowledge building using social software.

• Engage the students to work collaboratively with problem-based knowledge building using social software.

• The study evaluate how the e-assessment support students use of social software for knowledge building tasks.

• The study evaluate what affordances the social software have for engaging in collaborative knowledge building.

Web 2.0/Social Software - Spaces and Places for Knowing

• Explore in virtual and cross-cultural settings.• Multimodal experience and conventions.• Supports communities and collaboration.• Manipulation and sharing.• Consumers to become producers.• Working with ideas and making links between

sources.• Add and remix their own interpretations and

meanings.(Owen and colleagues, 2006; Manovich, 2001; Jenkins, 2006)

Knowledge in the Knowledge Society

• Weinberg (2006) describes knowledge as no longer organized as trees, but as «pile of leaves».

• The organization of knowledge in digital forms allows the same knowledge to be organized by many people in many ways through different digital constructions.

• Relationships with ideas determined by context, and each change in context can result in the ideas being reworked to meet the particular need.

(Weinberg, 2006; Siemens, 2006)

Knowledge

• The same idea might change its modality of representation and its structure at different times in different places (Siemens, 2006).

Instructional Method: Knowledge Building

• Based on the need to be able to work creatively with knowledge.

• Challenge is to engage learners in a developmental trajectory of creative knowledge production.

• Knowledge building defined as «the production and continual improvement of ideas of value to a community, through means that increases the likelihood that what the community accomplishes will be greater than the sum of individual contributions and part of broader cultural efforts.»

(Bereiter & Scardamalia, 2003)

Instructional Method: Knowledge Building

• «The goal is to advance the frontiers of knowledge as they perceive them.»

• Calls for «deep constructivism».• «Learning is an internal, unobservable process

that results in changes in belief, attitude or skill. Knowledge building, by contrast, results in the creation or modification of public knowledge…»

(Bereiter & Scardamalia, 2003)

Dialogic Literacy – Fundamental for Work in Knowledge Society

• «In every kind of knowledge-based, progressive organization, new knowledge and new directions are forged through dialogue….The dialogue in the knowledge age organizations is not principally concerned with narrative, exposition, argument, and persuasion but with solving problems and developing new ideas. Higher-prder knowledge age skills have to do with collaboration, initiative, communication, and creativity»

• Knowledge building has the potential to engage the students in dialogic literacy: «the ability to engage productively in discourse whose purpose is to generate new knowledge and understanding»

(Bereiter & Scardamalia, 2005)

Diigo – Social Content Network www.diigo.com

The students used the browser toolbar to work with information resources on the web

The students used a Diigo Education Group to

Collaborate on the knowledge building task

Methodology

• Study conducted over 10 days with 1 learning task.

• 4 participants volunteered.• Purpose-based sampling (Miles & Huberman,

1984) based on the argument by Owen and colleagues (2006) that no longer are learners constrained by institutional borders, but they can explore in virtual and cross-cultural settings.

• Data collection: screenshots of work in Diigo – social content network. Work stored in the students Diigo Education Group.

Participants

• Participants where recruited from PLN.• 3 in Norway and 1 in Australia.• All of them studying or/and working with

educational technology.• The students did not know each other.• 2 of them have used Diigo before.The participants:1. Thomas: a 21 years old student in the master in ICT-supported Learning program at a University College in Norway. Work as a teacher in a primary school in Norway.

Participants

2. John: a 37 years old student in the master in ICT-supported learning program at a University College in Norway. Work at an e-learning company and completing a Dr. Philos in instructional technology. 3. Andrew: a 31 year old educational technology researcher and a candidate to a PhD in Education at a university in Australia. 4. Peter: a 47 years old history teacher at an upper secondary school in Norway and a trainer for upper secondary school teachers in pedagogical use of technology.

Design of Learning Task and E-assessment Model

• Engage learners with ideas/resources on the Web.• Engage learners in dialogic literacy through

knowledge building using social content network.• Open-ended task with multiple possible solutions.

Design of Learning Task and E-assessment Model

• Competence 1: ability to find and collect relevant ideas/resources.

• Competence 2: ability to find, collect/sample, organize and describe the content in the ideas/resources by choosing descriptive tags of content.

• Competence 3: ability to add own interpretation and meaning to the content of the ideas/resources.

• Competence 4: ability to combine and remix ideas/resources into a new construction of idea/proposal/solution to the task.

Design of Learning Task and E-assessment Model

Assessment model given to the students at the start of the learning task.Purpose: 1. Support summative & formative assessment; 2. Self assessment; 3. Peer assessment; and 3. Transparency (Dalsgaard & Paulsen, 2009).

Results

• 30 bookmarks and 130 visits to the Diigo Group• Most of the engagement on competence 3 with

21 occurrences: Ability to add own interpretation and meaning

to the content of the ideas/resources.

Results: Competence 1

• The students bookmarks was initially not using descriptive tags of the content in source.

• Formatively scaffold the use of tags with pre-defined tags. • Edited one bookmark by adding tags to show how the tags could be

used to direct a focus on using the tags to describe the content in the source.

• Purpose of this was to create an organized "pile of leaves" in the knowledge folksonomy that the group was to work on further.

• After, descriptive tags were used more. • A shared purpose for using the tags is needed!• Scaffolding of using tags is a need!

Results: Competence 1

Students did not use tags but instead engaged directly with Competence 2 & 3.

Results: Competence 2

• The students used descriptive tags such as: game, addiction, psychological obsessive-compulsory to direct focus to what content in this resource is the focus in their work with this resource.

• The students sampled into the Diigo Group space the pieces of content that the tags was linked to.

• Also as they worked with this they engaged with competence 3 as they added their own interpretation by dialogically adding comments to the tagged sampled content.

• The process of sampling content from sources did not need any scaffolding and seemed to be an affordance for knowledge building that Diigo social content network supports.

Results: Competence 2

The students engaging in competence 2 by tagging the content in the source and sampling the content in focus.

Results: Competence 3

• The students started to engage with dialogic literacy at this stage.

• They used comments and sticky notes to add their own interpretation and meaning to the tagged content in their own bookmarks and to comments/sticky notes in other students bookmarks.

• Hence, an affordance of using Diigo social content network is in supporting the students to engage in dialogic literacy by adding interpretation and meaning to all the bookmarks in the collaborative Diigo Group.

Results: Competence 3

The students collaborate on adding interpretation and meaning to the content in the source:

Results: Competence 4

• The most challenging part of the learning task.• The students need to switch focus from dialogically creating an

organized "pile of leaves" knowledge folksonomy by adding interpretation and meaning to using this to propose a solution/solutions to the task.

• Hence, only two students attempted to reach this level. • The affordances of Diigo do not have good support for this

process!• Hence, more scaffolding is needed to extend the bookmarks to

solutions or combine Diigo with another tool to do this part (e.g., “post to blog” from Diigo).

Competence Level 4

Attempt to use discussion Topic in Diigo Group for Competence 4. The students did not choose to use this

Results: Competence 4

Students extending work on competence 3 to 4 by adding additional comment to bookmark

Conclusions

• The e-assessment model provides a useful analytical lens to evaluate the affordances of using different social software for knowledge building.

• It provides a useful framework for designing e-assessment for supporting students working on knowledge building task using different social software.

• The e-assessment model in action could be better as a summative assessment if the number of occurrences of work with competence levels would determine the final score instead of whether or not they do a competence.

Conclusion

• The affordances of using Diigo – social content network for knowledge building tasks:– Require scaffolding a shared purpose and how to do the

tagging of content relevant to the task in the sources.– The affordance of using Diigo is centered at competence

2 & 3: find, sample and add interpretation and meaning– For the students to engage with competence 4 the

extension of the dialogic work on competence 3 need to be scaffold for work on proposing solutions to task

– …or combine Diigo with another tool where the students can bring in the work on competence 3 to create and remix solutions to the task employing the bookmark dialogues.

Further Research

• A study designed with the needed scaffolding and/or tool combination for achieving task completion and longer dialogic literacy engagement in the knowledge building.

• This can extend the knowledge building activity to a competence level 5:

Competence level 5: Dialogically analyze and evaluate the proposed solutions up against each other to collaboratively remix a comprehensive

shared solution to the task.

THANK YOU!