Lenandlar SinghDepartment of Computer Science

University of Guyanalenandlar.singh@uog.edu.gy

Peerwise and Students’ Contribution: Experiences from the Field

CO13 – Connecting Online ConferenceFebruary 03, 2013

OutlineContributing Student Pedagogy (CSP)PeerWise and CSPPeerWise – the ToolBrief Literature ReviewExperimentResultsObservations/DiscussionFuture WorkReferences

Contributing Student Pedagogy(CSP) “…a pedagogy that encourages students to contribute

to the learning of others and to value the contributions of others”

“CSP in formal education is anticipatory of learning processes found in industry and research, in which the roles and responsibilities of ‘teacher’ and ‘student’ are fluid”

“Preparing students for this shift is one motivation for use of CSP”

“CSP approaches are linked to constructivist and community theories of learning, and provide opportunities to engage students more deeply in subject material”[Hamer et al., 2008]

Contributing Student Pedagogy(CSP) Traditionally, students are much more familiar with answering

questions set by their course instructor than they are with authoring questions of their own [Sykes et al., 2011] benefits of the latter is well documented [Nicol, 2007]

“By requiring students to engage in new roles, deeper and more substantial learning may emerge” [Hamer et al., 2008]

“…professional computing practice has largely diverged from hierarchical structures and now operates in a style markedly different from traditional formal education in which an authority disseminates textbook knowledge” [Hamer et al., 2008]

Students now produce much more context than before – Youtube Videos, Social Networks, etc… [Denny et al., 2010a]

use of technology makes it possible for students to efficiently share their authored questions with one another.

PeerWise and CSP PeerWise is a web-based system that allows for the

development multiple choice question banks entirely by students [Denny, 2008]

Students Create MC questions, provide answers and explanations

Students can answer MC questions, provide comments, rate questions

Give Students a rating score for contribution

Developed in 2007 at University of Auckland, NZ by Paul Denny [PhD Research]

Why PeerWise?Many tools available for creating questions –

MC, Open-response, etc [Sykes et al., 2011]

PeerWise “attempts to exploit the familiarity that students have with social software and the phenomenon of Web 2.0” [Denny, 2010]

“…familiarity of students with other social networking tools, and PeerWise’s intuitive interface, facilitates ease of use by students with a range of computing experience” [Sykes et al., 2011]

Literature Review Student Perceive PeerWise to be useful and

Enjoyable [Denny et al., 2008]

external motivators are needed only for question generation; exam performance is correlated with participation in on-line discussions; and, despite student enthusiasm, drill-and-practice use does not contribute to exam success[Denny et al., 2008b]

Measure benefits to student Learning [Denny et al., 2010b]

Literature Review Student who used PeerWise performed significantly better

in Final Exam compared to low activity users; was easy to set up by instructor, and students used it to study for final exam [Denny et al., 2010c]

Student reported positively about PeerWise with explicit recognition of its value to learning [Sykes et al., 2011]

Student contributed beyond what was expected and performance on peerwise was correlated significantly with exam scores[Sykes et al., 2011]

Students are eager participants, rate PeerWise highly, and produce large repository of relevant, good quality questions [Bottomley & Denny, 2011]

Literature Review – Subject DomainsComputer Science/Programming - [Denny,

2008]Engineering [Denny et al., 2009]Biomedical Engineering [Denny et al.,

2010a]Information Systems [Smith et al., 2011]Veterinary Medicine [Sykes et al., 2011]Biomedical Science [Bottomley & Denny,

2011]Introductory Physics [Bates et al., 2011]Introductory Cell Biology [Smith, 2011]

PeerWise – The Tool

PeerWise

Social PeerWise

Social PeerWise

PeerWise Uptake

UG

PeerWise at UG (Dept. of CS)

PeerWise at UG (Dept. of CS)

Experiment (2011 & 2012) CSI 312 (2011) & CSI 3102 (2012) – Introduction to Object

Oriented Programming

Approximately: 25 3rd year BSc in Comp Sci students (2011) 30 3rd year BSc in Comp Sci students (2012)

Peerwise used for Approximately 8 weeks in both years

Students were asked to created 3-5 MCQs, and answer, comment, and participate as much otherwise

2% of Courswork Marks allocated (2011) & 3% (2012)

No training on creating MC questions was conducted

Administrator’s Interface

Results – Contribution Summary

Results

Results – Contribution Breakdown

View Details

Results – Individual Contribution

Results – Individual Contribution

Results – Individual Contribution

Results – Sample Comments

Question

Details

Performance AnalysisMetric Total Average/Student

MCQs Written 126 7.4

Answered 894 52

Comments 342 20

Answered Correctly 693(78%) 41

Answered Incorrectly 201(22%) 12

Metric Total Average/Student

MCQs Written 115 4.0

Answered 666 23

Comments 61 2

Answered Correctly 415(62%) 15

Answered Incorrectly 251(38%) 9

Usage Pattern – CSI 312/2011

Usage Pattern – CSI 3102/2012

PeerWise Contribtion vs Final Exam Performance

Correlations

Answered Commented AnsCorrect

Final Exam Marks

Created 0.4 0.3

0.63 0.41

0.40.33

-0.86(0.25)

Answered -0.1670.05

Commented 0.4100.60

0.470.67

-.5070.31

AnsCorrect 0.980.95

-.1880.15

Student SurveyStudents were asked a number of questions

via an online survey about their experience using Peerwise:Ease of useImpacted their learningMotivation to use tool and contributeIf believe tool is useful for students

Students Response“Peerwise was an awesome tool once I got acquainted with it (not that it took much time to get acquainted with).”

Student SurveyStudents Response cont’d

“In choosing question content, it caused me to do research on particular areas before I posted a question. I had to be sure about something before I posted it”

“ Answering and reviewing other person's questions also taught me alot as it was important to research and be certain before presenting suggestions and reviews to questions.”

Student SurveyStudents Response cont’d

“I believe the interface was easy to use and presented a useful way of keeping my interest(the badges and leader board). In fact, once I got started, seeing my position rise on the leader board and getting new badges was addictive.”

“I definitely think the tool would be useful for students. My only regret is that

more students didn't post and critique questions posted.”

Student SurveyStudents Response cont’d

“My recommendation is that even more use of this tool be advocated for the next batch of students. Also, I wish more emphasis could be placed on them getting started earlier. I found it fun and think that persons will enjoy it once they get started.”

“I would also advocate its use for other subject areas, it's definitely a great learning tool.”

Student SurveyStudents Response cont’d

“Setting up questions was easy enough.”

“…the ability to add explanations and comments for questions was a big help in instances where I may have answered a question incorrectly”

“ a contributor should be notified maybe by email when a question that they supplied has comments especially in cases where the comments may be critical of the answer the contributor would have chosen”

Student SurveyStudents Response cont’d“ I actually used the tool in preparation for my

exams, i wasn't able to get through all of the questions however. I think it can be useful to students as a quick reference and a quick crash study before exams in order to sharpen up before.” “

“On the flip side questions that are incorrect need to be fixed so as to ensure students don't end up taking the wrong answers and thinking those are correct.”

Student SurveyStudents Response cont’d

“it was useful because it encompasses a new way of learning”

“its a very useful tool for student because it has a new method of learning whereas the student

post questions, also provide answer and explanation.”

“The idea of class participation motivated me whereas everyone is involved. Also both

contributing and answering question motivated me.”

“It was very easy, no complication at all”

Student SurveyStudents Response cont’d

“Well to my knowledge Ive learn a lot whereas the class is involved and everyone will try to compete to post "hard" questions, by doing

that i have leaned a lot.”

“I think it should be implemented for other courses especially in Computer Science”

Observation/DiscussionStudents contributed more than expected

Some evidence that ‘low-achieving’ students were more active in creating and answering questions

Negative/low correlation between number of questions created and performance on final exam

Negative/low correlation between all other variables and Final Exam performance

ConclusionDid it work?

Yes, for 2%/3% most definitely

Students enjoyed working with the tool

Future WorkFurther Analysis on Existing Dataset

Use Taxonomy to categorize/rate/rank questions

Develop/Use Taxonomy to Evaluate Questions

Use PeerWise extensively – Difference Groups/Courses

References Bates S.P., Galloway R.K., McBride K.L., (2011) Student generated content: using PeerWise to

enhance engagement and outcomes in introductory physics courses. Proceedings of the 2011 Physics Education Research Conference

Bottomley, S., & Denny, P. (2011). A Participatory Learning Approach to Biochemistry Using student Authored and Evaluated Multiple-choice Questions. Bi0chemistry and Molecular Biology Education, 39(5), 352-361.

Denny,P., Hamer, J., Luxton-Reilly, A., & Purchase, H. (2008). PeerWise. Koli Calling ’08 November 13-16, 2008, Koli, Finland

Denny, P., Luxton-Reilly,A., & Hamer, J.(2008b). Student Use of the PeerWise System. ITiCSE’08, June 30–July 2, 2008, Madrid, Spain.

Denny, P., Hamer, J., & Luxton-Reilly,A.(2009). Students sharing and evaluating MCQs in a large first year Engineering course . 20th Australasian Association for Engineering Education Conference , Adelaide, Australia.

Denny, P. (2010). Motivating Online Collaborative Learning. ITiCSE’10, June 26–30, 2010, Bikent, Ankara, Turkey

Denny, P., Micou, M., & Simon, B.(2010a). Evaluation of PeerWise as an Educational Tool for Bioengineers. American Society for Engineering Education.

.

References Denny, P., Hanks, B., & Simon, B., (2010b). Peerwise: replication study of a student-

collaborative self-testing web service in a U.S. setting. SIGCSE 2010. ACM, USA, pp.421-425.

Denny, P., et al. (2010c). Evaluation of PeerWise as an Educational Tool for Bioengineers. American Society for Engineering Education, 2010

Hamer,J. et al. (2008). Contributing Student Pedagogy. Inroads -Sigcse Bulletin, 40(4), 194-212.

Hooper, A.S.C., Park, S.J., & Gerondis, G. (2011). Promoting student participation and collaborative learning in a large INFO 101 class: Student perceptions of PeerWise Web 2.0 technology. Proceedings of Higher Education Research and Development Society of Australasia International Conference (HERDSA 2011).

Nicol, D., 2007. E-assessment by design: using multiple-choice tests to good effect. Journal of Further and Higher Education, 31 (1) pp.53-64.

Smith, K.M. (2011). Peerwise Participation –supporting students in a multiple choice environment. Proceedings of the Effective Learning in the Biosciences Conference, Edinburgh, UK.

Sykes, A., et al. (2011). PeerWise - The Marmite of Veterinary Student Learning