Download - 'Gamifying' a Library Orientation Tutorial for Improved Motivation and Learning · "Gamifying" a Library Orientation Tutorial for Impr oved Motivation and Learning Abstract This paper

Transcript
Page 1: 'Gamifying' a Library Orientation Tutorial for Improved Motivation and Learning · "Gamifying" a Library Orientation Tutorial for Impr oved Motivation and Learning Abstract This paper

AC 2012-4977: ”GAMIFYING” A LIBRARY ORIENTATION TUTORIALFOR IMPROVED MOTIVATION AND LEARNING

Mrs. Michelle Spence, University of Toronto

Michelle Spence has held positions in academic and public libraries, as well as a corporate setting. Hereducation includes a master’s degree in information studies (2007) and an honours bachelor’s of sciencedegree (2004), both from the University of Toronto. She is currently a reference and instruction librarianat the University of Toronto’s Engineering & Computer Science Library.

Mr. Jason A. Foster, University of TorontoDr. Robert Irish, University of TorontoMs. Patricia Kristine Sheridan, University of Toronto

Patricia Kristine Sheridan is a Ph.D. candidate with the Institute for Leadership Education in Engineeringat the University of Toronto. She holds a B.A.Sc. and M.A.Sc. in mechanical engineering from theUniversity of Toronto. She is a member of the teaching team and a course developer for the Praxiscornerstone design courses.

Mr. Geoffrey Samuel Frost, University of Toronto

Geoffrey Samuel Frost is a graduate student studying biomedical engineering at the University of Toronto.He completed an undergraduate degree in engineering science at the University of Toronto. He has workedas a Teaching Assistant for the Praxis suite of engineering design courses at the University of Toronto forthe past three years.

c©American Society for Engineering Education, 2012

Page 25.14.1

Page 2: 'Gamifying' a Library Orientation Tutorial for Improved Motivation and Learning · "Gamifying" a Library Orientation Tutorial for Impr oved Motivation and Learning Abstract This paper

"Gamifying" a Library Orientation Tutorial for Improved

Motivation and Learning Abstract This paper evaluates the process and outcomes of converting a library orientation tutorial into a game in a cornerstone design and communication class in the Engineering Science program at the University of Toronto. The purpose of the tutorial is to help students meet many of the Information Literacy Standards for Science and Engineering/Technology, as well as introduce them to important resources in their field. A hands-on tutorial session aims to move 300 students (100 per session) from a reliance on brute force and shallow approaches towards refining their techniques for information discovery and evaluation -- working smarter not harder. The idea of making a game aimed to create incentives for learning, to allow for self-paced learning, and to introduce students to the professional body of knowledge they will need as professional engineers. The idea of gamifying activities for students offers a new approach to teaching that we evaluate in terms of outcomes, and in comparison with the previous pre-game version of the activity. Our evaluation of the activity considers the design of effective achievement levels and learning areas, logistical concerns of large numbers of students in a library, and achievement of learning outcomes. Overall student were more motivated and attained library search strategies; moreover, we have seen improved learning transfer to subsequent course activities. For these reasons, we find that gamification offers motivation in the educational setting that activates the competitive nature of engineering students, while enabling faster development of skills than prior methods. 1. Introduction The world of libraries is foreign to most undergraduate engineering students. They are comfortable in the lab and in the classroom, but research is not an activity they take to readily or easily. In a cornerstone design course (called Praxis 1) at the University of Toronto, we have attempted to change the students’ attitude and comfort by introducing them to the library through an orientation tutorial. Students in the course have typically shown a shallow knowledge of information resources in their field. Relying on the low-hanging fruit found easily by using search engines such as Google, these students tend to rely on a brute force strategy rather than a sophisticated and planned approach to finding information sources. In the 2010-2011 academic year, Praxis I students found a more hands-on approach to library tutorials beneficial in their uptake of knowledge and skills related to information literacy instruction1. Because of the positive correlation between hands-on experience and improved uptake, we wanted to attempt to develop the students’ experience further and, hopefully, push them to a more informed and consistent use of the library in their ongoing design work in the course and beyond. To that end, we created a gamified library tutorial. While often games are designed in a virtual computer environment, we were intentionally aiming to get the students into a real environment of the library to interact with its resources and resource people. Hence, our game was more like a simple scavenger hunt; however, we mimicked two common features of modern computer-based role playing games by incorporating achievements and skill levels.

Page 25.14.2

Page 3: 'Gamifying' a Library Orientation Tutorial for Improved Motivation and Learning · "Gamifying" a Library Orientation Tutorial for Impr oved Motivation and Learning Abstract This paper

This paper describes the “gamified” library orientation tutorial that was designed to introduce engineering students in Praxis I to many of the information sources that are essential to the engineering field. Indirect evidence is then used to evaluate whether the students have taken-up the learning in ways that indicate deeper understanding than they did in the previous year’s tutorial. We take students’ use of complex sources in subsequent course assignments and projects as indicative of deep learning through the live-action library tutorial. By analyzing in this way, we acknowledge a degree of uncertainty in accounting for changes in student behaviour. 1.1 Why gamify a library orientation tutorial for first-year engineering students? Today’s students are part of the gamer generation. In Canada, approximately 80% of children aged 13-17 consider themselves gamers, and approximately 67% of adults aged 18-34 consider themselves gamers2. (A gamer is considered someone who has played a video game in the past 4 weeks2.) In addition, of self-identified gamers, 42% of teens, 49% of adult males and 45% of adult females play video games at least a few days per week, and 38% of teens, 24% of adult males and 27% of adult females play video games every day3. As first-year students are typically in the 17-18 year range, it is safe to say that gaming is not new to these students. The defining traits of a game are that it has a “goal, rules, a feedback mechanism, and voluntary participation4” (p.21). All games have these four key components. Everything else, from interactivity to the concepts of winners and losers, reinforces these traits. Additional aspects like graphics, number of players, and difficulty may contribute to the enjoyment of the game, but a game does not need to have these characteristics. Good games consist of unnecessary obstacles, challenges or work that make the game play more difficult. Good games also facilitate active learning, and multiplayer games can enhance this learning, as players communicate with each other, coaching and teaching each other along the way5. Games trigger a positive emotional response in their players, activating neurological and physiological systems4, and these “unnecessary obstacles” - in the form of challenges, or even simply the rules of the game - provoke these emotional responses more poignantly than we encounter in everyday life. These challenges can occur at different levels, and in good games, players are rewarded differently depending on the difficulty of the challenge5. The more you play the game, the better you get, and this mastery of skills enables a player to successfully tackle new challenges. This transformation of knowledge is highly applicable in the classroom setting, including information literacy instruction, where students may be required to find and use materials that they are not familiar with. In good classes, as in good games, students do not always realize that they are learning. Educational games harness the physical responses to challenges and the potential for creating transformational knowledge, engaging students in a way that is familiar, motivating and challenging. Engineering students respond to these challenges and obstacles particularly well, since engineers are typically creative, imaginative, inquisitive, and enjoy solving problems6. Games played in the real world (as opposed to virtual) offer opportunities for active, experiential, problem-based, and learner-centred learning and provide immediate feedback7. These different learning opportunities facilitate deep understanding and empower students to control their learning experience. Intrinsic motivation is common in games, and is a desirable in education, as students are more likely to spend more time, have a better sense of well-being and can transfer

Page 25.14.3

Page 4: 'Gamifying' a Library Orientation Tutorial for Improved Motivation and Learning · "Gamifying" a Library Orientation Tutorial for Impr oved Motivation and Learning Abstract This paper

the knowledge when they are intrinsically motivated8. Traditional library instruction, including lectures and demonstrations of database search techniques, is often not engaging, motivating or immersive. Engineering students view games as an opportunity for experiential learning, and consider it a valid teaching method, and their experience with gaming is correlated to their opinion of the validity of gaming as a teaching tool9. The use of games in delivering educational instruction in both library instruction and engineering is becoming a widely accepted teaching method. In fact, entire conferences and journal issues have been dedicated to a discussion of games or play in these forms of education (see the 2010 Workshop in Instruction in Library Use (WILU), themed “Design. Play. Learn”10 or the two game-themed issues of the peer-reviewed International Journal of Engineering Education11). Lantada12, in his guest editorial of the International Journal of Engineering Education issues related to gaming, notes:

“In recent years educational gaming has been progressively perceived as a very effective tool for improving teaching-learning activities in higher education. The use of such play-based methodologies for engineering education promotes several practical and communication skills of great value for students’ future professional development” (p. 480).

Information literacy instruction is inherently well suited to gamification. Information literacy practices, such as recognizing an information need, searching for information, interpreting information, and critical thinking, are already evident in game-playing13. The different stages of the process from discovering an information need to creating new information offer many points at which achievement levels can be created. Scaffolding activities creates de facto achievement levels, optimizing the activity for maximum motivation. Holding the tutorial in the library (as opposed to a lecture hall or other location) also has value. As in a regular game, playing in a subdomain of the real game (or in this case, the domain of the real academic activity) allows for better development of knowledge and skills5. If teaching and learning are carried out in the same location where students apply it, they become accustomed to the world where they will complete their challenges, immersing them in the environment and allowing them to develop the fundamental skills that they need to successfully conquer those challenges. By turning a previous year’s library tutorial into a live action game, we attempted to harness these preceding aspects of games to create an immersive learning experience that motivates and challenges students. The desired outcome is an increase in learning versus previous iterations of this type of tutorial. 2. Reasons for Activity Re-design Based on the results of the previous iteration of the activity1 we targeted a number of improvements to the structure, topics, and emphases of the activity. For reference, the original activity had each member of a student team work, independently of their teammates for a defined length of time, at a specific location (“Station”) where they worked on developing a singular, specific research skill by filling in the associated handouts.

Page 25.14.4

Page 5: 'Gamifying' a Library Orientation Tutorial for Improved Motivation and Learning · "Gamifying" a Library Orientation Tutorial for Impr oved Motivation and Learning Abstract This paper

The original activity was designed on the assumption that having more activities than could be completed by an individual would promote team interdependence. However, student feedback provided five areas of critique that were addressed in the re-design. First, students commented that the activity involved too many elements over too short a time for them to meet all of the learning objectives, and that learning indirectly from their teammates did not give them sufficient understanding to be able to use the skills. Second, students found that having Stations at specific physical locations constrained teams from exploring the spectrum of resources available in the library. For example some students might spend the entire activity at a computer terminal or in the book stacks, and never visit physical journals or the reference desk. Having the students restricted to a given physical location did not promote exploration and serendipity. Third, despite that the activities were intended to link to the students’ upcoming design projects, the students did not always see the relevance of the Stations. The Station that received the most negative student response was the one that focused on Search Strategies. This Station was included to counter students’ tendency to jump to specific search terms in common databases (e.g. “Google the first thing that comes to mind”) rather than taking the time to deliberately select more precise sources and more appropriate search terms. However, students found the Station too removed from their experience and preferred mode of research, the other Stations in the activity, and their upcoming design projects. In short, the disconnect meant students failed to realize the importance of properly planning a search. Fourth, students found our attempts to scaffold the activity both constricting, in that there was little flexibility for them to choose what to work on and for how long, and demoralizing, in that they could see at once that they could not complete all the material. Scaffolding was provided through a package of worksheets that students were expected to complete in a “fill in the blanks” mode, and through a defined schedule and sequence of transitions. Finally, the value of the handouts was questioned as there was no explicit validation or verification required during the activity. The package of worksheets was an extensive resource, but because it did not have a direct link and intuitive justification, students had trouble finding value in the package as a whole. As a result, the knowledge intended through the package was not well integrated into future endeavours. 3. Gamified Activity Design The new library tutorial builds upon the 2010 Praxis I library tutorial by retaining the essential aspects of the activity - finding and evaluating information, creating search strategies, and introducing students to the professional literature - but transforming, or “gamifying”, the activity into a free-wheeling scavenger-hunt style game.

Page 25.14.5

Page 6: 'Gamifying' a Library Orientation Tutorial for Improved Motivation and Learning · "Gamifying" a Library Orientation Tutorial for Impr oved Motivation and Learning Abstract This paper

3.1 Logistics and Game Rules The game challenged the students to complete specific challenges in the library across three topic areas: “Reference Designs”, “Codes and Standards”, or “Handbooks and DFx”. These three areas represent specific aspects of library searching that the students need for their engineering design work and beyond. For instance, “reference designs” introduced students to both the design literature and research journals that discussed design. As such, through that area, students were introduced to journal research as well as design literature. The challenges in each topic area were staged in three cards -- with increasingly sophisticated challenges on each card. Embedded within each challenge card were starting points that the students could follow to more quickly locate the information or resources that were their target. Prior to student arrival, the instructors and librarians were partnered and assigned as judges to one of three topic areas: Each pairing was provided with cards containing the 2nd and 3rd level achievements for their topic area, and with baseball hats that were colour coded to match their area. They then distributed themselves throughout the library. An additional group of instructors and librarians were designated as “General Purpose Helpers” and were given red baseball hats. With the instructors and librarians in place, the students were formed into teams of three, were given the instruction, Frequently Asked Questions, and “Level 1” cards for each of the topic areas, and were told to start the activity. At that point, student teams were free to structure themselves however they wished in order to pursue the challenge levels. Students could pursue the Level 1 challenges as a group or individually, and different teams adopted different collaboration styles. The students were also free to pursue one topic area in detail (e.g. earning Level 1, 2, and 3 in “Reference Designs” before moving on to “Codes and Standards”) or to acquire breadth before depth (e.g. earning all three of the Level 1 achievements before moving on to the Level 2 achievements). The flexible structure also allowed students to pursue serendipitous discoveries or to explore other aspects of a given topic area as they advanced. When students completed specific challenges they were required to demonstrate their understanding to the appropriate topic area judge (easily identified by correctly coloured hat). To complete, students had to convince a judge by demonstration. In the event that students were unconvincing, the judge would provide additional instruction verbally to enable them to complete the challenge. As students progressed the judges stamped the challenge cards when specific challenges were completed, so students were not limited to completing all challenges for a given card at once. The judge functioned as a quest giver. If the students achieved the Level N challenges, the judge could then unlock (or provide to them) the Level N+1 challenges. This allowed students to complete challenges in their desired order, and also allowed judges to keep track of if and when students achieved enough challenges to be issued a new challenge card. Students were only given access to subsequent levels after having successfully achieved the challenges posed at the current level.

Page 25.14.6

Page 7: 'Gamifying' a Library Orientation Tutorial for Improved Motivation and Learning · "Gamifying" a Library Orientation Tutorial for Impr oved Motivation and Learning Abstract This paper

To help the students self-manage the 110 minutes they had to complete the activity, announcements of the time remaining were made at regular intervals. 3.2 Associated Game Pieces Game pieces consisted of an instruction card (Figure 1), an FAQ card, challenge cards, TA identifiers (colour-coded baseball hats), and stamps.

Figure 1. Instruction Card A “Frequently Asked Questions” card, Figure 2, was also distributed to help frame the activity for the students, and specifically to legitimize asking for assistance from both the instructors and librarians associated with the activities and those who were providing general assistance. As shown in Figure 3, the Level 1 challenges focused on locating information. The Level 2 challenges, Figure 4, focused on analyzing the found information, specifically though extracting and organizing metadata. Finally, the Level 3 cards, Figure 5, asked the students to engage in a critical assessment of the found information through both data and metadata. Although the instructors and librarians felt that search strategies were a vital component of improving student research efficiency and effectiveness, we acknowledged that it was the rare student who would see either the value of such strategies or how to connect them to their projects. Accordingly the notion of a search strategy was relegated to being a “Bonus Level” for those students who completed the Level 3 challenges, Figure 6.

Page 25.14.7

Page 8: 'Gamifying' a Library Orientation Tutorial for Improved Motivation and Learning · "Gamifying" a Library Orientation Tutorial for Impr oved Motivation and Learning Abstract This paper

Figure 2. "Frequently Asked Questions" card

Figure 3a. "Reference Designs" level 1 card

Page 25.14.8

Page 9: 'Gamifying' a Library Orientation Tutorial for Improved Motivation and Learning · "Gamifying" a Library Orientation Tutorial for Impr oved Motivation and Learning Abstract This paper

Figure 3b. "Codes and Standards" level 1 card

Figure 3c. "Handbooks and DFx" level 1 card

Page 25.14.9

Page 10: 'Gamifying' a Library Orientation Tutorial for Improved Motivation and Learning · "Gamifying" a Library Orientation Tutorial for Impr oved Motivation and Learning Abstract This paper

Figure 4a. "Reference Designs" level 2 card

Figure 4b. "Codes and Standards" level 2 card

Page 25.14.10

Page 11: 'Gamifying' a Library Orientation Tutorial for Improved Motivation and Learning · "Gamifying" a Library Orientation Tutorial for Impr oved Motivation and Learning Abstract This paper

Figure 4c. "Handbooks and DFx" level 2 card

Figure 5a. "Reference Designs" level 3 card

Page 25.14.11

Page 12: 'Gamifying' a Library Orientation Tutorial for Improved Motivation and Learning · "Gamifying" a Library Orientation Tutorial for Impr oved Motivation and Learning Abstract This paper

Figure 5b. "Codes and Standards" level 3 card

Figure 5c. "Handbooks and DFx" level 3 card

Page 25.14.12

Page 13: 'Gamifying' a Library Orientation Tutorial for Improved Motivation and Learning · "Gamifying" a Library Orientation Tutorial for Impr oved Motivation and Learning Abstract This paper

Figure 6. Bonus level card Stamps were used to denote the achievement of specific challenges. Each card had a series of small, related challenges that a team would have to achieve. A team could come to a judge with one challenge met, or wait and produce the whole card. Once the team had demonstrated sufficient understanding to have completed the challenge, the judge would stamp the specific challenge on the team’s challenge card to denote its completion. Once a team had a completely stamped card, they could progress to greater challenge. By allowing challenges to be achieved individually, instead of as a group, it allowed the learning to be scaffolded to individual team needs, so that instructors and librarians could tailor support to the needs of a given team. 3.3 Objectives Our intention in creating the gamified version of the activity was not only to address the students’ negative feedback (described above), but also to advance some objectives that the more static activity had been unable to attain. We wanted students to explore the library - both physically and virtually, to take responsibility for their own learning - both in how they used their time and how they directed their attention, and to understand the relevance of the library activity to their own design work within the course. 3.3.1 Exploration To promote exploration of the physical library space, the concept of a Station was removed from the activity. Students could pursue their objectives throughout the library, and in no case was a specific location necessary to successfully complete an activity. To further promote exploration,

Page 25.14.13

Page 14: 'Gamifying' a Library Orientation Tutorial for Improved Motivation and Learning · "Gamifying" a Library Orientation Tutorial for Impr oved Motivation and Learning Abstract This paper

the instructors and librarians acting as judges in the activity, and who the students needed to find in order to progress through the activity, were given license to roam throughout the library. Success effectively required that the students succeed in a scavenger hunt that guided them though the library.

3.3.2 Development of Time and Task Allocation Skills To promote responsibility for learning, we reduced the number of parallel activities to equal the number of team members and then allowed teams to choose how to allocate tasks amongst themselves and gain the associated learning. Further, we eliminated the monolithic “fill in the blanks” handout to the “challenge cards” that were distributed on an as-needed basis. This allowed teams to make progress without feeling overwhelmed by the tasks ahead.

3.2.3 Self-Direction and Individual Objective Setting By allowing the students to choose which challenge card to start with, how to structure their pursuit of the goals, and where in the library they would go, the new design allowed students to work in their teams in ways that suited individual teams (depth vs. breadth, number of people per topic area). As a side-note, the teams formed at the outset of this activity were teams that would work together on the next project. As a result, this activity was a team forming exercise as well. By giving the students the opportunity to form their own path, they could begin to determine best ways of working together in low-stakes play before encountering a more challenging design activity.

3.2.4 Relevance to Course Work The critique of relevance was addressed primarily by framing the activity around the types of information being accessed, each of which had previously been introduced in lecture: “Reference Designs”, “Codes and Standards”, and “Handbooks and DFx”. Each of these types of information was explicitly mentioned in the assignment and assessment materials for the following design projects. Additionally, the reference designs resources were specifically targeted to be ones relevant to the subsequent assignment - since the next assignment involved critiquing the design of a bridge, all of the books associated with reference designs were related to bridges. To ensure that the skills associated with the original Stations were maintained in the revised activity, elements of each skill were assigned to one or more challenge levels across the different topic areas.

3.3 Engagement in the Game Collectively these changes radically changed the structure of the original activity while maintaining exposure to the essential library skills that were at its core. Focusing on the types of information, rather than the research skills, effectively inverted the structure of the activity. Moreover, by moving the more abstract idea of search strategies to a “bonus” level it became more acceptable that it was more of a remote obstacle -- it is expected that the top level of a game is more demanding. Rather than being perceived as “irrelevant” as in the previous year, it became incentivized as the ultimate goal -- what gamers might think of as “the big boss”.

Page 25.14.14

Page 15: 'Gamifying' a Library Orientation Tutorial for Improved Motivation and Learning · "Gamifying" a Library Orientation Tutorial for Impr oved Motivation and Learning Abstract This paper

Students wanted to reach the top level of the game because they wanted to be able to say they had done so. The value of bragging rights should not be underestimated in a competitive environment such as a game, or an engineering class. 4. Assessment of student learning We attempted to quantify how valuable the activity was to the students by comparing the output of two similar assignments. Specifically, we looked at assignments from the academic years of 2010 and 2011. In both years, the students were asked to evaluate a local bridge from the perspective of engineering designers. In 2010, the assignment was formative. In other words, the students were evaluated rigorously but the evaluation had no impact on their final grade. In 2011, the assignment was worth 10% of the student’s final grade. 4.1 Method To evaluate the success of the gamified library tutorial, we looked at a presentation that the students gave the following week. This presentation should have required students to apply library research to analyze the design of a local footbridge, including its adherence to code, its design features in support of maintainability, accessibility, or other similar “Design for X” (DFx) foci, and more generally the design of the structure. Our analysis uses a simple scoring method: in reviewing video-recordings of their presentations, we asked the question – did the students present a credible source to buttress their claims during the presentation? More specifically – was the source a logical outgrowth of the gamification activity described above? For each credible source that could be traced back to the library tutorial, the presentation was awarded a single point. Ten presentations were examined from each year. This represents roughly ten percent of the presentations presented in a given year. A year’s score was determined by adding the individual scores for each year’s presentations. Presentations were selected for analysis based on two criteria. First, we selected to ensure like was compared to like. To do this, we created two grading silos: (a) Average performing students, and (b) Low performing students. This was done to eliminate the bias that might result from comparing high performing teams from one year to low performing teams in another. We based our silos’ grade range on the University’s transcript guide and the course’s historical average. A presentation was deemed to belong to the average performing silo if it received a B (between 73% and 76%). Likewise, a low performing presentation scored a C (between 63% and 66%). Second, we selected for distinct evaluators for the presentations from within these grade silos. Distinct evaluators were selected to eliminate the bias that may be present had we analyzed presentations from a single evaluator. In sum, our analysis groups included ten presentations per year, with 2010 and 2011 forming our analysis years. Within each year, we considered five presentations that scored a B during the term, and 5 presentations that scored a C during the term. For the 2010 iteration of the activity, the students were not graded, as mentioned above. However, the formative was linked to a summative one on a related topic the week following. P

age 25.14.15

Page 16: 'Gamifying' a Library Orientation Tutorial for Improved Motivation and Learning · "Gamifying" a Library Orientation Tutorial for Impr oved Motivation and Learning Abstract This paper

Therefore, to determine which grade silo a presentation belonged in, we used a grade the students received on a similar presentation as a proxy grade for the formative assignment. 4.2 Results In the 2010 analysis group, a score of seven was awarded to the students – indicating that seven distinct credible sources that can be traced to the library tutorial were presented by the students. The 2011 analysis group presented fourteen credible sources. This indicates a two-fold increase in the use of credible sources from one year to the other. 4.3 Discussion In sum, the gamified library tutorial can be linked to a two-fold increase in the use of credible sources when first year engineering students are asked to examine an existing work of engineering design. Three weaknesses should give some pause to these results. First, the 2010 presentations were formative, which meant students may not have invested in them as deeply (although they were directly preparatory for a subsequent graded assignment). Second, the use of a proxy grade for determining grading silos limits the value of the grade silo. Third, the analysis of the presentations was not blinded (the analyst knew which presentations were given in 2011 and which were given in 2010). A stronger analysis would have blinded the analyst. An improved form of data collection could lead to even more reliable results. Regardless of these weaknesses in data collection, the magnitude of the difference indicates that students took a different approach to the presentations from one year to the next. The students in the 2011 cohort were making use of the source material (such as building codes, bridge inspection manuals, accessibility guidelines) in ways that were detailed and specific to justify evaluations of the bridges they analyzed. This difference suggests both better use of sources and more depth. 5. Perception of student engagement and learning At the end of the term, teaching assistants and studio instructors were surveyed about their perception of student engagement and learning during the gamified library tutorial, and of student’s understanding and use of credible sources after the tutorial. Those who had taught the course in previous years were additionally asked to comment on how the gamification of this year’s activity changed the way in which the students engaged with the activity and with the material. Responses from over eighty percent of the teaching team have been compiled to develop this discussion.

Regarding locating, identifying and understanding credible sources, the use of the stamps to denote achievement was seen to help with motivating students to engage in the material during the activity, but required significant teaching team facilitation to ensure the students were engaging at a level of deep understanding. Structuring the game as a scavenger hunt where students had to immerse themselves both in the library and in the learning experience increased the degree of kinaesthetic learning, and physically got the students to move beyond their begrudging attitude towards the library. The activity was not monotonous; students had multiple challenges over varying levels of difficulty to succeed at, which allowed students who were both

Page 25.14.16

Page 17: 'Gamifying' a Library Orientation Tutorial for Improved Motivation and Learning · "Gamifying" a Library Orientation Tutorial for Impr oved Motivation and Learning Abstract This paper

new to, or familiar with, the library to be challenged. There was general consensus among the teaching team that the interactive and fun nature of the activity should allow students to recall what they had seen at a later date and engage in appreciating its value at that time.

The collecting of stamps on the cards allowed students additionally to be reassured of their progress in the activity, however some students blindly “collected” resources solely for the purpose of getting the next stamp. This behaviour inhibited students from engaging with the importance of the resource and its credibility and sometimes resulted in students locating inappropriate resources. However, this problem was countered by the fact that the students had to convince the judge that they understood the objective of the challenge before being issued the stamp. When students were unable to answer bibliographic or credibility questions about the resource, or to comment on the types of information contained within the resource, they were given guidance and sent away to try again. This created a balance (and tension) between the fun of the scavenger hunt and meeting the real learning objectives. As a result of students’ selective emphasis on the finding of resources, creating a uniform understanding across the teaching team of the types of arguments that were required to achieve a stamp became crucial to the activity. Ensuring students were not able to “skim by” without presenting logical and credible arguments as to why their findings met the requirements of the task and that they should receive the stamp became critical to facilitating the learning experience desired from the gamified activity. Some students did not present any arguments as to why they had met the task’s objectives until it was explicitly asked for by the teaching team, nor did they assess for themselves whether they had completed the specified task, or their preferred interpretation of the specified task. Thus, these discussions between the teaching team and the students to achieve stamps became critical to creating the desired learning experience for the students. These discussions, after students’ first failed attempt at convincing an instructor to give them a stamp, pushed students to locate credible and identifiable sources. This method was largely successful, perhaps because the controls on stamp-giving were strict (stamps were only given by specific, designated, "expert" members of the teaching team), and/or because the level system often required students to understand the first level before being able to complete the next level. One of the predictors of success for the game was how students approached it; teams that strategized how they would tackle the challenges were more successful and completed more levels than those who wandered the library looking for answers. Although announcements were made to try to help students manage their time, the level of planning each group took before beginning the game was a contributing factor to how successful students were.

Regarding the visible use of credible sources in future discussions/work students were able to credibly use the example sources in the activity’s challenges, which can be seen in reference to the bridge design presentations. Students were able to readily identify the use of credible sources when critiquing others work during discussions (which were similar to the types of discussions the students engaged in with the teaching team to achieve stamps), but were not yet consistently effective at assessing their utility, credibility and effectiveness in their own documents. The activity was memorable which helped students remember different aspects when prompted in future work by the teaching team, such as ways to search for credible sources and the library

Page 25.14.17

Page 18: 'Gamifying' a Library Orientation Tutorial for Improved Motivation and Learning · "Gamifying" a Library Orientation Tutorial for Impr oved Motivation and Learning Abstract This paper

website; however, our desire that they learn knowledge/searching skills was not wholly realized. In part, this may be due to the fact that many students only completed the first level of the challenges, or completed one or two levels of one topic area but no levels of the other areas. These weaknesses offer us room to improve the design of the game, but overall the game itself has significantly improved both learning about and engagement with library resources. 6. Improvements for future iterations Future iterations of this activity will entail two changes, both aimed at improving the students’ ability to get further with the activity, so that they reach the more cognitively demanding activities that involve interpretation and research strategy. First, we plan to focus the challenges to reduce emphasis on “finding” resources and increase time spent examining and evaluating resources, and creating transferable knowledge that will allow them to find and assess materials on their own in the future. Students often did not spend any time examining the resources unless they were forced to by the teaching team when trying to obtain a stamp. While “finding” is obviously essential before information can be used, it is only the first step in a complex process of using and creating information. Restructuring Level 1 achievements in so that they require students to not only identify the location of a particular type of source, but also to examine it for relevance will move students more quickly into more critical handling of source material. Paring down the number of resources to find, while still maintaining a variety of types of resources, will also allow students to spend more time developing transferable information literacy skills that promote deeper understanding of the information sources that are essential to the engineering profession. Second, the layout of the cards themselves can improve everyone’s ability to see what the students are achieving. While the cards were easily transportable (size of a cue card), they did not facilitate their being stamped clearly. Space constraints often made it difficult to associate stamps with their correct achievement, and sometimes the stamps overcrowded the text of other achievements on the cards. Occasionally, this resulted in students being able to move forward to the next level without completing all achievements, as their cards were difficult to interpret. Future iterations will incorporate a more optimal layout of achievements and associated, segregated stamping areas so that judges can easily determine whether a student has completed all levels’ requirements on visual inspection. These two changes are complementary as fewer challenges will itself create more space for a segregated stamp area. 7. Conclusion In the final assessment, we are confident that the gamification of the library tutorial reached most of its objectives: it improved student engagement with the library as a physical space and important learning resource, it improved student learning about how to find and use library resources, and it facilitated in students’ better use of library resources in subsequent course work. Future improvements will aim to further these objectives, catch the weaknesses in the system, particularly to get the students working with the library resources in more sophisticated ways, and develop more rigorous means of quantifying student use of library resources and research skills.

Page 25.14.18

Page 19: 'Gamifying' a Library Orientation Tutorial for Improved Motivation and Learning · "Gamifying" a Library Orientation Tutorial for Impr oved Motivation and Learning Abstract This paper

Bibliography [1] Baratta, Michelle, Chong, Alan, and Foster, Jason A. “The Research Studio: Integrating Information Literacy into a First Year Engineering Science Course”, American Society for Engineering Education Conference,

Vancouver, Canada, 2011. [2] Entertainment Software Association of Canada. (2011). 2011 essential facts. Retrieved from http://www.theesa.ca/wp-content/uploads/2011/10/Essential-Facts-2011.pdf. [3] Entertainment Software Association of Canada. (2009). 2009 essential facts. Retrieved from http://www.theesa.ca/documents/EssentialFacts2009EN.pdf. [4]McGonigal, Jane. Reality is broken: why games make us better and how they can change the world. Penguin, 2011. [5] Gee, James Paul. What video games have to teach us about learning and literacy. Palgrave Macmillan, 2007. [6] Engineers Canada. (n.d.) Your child. Retrieved from http://www.whyengineering.ca/. [7]Federation of American Scientists (2011). Why games? Retrieved from http://www.fas.org/programs/ltp/games/why_games.html. [8] Malone, Thomas W. What makes things fun? A study of intrinsically motivating computer games. Cognitive and Instructional Science Series CIS-7. Palo Alto, CA: Xerox PARC. 1980. Retrieved from http://cci.mit.edu/malone/tm%20study%20144.html [9] Andreu-Andrés, María Ángeles and García-Casas, Miguel. (2010). Perceptions of gaming as experiential learning by engineering students. (2011). International Journal of Engineering Education. 27(4), 795-804. [10] Workshop in Instruction in Library Use. (2010). WILU 2010: Design, play, learn ~ créer jouer apprendre, Hamilton, Ontario. Retrieved from http://wilu2010.blog.lib.mcmaster.ca/. [11](2010) International Journal of Engineering Education. (2010). 27(3 & 4). Retrieved from http://www.ijee.ie/contents/c270311.html and http://www.ijee.ie/contents/c270411.html. [12] Lantada, Andres Diaz. (2010) Guest editorial. International Journal of Engineering Education 27(3), 480-481. [13] Gumuluk, Sabina and Webber, Shiela. Playing video games: learning and information literacy. Aslib

Proceedings: New Information 63(2/3), 241-255.

Page 25.14.19