2017 Provost’s Learning Innovations Grants

3

2017 PROVOST’S LEARNING INNOVATIONS GRANTS

APPLICATION

INSTRUCTIONS

1. Complete this Application Form, in its entirety, and save as “Lastname_Firstname_APP” (using your name).

2. Complete the Budget Worksheet and save as “Lastname_Firstname_BUDGET” (using your name).

3. Ask your Department Head to complete the Department Head Certification, scan and save as, “Lastname_Firstname_SIG” (using your name).

4. Email all documents to plig@rit.edu, no later than 11:59pm EST, January 25, 2017.

If you have any questions about completing this application, please email plig@rit.edu, or contact Michael Starenko at 585-475-5035 or mssetc@rit.edu.

APPLICANT INFORMATION

This application is for a:

Exploration Grant

Focus Grant

Principal Applicant name: Hadi Hosseini

Faculty title: Tenure-Track Assistant Professor Email: hhvcs@rit.edu Phone: (412) 6260672 (Full-time only)

College: Computing and Information Sciences Department: Computer Science

Department Head name: Mohan Kumar Email: mjk@cs.rit.edu

Others involved in the project (if any): Project name: Enhancing Active Learning in Computer Science Education Through Gameplay

Total funds requested (as calculated on the budget worksheet): $4,860 (requests of $1,000 to $5,000 will be considered)

2017 Provost’s Learning Innovations Grants

4

BUDGET

There is a fillable PDF worksheet to calculate your budget. You can download the worksheet at rit.edu/ili/plig.

The total shown on this worksheet must match the “Total funds requested” in the Applicant Information section of this application form

If awarded, additional funds will be provided to cover any benefits and ITS expenses associated with the salary budget requested

Note that any equipment or other materials purchased with grant funds are the property of your department and revert to the department after your project is completed

TIMELINE

Please indicate any variances to the planned PLIG 2017 schedule and your reasons. If you do not intend to deviate from the schedule, you may leave this section blank.

Task Date Proposed Variance and Reason

Full project plan submitted August 23, 2017

Preliminary findings submitted January 10, 2018

Summary of final findings submitted August 22, 2018

Final budget accounting submitted August 22, 2018

Teaching and Learning Commons submission due (posting a summary of findings, examples of teaching designs or materials, etc.)

October 3, 2018

Participation in Teaching and Learning Services PLIG dissemination event (e.g., PLIG Showcase)

November 2018

2017 Provost’s Learning Innovations Grants

5

STATEMENT OF UTILITY (two pages maximum)

Using the evaluation criteria outlined in the Proposal Evaluation section of the PLIG website, please provide an overview of the project you are proposing, including:

Project objectives

An explanation of the teaching/learning problem(s) it is designed to address

An explanation of the significance of the project to student outcomes and/or the student experience.

A brief description of how the project integrates with activity already underway at RIT in a priority area and/or

how this approach has been successfully used at RIT already.

Project summary

The primary objective of this proposal is to design, implement, and evaluate new instructional development opportunities using game-based learning. Integrating games in higher education pedagogy will provide the opportunity for students to develop problem-solving and analytical skills through gameplay in a participatory and engaging approach. We plan to demonstrate the effectiveness of game-based teaching by enhancing students’ in-class learning experience at RIT.

Project objectives

The objectives of this proposal are twofold:

1. Design and implement a set of in-class, game-based activities for a first-year undergraduate course in RIT’s College of Computing and Information Sciences. The outcome of this proposal will be integrated into the syllabus to enhance classroom engagement.

2. Evaluate the effectiveness of game-based techniques in improving classroom participation and engagement in an introductory course in computer science. We evaluate the outcomes through qualitative (interviews) and quantitative (statistical analysis on surveys and grades) methods.

Explanation of Teaching/Learning Problems

Higher education pedagogy is an area of increasing interest to researchers and educators around the world, particularly in the STEM fields. Several teaching and delivery techniques have been proposed to enhance student engagement in order to cultivate a deep learning environment. For example, active learning methodologies have been widely celebrated in recent years as pedagogical processes that engage students in a series of activities to excite cognitive abilities and promote deep learning. Experimental studies have shown the effectiveness of active learning methods over their traditional counterparts [Freeman et al., 2014]. Nevertheless, deploying such techniques do not necessarily guarantee intrinsic motivation for learning among the students. On the other hand, games and game-like activities entice such internal learning motivation by introducing various “joyful” elements in learning syllabi.

In computer science education, active learning has been deployed to provide a richer learning experience to undergraduate students. However, most of these approaches rely on extrinsic motivations for students to partake in the in-class activities. We believe that game-based learning could complement the advantages of active learning by targeting all various domains of learning in Bloom’s taxonomy of learning. Games provide a natural platform for cognitive analysis while addressing the affective and psychomotor domains of learning [Linnenbrink , 2007; Hosseini and Hartt, 2016].

In this project, we aim to address the following questions:

What is the first-year/second-year students’ perception of learning when game-based learning techniques are used?

2017 Provost’s Learning Innovations Grants

6

STATEMENT OF UTILITY (continued)

Does game-based learning improve in-class participation in computer science education?

Are there any significant improvements in students’ performance?

Our plan is to redesign the current syllabus of an introductory course in computer science, introduce elements of modular game design to a number of topics, and measure the effectiveness of such teaching strategies in students’ engagement in the activities through online questionnaires and analyzing the summative assessments.

Integration to RIT and Broader Impact

One of the core academic promises of RIT is education through experiential learning. Game-based teaching strategies are fundamentally experiential: students are often situated in realistic, and problem-solving states, in which they learn the rules of the gamified activity and familiarize themselves with various roles, mechanisms, and possible actions. This type of experiential activity is intrinsic in gameplay, which in turn leads to first-hand experience of the learning activities and allows players to delve into the problem with a first-person perspective.

We believe that redesigning course content to include game-based activities will contribute to RIT’s pioneering role in experiential learning in higher education. Moreover, the outcomes of this study would provide a set of actionable resources to other educators at RIT and will enable them to integrate game-based learning into their syllabi. Moreover, in previous work we observed that games and gameful activities target various students with different learning styles, particularly students from minority groups and female students. Thus, we believe that including game-play will perhaps help students with disabilities and other underrepresented groups to do well in computer science. In addition, game-play will make such technical classes more accessible for non-CS majors, which will eventually promote interdisciplinary study within RIT.

References:

Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., & Wenderoth, M. P. (2014). Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences, 111(23), 8410-8415.

Hosseini, Hadi and Hartt, Maxwell (2016) "Game-based Learning in the University Classroom," Teaching Innovation Projects: Vol. 6 : Iss. 1 , Article 4.

Linnenbrink, E. A. (2007). The role of affect in student learning: A multi-dimensional approach to considering the interaction of affect, motivation, and engagement. In P. A. Schutz & R. Pekrun (Eds.), Emotion in education. Educational psychology series (pp. 107-124). San Diego, CA: Elsevier Academic Press.

2017 Provost’s Learning Innovations Grants

7

STATEMENT OF CREATIVITY (three paragraphs maximum)

Provide a brief description of how this is a novel approach, or a new application of an existing mode or model of teaching and learning, and/or research about how teaching and learning represents a new paradigm. (Please note that special consideration will be given to proposals that demonstrate a new use/application of a model, system, or technology already in use at RIT.)

The use of game-design elements and game principles in non-game contexts, such as general education and language learning, have attracted attention among educators. However, the current literature on gamification does not provide concrete rationale and methodologies for deploying game-based techniques, and most gamification techniques rely on technologies such as computers, handheld devices, online applications, etc. [Zichermann et al., 2011; Deterding et al., 2011]. On the other hand, gameplay has been an intrinsic element of human social behavior, from sports and physical recreational activities, to the more cognitive and mental exercises of playing cards, competing over board games, etc. Games are effective platforms to engage all domains of Bloom’s taxonomy learning: Cognitive domain by engaging learners in a series of problem-solving activities and higher-order thinking; Affective domain by emotional and psychological engagement such as excitement, arousal, and desires; Psychomotor domain by following game mechanisms, using gestures, and adapting to changes in the gameplay [Hosseini and Hartt, 2016].

We focus on the use of games and gameplay in their most fundamental way, without the use of technology. Games and game dynamics do not only incentivize learners to engage in the classroom [Richter et al., 2015], but also activate positive psychological arousal and increase the learner's focus and memory. Learners associate the game elements, such as objects, tasks, and events, to a subject matter. Likewise, fun group activities that induce a level of competency indirectly force the analytical cognition to capture the main ideas. Positive emotions and pleasant experiences cause cognitive activation and psychological arousal. Positive affections are caused by enjoyment, excitement, hope, and group synergy. Such emotions increase learners’ attention and motivation and help students acquire competencies [Linnenbrink, 2007; Pekrun et al., 2002].

In previous work, we developed an exploratory study for introducing game-based learning and to evaluate students’ perception of learning and acceptance of this new paradigm. Our results and observations were encouraging: students generally enjoyed the gamified sessions and were engaged throughout the activities. In this proposal, we are hoping to delve deeper into measuring the effectiveness of game-based learning in an introductory programming course in computer science. This would be the first focus study that requires redesigning some of the course material and revamping a few of the lectures. Our goal is to deploy game-based teaching techniques and design a systematic way for evaluating students’ performance and engagement in introductory computer science courses.

References:

Deterding, S., Dixon, D., Khaled, R., & Nacke, L. (2011, September). From game design elements to gamefulness: defining gamification. In Proceedings of the 15th international academic MindTrek conference: Envisioning future media environments (pp. 9-15). New York, NY: ACM.

Hosseini, Hadi and Hartt, Maxwell (2016) "Game-based Learning in the University Classroom," Teaching Innovation Projects: Vol. 6 : Iss. 1 , Article 4.

Linnenbrink, E. A. (2007). The role of affect in student learning: A multi-dimensional approach to considering the interaction of affect, motivation, and engagement. In P. A. Schutz & R. Pekrun (Eds.), Emotion in education. Educational psychology series (pp. 107-124). San Diego, CA: Elsevier Academic Press.

Pekrun, R., Goetz, T., Titz, W., & Perry, R. P. (2002). Academic emotions in students' self-regulated learning and achievement: A program of qualitative and quantitative research. Educational Psychologist, 37, 91-105.

Richter, G., Raban, D. R., & Rafaeli, S. (2015). Studying gamification: the effect of rewards and incentives on motivation. In T. Reiners & L. C. Wood (Eds.), Gamification in education and business (pp. 21-46). Cham: Springer International Publishing.

Zichermann, G., & Cunningham, C. (2011). Gamification by design: Implementing game mechanics in web and mobile apps. Sebastopol, CA: O'Reilly Media.

2017 Provost’s Learning Innovations Grants

8

STATEMENT OF EFFICACY (two pages maximum)

Provide a brief description of the experiment/research design, methodology, and methods of data collection you will use to gauge efficacy.

This proposal aims to measure and evaluate students’ perception of learning and engagement in introductory computer science courses. In addition, we plan to collect secondary data on students’ performance and carry out statistical analysis of grades to provide a quantitative measure of students’ learning process. Below, we will first describe the steps for carrying out this study and then provide details on our qualitative and quantitative measures to evaluate and assess the effectiveness of game-based learning.

We focus on an introductory course in computer science (CSCI 141: Introduction to Computer Science). This course is the first course in computer science programming and data structure and is required for a large number of undergraduate students at RIT. For most students, this course is the first technical course in computer science. Traditionally, the course is delivered by direct lecturing along with few group activities. However, the current teaching approaches do not adequately provide intrinsic motivations for deeper learning. Therefore, we believe redesigning some of the course modules to include game-based activities (through constructive competition, cooperation, and engagement) would benefit a wide range of first-year students at RIT’s College of Computing and Information Sciences.

This project has various phases, including consultation with current instructors to data collection and analysis. The main steps of this project are:

Consultation: we will be consulting with some of the current and past instructors of this course. This will include informal discussions to better gauge instructors’ teaching concerns and gather a more comprehensive set of teaching strategies and delivery styles that are currently being. The PI has previous experience teaching similar courses to junior undergraduate students in large classes.

Identifying topics: we will focus on various topics in this introductory computer science course. We plan to take a modular approach to game design (as opposed to immersive) [Hosseini and Hartt, 2016]. In the modular game design, we identify the topics that are more appropriate for redesigning and remodeling the delivery style and then combine these gamified modules with other delivery techniques (active learning techniques and direct lecturing).

Modeling: we will be modeling the content using game-play and game-based, in-class activities to include in the new delivery style. The focus is on traditional games (as opposed to online and electronic games) using various props such as cards, coins, monopoly pieces, etc.

Experimental study design: in this step, we will be designing our experimental study. This step includes the design of qualitative and quantitative evaluation measures. In the next section we will explain the details of our evaluation metrics.

Data analysis: this would be the last step after data collection which will include the analysis of conducted interviews, analysis of anonymous surveys, and statistical analysis of grades.

Dissemination: we will be disseminating the outcomes of our study in various informal and formal venues (See the next section for the detailed dissemination plan).

Evaluation methods:

This introductory computer course provides a great opportunity for cross validation of our results. This course is often offered in various sections in each semester (particularly in the Fall semester). Thus, our plan is to work with the instructors to redesign the course delivery and teaching strategies of half of the sections to include game-based activities. The other half of the sections will act as the control group and will be run in

2017 Provost’s Learning Innovations Grants

9

STATEMENT OF EFFICACY (continued)

the traditional non-gamified lectures. The summative assessments of all sections are traditionally coordinated within all sections, and all sections use the same assignments and final examinations.

To measure the effectiveness of our proposed method, we plan to use a combination of qualitative and quantitative measures:

Questionnaires and surveys: our plan is to conduct an online anonymous survey after the implementation of our methods. The questions of this online survey are derived from Economic and Social Research Council questionnaire (2002) which will include sections on demographic of students (about you), their view of lectures, and their perception of engagement and learning.

Interviews: to better capture anecdotal evidence we will be conducting voluntary interviews of students. These interviews will be conducted by the research assistant who is not affiliated with the course.

Observational data collection: we will be randomly selecting a few sessions for systematically observing the interactions in the classroom, number of questions asked by students, number of students working together, the frequency of participation in the discussions, etc. The data will be collected by the research assistant for further analysis.

Statistical analysis: we will be evaluating the impacts of the gamified contents on student grades through analytical evaluation of students’ performance on their assignments and grades. In this step, we plan to assess the collected data on grades and student performance to identify any statistical significance in our proposed approach.

References:

Economic and Social Research Council. 2002. Experiences of Teaching & Learning Questionnaire, University of Edinburgh.

2017 Provost’s Learning Innovations Grants

10

DISSEMINATION PLAN (optional)

Provide details about the journal, conference, show, or other external vehicle with strong potential for dissemination of your results. Include supporting documentation, such as preliminary interest or acceptance, with your application, if available. (Please note that special consideration will be given to proposals that have a defined opportunity for external dissemination, such as an academic journal or professional conference.)

ILI/TLS will assist with arranging channels for disseminating results within RIT (e.g., annual PLIG Showcase).

Elements of game-based learning can be incorporated into the teaching and/or assessment approach of any discipline. As such, the results of this research will be disseminated to a wide audience within RIT and outside RIT.

First and foremost, we intend to compile a set of game-based techniques to redesign course modules in various disciplines. This would provide a resource to other educators at RIT to integrate game-based learning into their syllabi.

Second, the PI is planning to publish the results of this study in a prestigious conference in Computer Science Education (Special Interest Group on Computer Science Education). The PI’s previous works on game-based learning have been presented in the Society of Teaching and Learning Conference and has received positive response from the participants. The results of the proposed study will complement the PI’s current broader research on game-based learning (Hosseini & Hartt, 2016; a journal paper under review). Our recommendations will also be included in a book chapter on game-based learning in an upcoming book (“Play Collection”, publisher: Palgrave Macmillan). Moreover, the PI plans to use this study to leverage external funding that would then be of benefit to educational development and research at RIT.

References:

Hosseini, Hadi and Hartt, Maxwell (2016) "Game-based Learning in the University Classroom,"Teaching Innovation Projects: Vol. 6 : Iss. 1 , Article 4.

2017 Provost’s Learning Innovations Grants

11

ADDITIONAL CONSIDERATIONS

Please address these questions, if needed.

Will your project require assistance for extensive or unusual media, multimedia, simulation, and/or software development? If so, please explain?

All courses offered by RIT must be accessible to students with disabilities, according to Section 504 of the

Rehabilitation Act of 1973 and Title II of the Americans with Disabilities Act of 1990

(rit.edu/studentaffairs/disabilityservices/info). Is your proposed teaching approach accessible to all students, with

reasonable accommodation? If not, please explain.

RIT abides by the Family Educational Rights and Privacy Act of 1974 (FERPA), which prohibits instructors from

making students' identities, course work, and educational records public without their consent

(rit.edu/xVzNE). Will any data gathering or sharing for your project raise any FERPA issues? If so, please explain.

Not applicable.

Yes, the proposed approach combines various domains of learning, and as a result, is highly accessible to students with disabilities.

No. All data collected will be anonymized. We will work directly with the RIT’s office of compliance and ethics.

2017 Provost’s Learning Innovations Grants

12

DISSEMINATION AGREEMENT

By completing this grant application, I agree to provide the materials described here, in support of disseminating what is learned from this project to other faculty at RIT.

I also agree to return all/a portion of the funds that I receive for this project to RIT if I fail to complete or provide the materials described here.

Full project plan (including roles and responsibilities, milestone dates, and pertinent project details)

Overview of preliminary findings (may include experiment/study design, lessons learned, initial data

collection, and/or literature review summary)

Final summary of findings (including data collection, lessons learned, implications for further study, and which

may be in the form of an article abstract, conference presentation outline, or short report)

Final budget accounting (reconciliation of budget provided with your application and the actual project

expenses)

Teaching and Learning Commons posting (a summary of findings and examples of teaching designs or

materials)

Participation in an ILI/TLS dissemination event (e.g., PLIG Showcase)

By submitting this application, I accept this agreement. _HH___ (applicant, please initial here)