The Fetch Simulation Project is an innovative exercise ...

Creative Teaching Award ‐ Application 2014‐15 The Fetch Simulation Project – Hartzel & Pike

Page 1 of 21

Course and Context

The Fetch Simulation Project is an innovative exercise that was incorporated into GRBU 544 Systems

Analysis and other information systems courses since 2011. GRBU 544 is part of the Master’s program in

Information Systems Management in the Palumbo-Donahue School of Business. Information systems are the

integration of people, business processes, and technology, and this course focuses on analyzing information system

requirements.

Analyzing system requirements encompasses investigating and interviewing the stakeholders involved in

relevant business processes to identify how technology can enhance existing business processes. First the current

business process must be examined, and then an improved process can be proposed. The result of analyzing system

requirements is a specification of system functionality that can guide the development of a new information system

to support the modified business processes (Hoffer & Valacich, 2014). The specifications are documented and

communicated using industry-standard modeling techniques, which are also taught in GRBU 544. In short,

analyzing system requirements has two main areas – gathering requirements and communicating what was

gathered.

The Fetch Simulation Project was delivered as roughly 50 percent of GRBU 544, as the other half included

the delivery of course content to enable students to complete the project and move toward achieving course learning

goals. Students enrolled in GRBU 544 are primarily working through two programs – the Master’s in Information

Systems Management (MSISM) and the Master’s in Health Management Systems (MSHMS). The course is listed

as part of both curricula. From a broader perspective, a course focusing on systems analysis is part of nearly every

undergraduate and graduate information systems program. Further, a systems analysis course is recommended in

the curriculum guidelines for undergraduate degree programs in information systems by the Association for

Computing Machinery (ACM) and the Association for Information Systems (AIS), both internationally recognized

authorities (Topi et al., 2010).

Motivation for the Innovation and Benchmarking of Innovativeness

Requirements analysis demands skill and deep knowledge that is best acquired through engagement and

practice in a realistic setting. Understanding and being able to perform requirements analysis is the foundation for

not only other courses in the curriculum, but also for a career in information systems. In order to develop systems

that create value (e.g., efficiency, effectiveness, cost reduction, time savings) for an organization, a meticulous


Page 2 of 21

analysis of what is happening now (“as-is process”) and how people, technology, and processes could work better

together (“to-be process”) must be conducted. Acquiring these skills requires practice, which can be a challenge in

a classroom setting. This challenge is created because gathering requirements is an iterative process that involves

multiple interactions with many stakeholders over an extended period of time. In addition, the business process and

recommended refinements must be couched in the context of an organization. Naturally, one would like to invite

professionals working in an organization to work with the students. However, asking professionals to act as

stakeholders and participate in this extensive project would be time intensive and simply a commitment too big to

ask for. As a result, in order to provide a hands-on, application-driven experience for students that enables them to

gather requirements, make and test assumptions, and then communicate the requirements, a simulation approach

was selected. Furthermore, involvement of professionals was incorporated through a panel of judges. More

generally, a simulation offered the benefits of: 1) a shared experience for the participants; 2) a common intellectual

experience, which is considered a high-impact educational practice (Kuh, 2008); and 3) active student

involvement, which leads to a deeper understanding of the issues (Montgomery, Brown, & Deery, 1997). While

simulations have been used in business courses before, this simulation was homegrown, integrated into the entire

semester and was paired with an oral defense. We created the simulation completely from scratch after searching

the literature and publishing houses for something comparable and finding nothing. Thus, following the

development of the simulation, we published a conference paper on the Fetch Simulation (Hartzel & Pike, 2014).

Without this simulation project, students would only have had the opportunity to engage written, discrete exercises

and cases and attempt to create models. The act of actually gathering the requirements would not have been part of

their practicing because they would have already been written into the case. Previous versions of this course were

more lecture intensive and utilized said exercises and cases. The introduction of The Fetch Simulation Project

followed a more intentional learner centered teaching (LCT) approach because the students could not just listen

passively to a lecture (Doyle, 2011). They needed to engage in doing the work of learning, and it is this work or

effort that leads to more enduring representations in the brain (Brown, Roediger III, & McDaniel, 2014; Doyle,

2011).

Scope of the Innovation

The innovation involved students in the MSISM degree program, several Information Systems faculty

members, and external stakeholders. While the current iteration of The Fetch Simulation Project was integrated


Page 3 of 21

into GRBU 544 with 31 students, including both MSISM students and MHMS students, prior versions were used in

undergraduate information systems courses we taught and involving nearly 200 students (Appendix D, Figure 1

shows instances of the innovation’s use). Students were simultaneously registered in both ISYS 381W Systems

Analysis and Design and ISYS 382 Data and Information Management and a cross-course design was

implemented. Other internal stakeholders included faculty members, who participated in the defense and

evaluation process. External stakeholders, senior executives in the field of information systems, also participated in

the defense and evaluation process. Through this innovation, course objectives and a program level objective were

assessed by both internal and external stakeholders.

Learning Goals

The Fetch Simulation Project created an opportunity for students to learn and reinforce their understanding

of system requirements gathering and documentation. The learning goals for the project were mapped to the course

learning objectives and the MSISM program learning objective shown in Table 1.

Table 1. Learning Goals for the Fetch Simulation Project Learning Goal* Learning Objective CLO3 Students will be capable of documenting and modeling existing business processes (as-is). CLO4 Students will be capable of documenting and modeling requirements for desired business

processes (to-be). CLO5 Students will be able to perform a gap analysis between the as-is and to-be model and

clarify user requirements. PLO1 Modeling Skills: M.S. Information Systems Management graduates will be able to use

process, data, and object-oriented techniques for capturing and communicating user requirements.

* CLO (course learning objective) and PLO (program learning objective) numbers are in reference to the master syllabus course learning objectives and documented MSISM program learning objectives in WEAVE.

Description of the Innovation

In GRBU 544, The Fetch Simulation Project involved the combination of a physical simulation, associated

assignments and reports, and a defense evaluated by internal and external stakeholders. The Fetch Simulation

Project asked students to dive into an information problem being experienced by a fictional manufacturing

company called Fetch, Inc. Students completed this project in teams of 4 to 5 students. The simulation allowed

students to immerse themselves in a rich environment with a messy problem. Students were encouraged to both

make and test their assumptions, and, conveniently, their classmates were knowledgeable stakeholders that could

help them test their assumptions and provide additional prospective.

The first step in the project was to run the physical simulation during a designated class. In a physical

simulation, students have the opportunity to change a system characteristic, such as level of production or routing

pattern, and then watch the result (Lunce, 2006). During this simulation, students managed and operated the firm,


Page 4 of 21

specifically focusing on order fulfillment and inventory management. The manufacturing context was chosen

because it was possible to make the business processes tangible. The physical simulation made the business

context more understandable to the students in comparison to other less tangible contexts, such as accounting.

During the class session prior to the simulation, the business context and goals of The Fetch Simulation

Project were covered in detail. When students arrived at class the day of the simulation, they were assigned a job in

the organization and provided with a job description (see Appendix E, Figure 1 for a sample) and any documents

needed to complete their assigned duties. There were six different jobs, and thus multiple students were assigned to

each role. All attempts were made to assign different jobs to each member of a team. When these six jobs work

together properly, order entry clerks take orders from customers, the operations manager determines whether orders

can be filled or if additional production needs to be scheduled, shop floor employees build products as specified by

the operations manager, the shipping clerks ship fillable orders, the purchase clerks replenish raw materials as

needed, and the general manager oversees all of the above. These were the duties that were described in great

detail and with step-by-step instructions to each student when he/she was assigned a role. Forms and report were

also provided to assist in the completion of these duties. In total, over 30 different documents (e.g., job

descriptions, forms, and reports) were distributed to the students during this simulation to simply describe the

current business processes. To simulate the manufacturing process, tinker toys were utilized as raw materials. A

bill of materials describing the products manufactured by Fetch, Inc. and how to assemble them was also provided.

An excerpt from the bill of materials is shown in Appendix E, Figure 2.

While the students were charged with operating Fetch, Inc. during the simulation, their broader goal was to

develop an understanding of the existing business processes through the hands-on experience. A motivation for

refining the business processes was also built into the simulation: when the students run the company as described,

it becomes dysfunctional within 45 minutes because there will be a distinct lack of real-time information available

to make decisions. However, we intentionally designed this to be unapparent to the students at first glance. While

working during the simulation, students experienced the manufacturing process grind to a halt as component parts

become unavailable and orders sit unshippable, which resulted from decisions made based on out-of-date inventory

levels. Neither management nor the purchasing or shipping departments was in a position to correct the problem

because they were all working with bad information, whether it be stale or simply inaccurate. The experience of

working at Fetch provided students with both a low-level detailed experience based on their role and a high-level


Page 5 of 21

overview of how the departments operate and experience failure. When it was clear that no progress was being

made, despite additional orders coming in from customers, the instructor stopped the simulation.

After the physical simulation, a debrief was held in order to identify what went wrong: why couldn’t Fetch

operate despite its meticulously crafted job descriptions, forms, reports, and business processes? During the

debrief, individuals who held each position had an opportunity to explain why they were unable to complete their

job duties. The outcome of the physical simulation was discussed as well (e.g., how many orders were received,

how many orders were fulfilled and shipped, how many purchased orders were sent, how many purchase orders

were fulfilled). Fetch’s messy information project involving multiple stakeholders was identified, and the

challenge to fix it was launched.

Throughout the remainder of the semester, students created diagrams modeling the system requirements,

data structures, and processes for an information system that could correct the inherent data timing and quality

problems in the operations of Fetch, Inc. To create these models, students used their own simulation experience,

interviewed their classmates, and studied documentation and forms used in the simulation. With this approach,

students deal with the messiness of the problem by drawing on their own experience working at Fetch, review the

documentation generated during the simulation, make assumptions, and have the opportunity to verify their

assumptions and analyses by working with other Fetch employees - their classmates.

Each homework assignment throughout the semester required students to model one of Fetch’s processes,

functions, or entities. In this approach, the assignments served as scaffolding, which supported the construction of

each team’s final project. After a new modeling technique was introduced through a relatively short lecture,

including examples and time for questions, teams were required to use that new technique to model one of the

processes, functions, or techniques. The teams were encouraged to ask the instructor questions or seek preliminary

feedback on their models. When submitted, all modeling assignments were given written feedback and also face-

to-face feedback was provided when the students failed to master a technique. Failing to successfully complete the

assignment was to be viewed as an opportunity to grow their skills and knowledge of modeling system

requirements. Viewing failure as an opportunity to improve has been described by Dweck (2006) as a growth

mindset and can be a valuable lesson for the students.

At the end of the semester, students compiled all of their revised assignments into a report outlining the

specification for the new information system that can support Fetch, Inc. The contents of the final reports are


Page 6 of 21

shown in Appendix E, Figure 4. These reports were reviewed by a panel of judges, comprised of both internal and

external stakeholders. In the fall semester of 2014-2015, the internal stakeholders who acted as judges were Drs.

Jacqueline Pike and William Spangler. Pike is a co-developer of the Fetch Simulation Project but did not work

directly with this group of students while they developed their reports. Spangler is an information systems faculty

member as well as the Associate Dean for the School of Business. The external stakeholders who acted as judges

were Catherine Schilken, Director of Systems Delivery at Big Heart Pet Brands, and George Rosato, Vice President

of Information Systems and Technology at CONSOL Energy, Inc. Rosato is also a 1997 graduate of the MSISM

program at Duquesne. These professionals graciously volunteered over 5 hours of their time to review the final

reports and attend the defenses.

For the defense, each team was allotted twenty minutes, during which they were asked seven questions

focused on determining the students’ depth of understanding of the content of the report. Twenty minutes prior to

their defense, they were given access to the list of seven questions and were allowed to meet as a team to discuss

them (see questions in Appendix C). Each team defended with no other teams present in a conference room with

the judges on one side and the team on the other side. Each of the questions was read by one of the judges and

directed to a specific team member, randomly selected. After this team member provided an answer, other team

members could add comments, and judges could ask follow-up questions.

Innovation’s Contribution to Student Learning

To directly measure the Fetch Simulation Project’s contribution to student learning, an evaluation rubric

(see Appendix E, Figure 3) was completed for each team by each judge following the review of the final report and

the oral defense. Items on the evaluation rubric were designed to assess the students’ performance and the

simulation’s success in achieving course and program learning objectives. Other items on the assessment were

included to measure students’ professionalism and presentation skills.

The results of the assessment, presented in Appendix C, report the grand mean of the average scores

recorded by the judges at the team level. Item 1 on the rubric assessed the students’ performance in conducting a

“gap analysis” (CLO5). All of the teams were able to identify the critical differences between the existing business

processes and their proposed business processes, which is an indication that the objective regarding performance of

a gap analysis was met (CLO5).

Rubric items 2, 3, 5, and 6 measured the students’ ability to document (model) both the “as-is” (CLO3) and


Page 7 of 21

“to-be” (CLO4) systems and discuss their content in a professional manner. These learning objectives were

collapsed in this report because the same skills are required to model both “as-is” and “to-be.” Functional (item 2),

behavioral (items 3 and 5), and structural (item 6) properties of the systems were specifically assessed. To

determine if these learning objectives were met, the teams were asked to identify and explain how different aspects

of their final report addressed particular problems in the original simulation (e.g., lack of communication, lack of

real-time information). Most teams directed the panel of judges to specific pages of their reports and talked through

how the models remedied the problems and would lead to more efficient and effective manufacturing operations.

For three out of four of the items (2, 3, and 5), the performance of all teams met or exceeded the expectations of the

judges. One team failed to meet the judges’ expectations for structural modeling (item 6).

Items 8, 9, 10, and 11 on the rubric measured students’ performance of the modeling skills and the

techniques utilized to convey system requirements in the final report. These items will also be used in the annual

MSISM program-level assessment. Similar to the above, most teams met or exceeded the expectations of the

judging panel regarding their technical documentation, with only one team failing to meet the judges’ expectations

on every item. This was the same team that failed to meet expectations on rubric item 6.

Despite instructing the judges to differentiate between team performance levels as appropriate, the judges’

overall evaluations indicated an overwhelming majority of the teams performed at a level that met or exceeded the

faculty and professionals’ expectations regarding both the content of their final reports and their ability to explain

and defend said content. Furthermore, looking at the results for each team, 6 of the 7 teams received an overall

assessment that indicated they met or exceeded expectations on all items. These results show that when students

engage in the Fetch Simulation Project, the course and program learning objectives can be achieved. The

variability in the performance figures also demonstrated that the project was challenging for the students to

complete.

Notably, at the conclusion of the defenses, the external stakeholders requested resumes and contact

information for 11 out of 31 students that completed the Fetch Simulation Project. This suggested that the students

not only had achieved the learning goals, but were also able to demonstrate and communicate their knowledge

professionally in an intense setting (e.g., the defense).

Indirect measures of the project’s contribution to the learning measures were also assessed using a post-

project attitudinal survey. The results of this survey are shown in Table 2 of Appendix C. To evaluate whether the


Page 8 of 21

students believed that they had gained knowledge relevant to their career and learned as a result of the Fetch

Simulation Project, questions 1, 2, and 3 focused on students’ perception of their enhanced career confidence and

questions 4, 5, and 6 focused on the promotion of learning. The grand average of the responses for both enhanced

career confidence and promotion of learning was 5.9, on a scale from 1 to 7 (strongly disagree to strongly agree).

This suggested that the students found it to be a valuable experience that both promoted learning and gave them an

opportunity to practice career-relevant skills. During the design of the project, the instructors intended to create a

learning opportunity that the students found engaging and were excited to work on. The results of questions 7, 8,

and 9, which focused on student motivation and averaged 5.9 on a 7-point scale, suggested that this was also

accomplished as the students reported that they found the approach interesting and think it should be utilized again.

Conclusion

Based on the description of the Fetch Simulation Project, its alignment with course and program learning

objectives, and both the direct and indirect evidence that the learning objectives were met, we believe that a

creative and innovative teaching approach was implemented and impacted nearly 200 Duquesne students. Thus, we

respectfully submit the Fetch Simulation Project for consideration for the 2014-2015 Creative Teaching Award.

For additional information, please also review Appendix A - Letters of Support, Appendix B – GRBU 544

Syllabus, Appendix C – empirical analysis of student learning, Appendix D – timeline of the innovation, and

Appendix E – sample materials from the Fetch Simulation Project.

References

Brown, P. C., Roediger III, H. L., & McDaniel, M. A. (2014). Make It Stick: The Science of Successful Learning. Cambridge, MA: Harvard University Press.

Doyle, T. (2011). Learner Centered Teaching: Putting the Research on Learning into Practice. Sterling, VA: Stylus Publishing.

Dweck, C. S. (2006). Mindset: The New Psychology of Success. New York, NY: Ballantine Books. Hartzel, K. S., & Pike, J. C. (2014, 2014). Live, Model, Learn: Experiencing Information Systems Requirements

through Simulation. Paper presented at the Information Systems Educators Conference, Baltimore, Maryland.

Hoffer, G., & Valacich, J. S. (2014). Modern Systems Analysis and Design (7th ed.). Upper Saddle River, NJ: Prentice Hall.

Kuh, G. D. (2008). High-Impact Education Practices: What They Are, Who Has Access to Them, and Why They Matter. Washington, D.C.: Association of American Colleges & Universities.

Lunce, L. M. (2006). Simulations: Bringing the benefits of situation learning to the traditional classroom. Journal of Applied Educational Technology, 3(1), 37-45.

Montgomery, K., Brown, S., & Deery, C. (1997). Using experiential learning to add relevancy and meaning to introductory courses. Innovative Higher Education, 21(3), 217-229.

Topi, H., Valacich, J. S., Wright, R. T., Kaiser, K. M., Nunamaker, J. F., Sipior, J. C., & de Vreede, G. J. (2010). IS 2010 Curriculum Guidelines for Undergraduate Degree Programs in Information Systems. Retrieved January 4, 2015, from http://www.acm.org/education/curricula

The Fetch Simulation Project is an innovative exercise ...

Documents

Transcript of The Fetch Simulation Project is an innovative exercise ...