Multidisciplinary Senior DesignKate Gleason College of Engineering

Rochester Institute of TechnologyFall 2017 & Spring 2018

Project Number: P18711

TEAM ASSIGNMENT PROCESS IMPROVEMENT

Ann BrautigamIndustrial & Systems Engineering

Madeline GalvinIndustrial & Systems Engineering

Joseph YeiterIndustrial & Systems Engineering

ABSTRACT In Multidisciplinary Senior Design at RIT, Kate Gleason College of Engineering students must be assigned to

approximately eighty projects every Fall semester and approximately twelve projects every Spring semester. Enrollment for MSD is increasing each year and the process to assign students is currently manual and extremely labor intensive, requiring significant effort over multiple weeks. There are several factors and constraints that need to be considered when making the team assignments, adding complexity to the system. This project was completed to improve the current system in which students are assigned to projects in MSD. A system was chosen to automate the assignment process to limit the number of manual steps and reduce chance of error in manually inputting data. This assignment system takes student data and project data to calculate a degree of fit for each student on each project. That calculated score is optimized in Python to create best possible team solutions. The previous two semesters of MSD students were used to simulate assignments with the new system and project assignments were comparable to actual assignments made.

INTRODUCTION The Multidisciplinary Senior Design (MSD) program at RIT is completed by nearly all (approximately 375 per

year) engineering students in the Kate Gleason College of Engineering. Students are assigned to an MSD project team prior to the start of the semester based on a variety of factors, including engineering discipline, skills acquired through coursework or co-op, and the class meeting time they have chosen. In addition, some students propose (and must then be assigned to) their own projects. So, this is a classic optimization problem, with an objective of “best fit of students to projects”, and a number of constraints to be accounted for.

The current process (in place for several years) to create project team assignments is executed by faculty and staff over multiple weeks, and involves manually transferring many pieces of information and manually placing each student on a team. As Fall enrollment for MSD has increased in students from five different departments across three different meeting times, challenges faced for creating optimal team assignments have also increased. Tracking fluctuating project staffing, student drop/adds, project drop/adds, and class section limits in multiple locations combine to introduce many potential sources of error in the system.

The goal of this project is to improve the process for creating team assignments by increasing efficiency, limiting errors, and maintaining the current level of effectiveness. To accomplish this, the team has revamped the methods of collecting incoming MSD student information (interests, skills, project preferences) and project data (scope, required staffing, and skills). The data that are collected will be standardized for optimal use by the MSD Office staff and Department Faculty Leads who are responsible for approving project details. The student and project data that are collected and finalized will be used to generate metrics to quantify how well each student is matched to each project. This step will create a score for use in creating the assignments of students to projects. A project score will also be calculated based on the total of the scores for each student assigned to a project. The scores will be optimized in a Python program that will automatically generate team assignment solutions.

Multidisciplinary Senior Design Page 2

The work done by the MSD Office and Department Leads to gather student and project data to create team assignments will be simplified, the MSD Students will be assigned to preferable projects, and Project Proposers will be able to submit information more efficiently. Those involved in the project are summarized in Table 1.

Table 1. Project resources

Stakeholders Primary Customer Project Guide Project Team

Beth DeBartoloChris Fisher

John KaemmerlenLou Beato

Jennifer BaileyEd Hanzlik

Mark Indovina

Beth DeBartolo John Kaemmerlen Madeline GalvinAnn Brautigam

Joe Yeiter

PROCESSThe needs of the customer were determined by Beth DeBartolo, the head of the MSD Office. Table 2

summarizes the customer requirements used to generate the engineering requirements. The overall focus of the project was to make the process of assigning students easier for the MSD Office, this would decrease time to complete assignments and increase accuracy of assignments.

To improve the team assignment process, the team needed to understand the current constraints that the MSD Office had to consider while making assignments. Through observation and discussion with the customer, a list of system constraints was created and is documented in Table 3. Classifications are included that specify how the constraint was currently being used in the assignment process and how it would be used in the new process. If these same constraints were incorporated in the new system, then the automatically assigned projects would have similar results to the manually completed assignments, satisfying the customer requirement.

Table 2. Summarized customer requirements determined by the team and the primary customer

Customer Requirement Importance

Improve the current team assignment selection process while maintaining the current level of success

9

Update the current MSD Pre-Course Survey and PRP Template to support process improvement 3

Eliminate missed students in assignment process 3

Create a reliable system with repeatable results 3

Reduce time taken to complete full assignment process 3

Decrease the number of manual steps in the assignment process 3

Table 3. Assignment constraints and corresponding status in the new system

Project P18711

Multi-Disciplinary Senior Design Conference Page 3

Constraint Priority Currently Considered New System Status

Student proposed project 1 Manually determined Automatically calculated

Sponsor linked student 1 Manually determined Manually determined

Guide availability 1 Manually determined Manually determined

Class section 2 Manually determined Automatically calculated

Discipline 2 Manually determined Automatically calculated

Intellectual property considerations 2 Manually determined Automatically calculated

U.S. Citizenship 2 Not considered Automatically calculated

Skills 3 Manually determined Automatically calculated

Interests 3 Manually determined Automatically calculated

Desired project 3 Manually determined Automatically calculated

Dual Degree/Dual Major/Minor 4 Manually determined Manually determined

Co-op experience 4 Manually determined Manually determined

Measurable engineering requirements were determined to meet each customer requirement, and were tracked throughout the project to ensure they were being met. A summary of the critical engineering requirements is found in Table 4. Test plans were developed iteratively in the design phase that could be tested and evaluated during each sprint. Results of each sprint were demonstrated to the customer and the next sprint’s features were collectively chosen and incorporated. By operating in this fashion, the team could deliver a complete system and meet the engineering requirements.

To test the engineering requirements, sets of data were created from the past two semesters of MSD classes. Student survey data and project data files were cleaned and developed to match the new proposed format. The results of the actual assignments would be compared to the results of the simulated assignments created with the new system.

Additional documentation, including complete lists of requirements can be found on the team’s EDGE website. Please visit http://edge.rit.edu/edge/P18711/public/Home for more information.

Table 4. Summarized engineering requirements with modified descriptions and status

Engineering Requirement Specification Metric Status

Reduce students without a project Students Missed Target met

Increase effectiveness/usefulness of pre-course surveys

(1) Decreased percentage of questions where data should be manually interpreted (2) Increased percentage of survey questions that map directly to a deciding factor

Target met

Ensure that students who proposed a project or have a sponsor relationship, are assigned to the correct project team

Number of students who are not assigned to their correct project if pre-assigned

Target met

Increase student satisfaction of students who have not proposed a project or do not have a sponsor relationship

(1) Percentage of students assigned to preferred projects based on survey results (2) Percentage of students assigned to projects with appropriate skill/interest similarities

Target met

Decrease processing time Time taken to complete critical process steps Target met

Project P18711

Multidisciplinary Senior Design Page 4

For the scores to be calculated automatically, the project and student data needed to be collected in a standardized way. The pre-course survey for students was asking for information slightly different than the project template, and most answers were open ended. This forced the MSD Office to manually input project data as it was collected and to individually process each student survey response. A new Google Form survey and Project Readiness Package template were created for the office to use to collect project data in an automatic way. Figure 1 includes both the Google Form and Word template that will be sent to project proposers. These documents simplify the way that information is gathered and shared amongst MSD staff.

Standardizing this data will also allow the students to better understand the projects that will be available in the upcoming semester. The student pre-course survey requests that students choose their top five project choices, and if they have access to all the project information, then responses will be more accurate. Increasing the accuracy of student surveys should correspond to increasing student satisfaction on their assigned project.

Figure 1. Project Readiness Package template documentation

RESULTS AND DISCUSSION The MSD teams are assigned based on class enrollment, student survey data collected before the semester starts,

and project information reviewed by Department Leads. Figure 2 illustrates how data was flowing through the system of project proposers, students, MSD Office staff, and guides in the current state and what the desired future state will look like. In comparison, there will be a significant reduction in the amount of manual data inputs that will need to be made. Reducing these steps will limit the possibility of introducing human error in the system. Google Drive will be utilized in full potential to efficiently share data among all stakeholders. By having automatic access to files of this sort, the overall processing time should decrease with less manual in-between steps. Another major change is the elimination of the need to print out student or project information to make assignments. The data will be transferred directly into the Python optimization and resulting assignments will populate an Excel file.

Project P18711

Multi-Disciplinary Senior Design Conference Page 5

Figure 2. Process flow diagram before and after proposed improvements

The optimization program will be maximizing the student scores on projects. A score is calculated for each student on each project that represents the degree of fit based on comparing the standard project and student data. Two examples of how scores would be calculated for two different students on two projects can be found in Figure 3. The first student does not satisfy the major requirement of the first project and only has one of the skills needed for the project. The second student matches other constraints and fails to meet the intellectual property (IP) requirements. The last constraint pictured is a “Beth” constraint that allows the MSD Office to manually link a student to a project if there are extra matching factors.

Figure 3. Example of matching students to projects based on constraints

The score calculations are automatically calculated in one Excel file based on the input of three data files. A matrix is used to match all constraints of students and projects. Figure 4 demonstrates how these matrices are used.

Figure 4. Student skills and project skills matrices

The team aimed to create a resulting score that would best mimic the current assignment system. The following chart in Figure 5 states the weights of assignment factors chosen by the MSD Office. These scores are out of a total score of 100 meaning that student has a perfect match on a project. A score of 1000 could also be a result

Project P18711

Multidisciplinary Senior Design Page 6

of a student proposing a project or a student being manually assigned to a student for reasons determined by the MSD Office. Scores are calculated based on the students major matching a project position, the students’ interests matching project categories, the students’ skills matching skills required for a project, and the students top five preferences for which project they would like to be on.

Figure 5. Breakdown of score calculation

An example of the scores calculated for students in the Spring semester are included in Figure 6. Students with a score of zero fail to meet one or more of the binary constraints and students with scores above 35% match skills, interests, and/or top five preferences.

Figure 6. Example score calculations for 12 students on 6 projects

The complete assignment process was completed for students enrolled in Spring 2018. The actual assignments that were made were compared to the assignments that were simulated with the algorithm. Figure 7 summarizes the total scores for both the actual and simulated assignments. The 12 projects that were run in Spring either had higher or the same scores when simulated with the new assignment system.

Project P18711

Multi-Disciplinary Senior Design Conference Page 7

Figure 7. Average project scores for individual projects 1-12 and total project scores

Both the project data and the student data from the Fall 2017 semester were modified to match the new data formats. This means that an exact comparison could not be made between the actual assignments and the simulated assignments as some projects did not have all the required information and students were not able to definitively select their top project choices. Missing information was inferred where possible to create the following comparisons in Figure 8. It is important to note that only a sample of projects are graphed, there were a total of 70 projects in the Fall 2017 semester.

Figure 7. Subset of average project scores for individual projects and assignment accuracy comparison

CONCLUSIONS AND RECOMMENDATIONSThe proposed assignment system is ready to be implemented in the upcoming Fall 2018 semester. The MSD

Office is trained in use of the system and a detailed User Guide has been provided that fully documents the entire process. Documentation has also been provided that will aid the MSD Office to modify the system if any additional features were to be added or if existing functionality were to be changed.

The refined methods for collecting and analyzing student and project data remains manual at some critical steps. While the actual input of data is automated, the process to transfer that data to different locations still needs to be copied by the MSD Office. This method reduces the number of errors that could be introduced to the system, but does not eliminate them completely. An additional improvement that could be made would be to create an application or a website that is used to automatically collect and sort the data as needed. The MSD Team lacked the

Project P18711

Multidisciplinary Senior Design Page 8

software and website design skills needed to elevate the sophistication of the system. This project could be continued in upcoming semesters with Computer Science or Software Engineering students.

In the future, this project could also be used to address issues creating team assignments in similar circumstances like other capstone courses. Currently, the market is lacking online scheduling tools that can handle multiple majors, sections, and other specialized constraints. With a web based platform, the algorithm and optimization program could be customized to suit the needs of countless customers. ,

REFERENCES[1] Dr. Ohland, CATME Smarter Teamwork. http://info.catme.org/[2] Shi-Jie Chen and Li Lin, 2004, “Modeling Team Member Characteristics for the Formation of

Multifunctional Team in Concurrent Engineering”, IEEE Transactions on Engineering Management, 51(2).[3] Saaty, T.L., 2008, “Decision Making with the Analytic Hierarchy process”, Int. J. Services Sciences, 1(1)

pp. 83-98.

ACKNOWLEDGMENTS The team would like to thank our customer, Dr. Beth DeBartolo, and the rest of the MSD Office for working

with us closely throughout the past year to develop this system. We would also like to thank Dr. Katie McConky and Dr. Ruben Proano for their continued support in designing the algorithm and the optimization program. Finally, thank you John Kaemmerlen for guiding us through the design process.

Project P18711