Teaching Operating Systems Using Code Reviecdall/pubs/sigcse2014_talk.pdf · Conclusions •...
Transcript of Teaching Operating Systems Using Code Reviecdall/pubs/sigcse2014_talk.pdf · Conclusions •...
![Page 1: Teaching Operating Systems Using Code Reviecdall/pubs/sigcse2014_talk.pdf · Conclusions • Implemented and evaluated GradeBoard • Students learn better with GradeBoard for 2 reasons:](https://reader033.fdocuments.in/reader033/viewer/2022042321/5f0b44037e708231d42fa9db/html5/thumbnails/1.jpg)
Teaching Operating Systems Using Code Review
Christoffer Dall and Jason Nieh Columbia University
![Page 2: Teaching Operating Systems Using Code Reviecdall/pubs/sigcse2014_talk.pdf · Conclusions • Implemented and evaluated GradeBoard • Students learn better with GradeBoard for 2 reasons:](https://reader033.fdocuments.in/reader033/viewer/2022042321/5f0b44037e708231d42fa9db/html5/thumbnails/2.jpg)
Learning OS is hard
• Operating Systems are large and complicated
• Hands-on experience is crucial
• Modification to large and unfamiliar code bases
![Page 3: Teaching Operating Systems Using Code Reviecdall/pubs/sigcse2014_talk.pdf · Conclusions • Implemented and evaluated GradeBoard • Students learn better with GradeBoard for 2 reasons:](https://reader033.fdocuments.in/reader033/viewer/2022042321/5f0b44037e708231d42fa9db/html5/thumbnails/3.jpg)
Teaching OS is hard
• Programming assignments must be reviewed
• Code is dense and spread out
• Disproportionate amount of time spent evaluating compared to teaching
![Page 4: Teaching Operating Systems Using Code Reviecdall/pubs/sigcse2014_talk.pdf · Conclusions • Implemented and evaluated GradeBoard • Students learn better with GradeBoard for 2 reasons:](https://reader033.fdocuments.in/reader033/viewer/2022042321/5f0b44037e708231d42fa9db/html5/thumbnails/4.jpg)
GradeBoard
• Code review system designed for the classroom
• Web-based, intuitive, easy-to-use
• Build on Git and ReviewBoard
• Supports entire workflow of teaching OS
![Page 5: Teaching Operating Systems Using Code Reviecdall/pubs/sigcse2014_talk.pdf · Conclusions • Implemented and evaluated GradeBoard • Students learn better with GradeBoard for 2 reasons:](https://reader033.fdocuments.in/reader033/viewer/2022042321/5f0b44037e708231d42fa9db/html5/thumbnails/5.jpg)
Outline
• Usage Model
• System Architecture
• Evaluation
• Conclusions
![Page 6: Teaching Operating Systems Using Code Reviecdall/pubs/sigcse2014_talk.pdf · Conclusions • Implemented and evaluated GradeBoard • Students learn better with GradeBoard for 2 reasons:](https://reader033.fdocuments.in/reader033/viewer/2022042321/5f0b44037e708231d42fa9db/html5/thumbnails/6.jpg)
Usage Model
• Instructors and students
• Instructors create, distribute, and grade programming projects
• Students download, backup, collaborate, and submit programming projects
![Page 7: Teaching Operating Systems Using Code Reviecdall/pubs/sigcse2014_talk.pdf · Conclusions • Implemented and evaluated GradeBoard • Students learn better with GradeBoard for 2 reasons:](https://reader033.fdocuments.in/reader033/viewer/2022042321/5f0b44037e708231d42fa9db/html5/thumbnails/7.jpg)
WorkflowNew Assignment Create template
repositoryDistribute student
repositories
Student WorkGit push to serverDeadlineUpload to Review
Board
Instructors grade assignments
Release comments
Students discuss comments
![Page 8: Teaching Operating Systems Using Code Reviecdall/pubs/sigcse2014_talk.pdf · Conclusions • Implemented and evaluated GradeBoard • Students learn better with GradeBoard for 2 reasons:](https://reader033.fdocuments.in/reader033/viewer/2022042321/5f0b44037e708231d42fa9db/html5/thumbnails/8.jpg)
The Web Interface
![Page 9: Teaching Operating Systems Using Code Reviecdall/pubs/sigcse2014_talk.pdf · Conclusions • Implemented and evaluated GradeBoard • Students learn better with GradeBoard for 2 reasons:](https://reader033.fdocuments.in/reader033/viewer/2022042321/5f0b44037e708231d42fa9db/html5/thumbnails/9.jpg)
The Web Interface
![Page 10: Teaching Operating Systems Using Code Reviecdall/pubs/sigcse2014_talk.pdf · Conclusions • Implemented and evaluated GradeBoard • Students learn better with GradeBoard for 2 reasons:](https://reader033.fdocuments.in/reader033/viewer/2022042321/5f0b44037e708231d42fa9db/html5/thumbnails/10.jpg)
Instructor FeedbackGrade Student A
Grade Student B
Grade Student C
Grade Student D
Time
Release all grades
![Page 11: Teaching Operating Systems Using Code Reviecdall/pubs/sigcse2014_talk.pdf · Conclusions • Implemented and evaluated GradeBoard • Students learn better with GradeBoard for 2 reasons:](https://reader033.fdocuments.in/reader033/viewer/2022042321/5f0b44037e708231d42fa9db/html5/thumbnails/11.jpg)
![Page 12: Teaching Operating Systems Using Code Reviecdall/pubs/sigcse2014_talk.pdf · Conclusions • Implemented and evaluated GradeBoard • Students learn better with GradeBoard for 2 reasons:](https://reader033.fdocuments.in/reader033/viewer/2022042321/5f0b44037e708231d42fa9db/html5/thumbnails/12.jpg)
System Architecture
Git ReviewBoard
Management
![Page 13: Teaching Operating Systems Using Code Reviecdall/pubs/sigcse2014_talk.pdf · Conclusions • Implemented and evaluated GradeBoard • Students learn better with GradeBoard for 2 reasons:](https://reader033.fdocuments.in/reader033/viewer/2022042321/5f0b44037e708231d42fa9db/html5/thumbnails/13.jpg)
![Page 14: Teaching Operating Systems Using Code Reviecdall/pubs/sigcse2014_talk.pdf · Conclusions • Implemented and evaluated GradeBoard • Students learn better with GradeBoard for 2 reasons:](https://reader033.fdocuments.in/reader033/viewer/2022042321/5f0b44037e708231d42fa9db/html5/thumbnails/14.jpg)
Review Board
• Web-based interface
• Commercially maintained open-source tool
• One trivial change to the authentication system to prevent students from modifying their submissions
![Page 15: Teaching Operating Systems Using Code Reviecdall/pubs/sigcse2014_talk.pdf · Conclusions • Implemented and evaluated GradeBoard • Students learn better with GradeBoard for 2 reasons:](https://reader033.fdocuments.in/reader033/viewer/2022042321/5f0b44037e708231d42fa9db/html5/thumbnails/15.jpg)
Management Tool
• Abstracts away low-level commands to classroom commands like: • create-homework • distribute-homework • upload-reviews
• Keeps single central list of registered students
![Page 16: Teaching Operating Systems Using Code Reviecdall/pubs/sigcse2014_talk.pdf · Conclusions • Implemented and evaluated GradeBoard • Students learn better with GradeBoard for 2 reasons:](https://reader033.fdocuments.in/reader033/viewer/2022042321/5f0b44037e708231d42fa9db/html5/thumbnails/16.jpg)
OS @ Columbia• 6 two-week programming assignments
• Modifying the Linux kernel for Android
• Intro to C, system calls, synchronization, virtual memory, scheduler, file systems
• Live demonstrations
• Careful code review
![Page 17: Teaching Operating Systems Using Code Reviecdall/pubs/sigcse2014_talk.pdf · Conclusions • Implemented and evaluated GradeBoard • Students learn better with GradeBoard for 2 reasons:](https://reader033.fdocuments.in/reader033/viewer/2022042321/5f0b44037e708231d42fa9db/html5/thumbnails/17.jpg)
Evaluation
• Used GradeBoard at Fall 2011 Columbia OS course
• Over 100 registered students
• Survey participation: 57%
![Page 18: Teaching Operating Systems Using Code Reviecdall/pubs/sigcse2014_talk.pdf · Conclusions • Implemented and evaluated GradeBoard • Students learn better with GradeBoard for 2 reasons:](https://reader033.fdocuments.in/reader033/viewer/2022042321/5f0b44037e708231d42fa9db/html5/thumbnails/18.jpg)
GradeBoard Improved Homework Understanding
92%
8%
No Yes
![Page 19: Teaching Operating Systems Using Code Reviecdall/pubs/sigcse2014_talk.pdf · Conclusions • Implemented and evaluated GradeBoard • Students learn better with GradeBoard for 2 reasons:](https://reader033.fdocuments.in/reader033/viewer/2022042321/5f0b44037e708231d42fa9db/html5/thumbnails/19.jpg)
Preferred compared to e-mailed score sheets
81%
19%
No Yes
![Page 20: Teaching Operating Systems Using Code Reviecdall/pubs/sigcse2014_talk.pdf · Conclusions • Implemented and evaluated GradeBoard • Students learn better with GradeBoard for 2 reasons:](https://reader033.fdocuments.in/reader033/viewer/2022042321/5f0b44037e708231d42fa9db/html5/thumbnails/20.jpg)
Preferred compared to comments in Git
81%
19%
No Yes
![Page 21: Teaching Operating Systems Using Code Reviecdall/pubs/sigcse2014_talk.pdf · Conclusions • Implemented and evaluated GradeBoard • Students learn better with GradeBoard for 2 reasons:](https://reader033.fdocuments.in/reader033/viewer/2022042321/5f0b44037e708231d42fa9db/html5/thumbnails/21.jpg)
Encouraged well-formatted code
46%54%
No Yes
![Page 22: Teaching Operating Systems Using Code Reviecdall/pubs/sigcse2014_talk.pdf · Conclusions • Implemented and evaluated GradeBoard • Students learn better with GradeBoard for 2 reasons:](https://reader033.fdocuments.in/reader033/viewer/2022042321/5f0b44037e708231d42fa9db/html5/thumbnails/22.jpg)
–Nicolas Viennot, Head TA Fall 2009
“I wish I had this tool when I taught operating systems last year.”
![Page 23: Teaching Operating Systems Using Code Reviecdall/pubs/sigcse2014_talk.pdf · Conclusions • Implemented and evaluated GradeBoard • Students learn better with GradeBoard for 2 reasons:](https://reader033.fdocuments.in/reader033/viewer/2022042321/5f0b44037e708231d42fa9db/html5/thumbnails/23.jpg)
Unexpected Benefit
![Page 24: Teaching Operating Systems Using Code Reviecdall/pubs/sigcse2014_talk.pdf · Conclusions • Implemented and evaluated GradeBoard • Students learn better with GradeBoard for 2 reasons:](https://reader033.fdocuments.in/reader033/viewer/2022042321/5f0b44037e708231d42fa9db/html5/thumbnails/24.jpg)
Conclusions• Implemented and evaluated GradeBoard
• Students learn better with GradeBoard for 2 reasons: • Comments inlined with student work • Student feedback identifies problem areas
• Over 80% of all submitted projects were discussed
• Free and maintained
• Students and instructors prefer GradeBoard over known alternatives
![Page 25: Teaching Operating Systems Using Code Reviecdall/pubs/sigcse2014_talk.pdf · Conclusions • Implemented and evaluated GradeBoard • Students learn better with GradeBoard for 2 reasons:](https://reader033.fdocuments.in/reader033/viewer/2022042321/5f0b44037e708231d42fa9db/html5/thumbnails/25.jpg)
Questions?
![Page 26: Teaching Operating Systems Using Code Reviecdall/pubs/sigcse2014_talk.pdf · Conclusions • Implemented and evaluated GradeBoard • Students learn better with GradeBoard for 2 reasons:](https://reader033.fdocuments.in/reader033/viewer/2022042321/5f0b44037e708231d42fa9db/html5/thumbnails/26.jpg)
Backup Slides…
![Page 27: Teaching Operating Systems Using Code Reviecdall/pubs/sigcse2014_talk.pdf · Conclusions • Implemented and evaluated GradeBoard • Students learn better with GradeBoard for 2 reasons:](https://reader033.fdocuments.in/reader033/viewer/2022042321/5f0b44037e708231d42fa9db/html5/thumbnails/27.jpg)
Potential Solutions
• Grade Sheets: Manual and tedious
• Inline comments: requires use of command-line tools, time consuming
• Difficult for students to use
![Page 28: Teaching Operating Systems Using Code Reviecdall/pubs/sigcse2014_talk.pdf · Conclusions • Implemented and evaluated GradeBoard • Students learn better with GradeBoard for 2 reasons:](https://reader033.fdocuments.in/reader033/viewer/2022042321/5f0b44037e708231d42fa9db/html5/thumbnails/28.jpg)
Problem Space
• Most hands-on OS assignments are based on some existing code base
• Both teaching OSes and commercial OSes
• No read-from-start-to-finish
![Page 29: Teaching Operating Systems Using Code Reviecdall/pubs/sigcse2014_talk.pdf · Conclusions • Implemented and evaluated GradeBoard • Students learn better with GradeBoard for 2 reasons:](https://reader033.fdocuments.in/reader033/viewer/2022042321/5f0b44037e708231d42fa9db/html5/thumbnails/29.jpg)
Problem Space: Options
• Live demonstrations
• Automated Testing
• Doesn’t work, must evaluate the code