“The Fantastic Four” ­ Sloane S, Marie V, Tina V, Karna C

Studio: Crowdpower 1:30

Low Fidelity Prototyping

Mission Statement

EnGauge gives students a platform to anonymously and immediately give feedback to

professors, and allows professors to address this feedback during the lecture.

Problem/Solution

Through our needfinding, we discovered that professors need a way to gauge the level

of understanding of all, rather than just some, students in large lectures, and students need a

way to stay engaged and take action to try to understand the material when they are confused.

Through EnGauge, students can show their level of understanding in real time as well as

anonymously ask clarifying questions, and the professor can see their feedback.

Tasks: Student Interface

1. Student to give feedback to professor based on the speed of class

2. Student to ask questions anonymously to professors

3. Students to upvote the questions they like to be answered

Tasks: Professor

1. Professor to know what is the state of student

2. Professor knows which questions to answer

3. Professor can analyse the class understanding in different topics taught

Interface Sketches

Fig 1 An overview of the many sketches we created in our brainstorming session.

Fig 2­3 Clockwise, starting at top left: the professor and student interfaces working together; the question­asking view.

Fig 4­5 The crowdsourced meter for professors; the ranking view.

Storyboards

Storyboard 1: Professor Interface 1

Fig 6 Meter at top to show current understanding of class. Line plot breakdown of class

understanding after class.

Storyboard 2: Professor Interface 2

Fig 7 Gauge to show current understanding of class. Pi charts of each topic to breakdown class

understanding after class.

Storyboard 3: Student Interface 1

Fig 8 Green speed up, red slow down, yellow happy face just right. Best question in an

especially large font.

Storyboard 4: Interface 2

Fig 9 Top and new questions on same screen.

Selection of User Interface:

Of our storyboarded ideas, we chose Student Interface 1 for the student side of the UI.

We made our selection on the basis of readability and ease of use. Since students will be using

the product during lecture, we wanted the cleanest and simplest interface such that usability

remained obvious. We decided that the top versus new question choices in Student Interface 2,

while helpful in allowing students to see all questions, ultimately detracted from the ability to

quickly scan questions. Student Interface 1 biases towards top rated questions, but in doing so

pushes unnecessary distracting questions out of the way. Student Interface 1 also provides

clear button commands both in color and shape (as opposed to words).

For the professor side of the UI, we chose Professor Interface 1. By using a meter, the interface

limits the space used up during lecture while remaining readable. The line plot after lecture

provides a cohesive view of how the lecture went, as well as a breakdown of specific topics,

whereas the breakdown of Professor Interface 2 provides only a breakdown by topic. The

storyboard for Student Interface 1 above shows two task flows: changing the lecture speed of

the professor and asking a question. The storyboard for Professor Interface 1 shows two

different tasks flows: gauging the current understanding of the class (through meter and

questions) and gauging the overall understanding of the class after the lecture.

Prototype Description

Student User Interface Prototype

We used paper cut­outs slightly larger than an smart phone screen. The user was

presented the prototype as a smartphone app, and thus used a touch screen to interact with the

app.

The student’s first task was to use the red, yellow, and green buttons to show their level of

understanding. The student’s second task was to either ask a question using the ask button, or

upvote one of the existing questions.

Professor User Interface Prototype

We used full paper for the professor prototype to represent a laptop or iPad screen. This

prototype included a slide view, as the professor would be going through a slide deck while

teaching, with the components of our prototype around the slide view.

The professor’s first task was to view the interface and respond according to the

student’s needs: slow down if the bar at the top is red, speed up if it is green, and answer the

questions that appear on the questions bar. We switched the bar by switching out small green,

red, and yellow pieces of paper according to the buttons the student pressed. We used pieces

of paper with questions written to show an incoming question.

The professor’s second task was to view the overall progress of the class. We presented

the professor with a graph that showed how the average student level of understanding as time

progressed. When the professor tapped on a dot representing a question, we slid a sticky note

with a question from that class onto the screen.

Method

We sourced three Stanford students to test our prototype, codenamed S, R, and A (for

privacy reasons). R and S are both CS 106 section leaders, so they were particularly good

testers for the professor’s view of EnGauge. The environment was simple: Tina acted as the

second half of the professor­student exchange, learning from our user when they were acting as

the professor and teaching a brief history of Stanford Co­Op houses when the user was acting

as a student.

During each test, they modeled our first two tasks as both professors and students. As a

student, they used the gauge to speed up and slow down and as the professor they changed

the pace of the lecture given feedback. Similarly, they asked questions through the app and

then answered the top questions from the professor view. Finally, only as professors, were they

able to view the post­class report and see how the class performed over the course of the

lecture.

Through observation, we tried to determine how intuitive and useful the app was. We

used a few basic measurements, including how long it took for the user to interact with the app

for the first time, how frequently they looked at or were distracted by the app, and how many

questions ‘students’ submitted or ‘teachers’ answered via the app.

Results

Regarding the task gauging the speed of the class, students got confused (Pic 7.1) by

up/down arrow. They didn’t know what it meant: they understood the red button stood for

confused but not slow down. Similarly they understood green stood for understanding but not

speed up. One student was confused by the similarity in icons between the green speed up

button and the green upvote button next to the questions. They also didn’t see any value of the

Yellow Button. Similarly, the professor was confused by the yellow bar: they thought it meant

slow down rather than doing good, where yellow meant slow down and red meant slow down

extremely. They instead saw green as doing great.

For the task of asking relevant questions students did not understand that the “Ask”

button meant asking your own question, until they clicked on the button. ( Pic 7.2). They

assumed that they would need to select an existing question, and click ask to ask it. They easily

understood the concept of upvotes, but were confused with how it related with the ask button.

However, they struggled to get to the questions page on their own at the first time, mainly

because they were too involved in the stoplight buttons. At the same time, the professor

immediately started answering the questions asked from the platform.

For the task of reviewing the report, the professor liked the dots and questions part of the report.

But, it was not obvious to them to immediately press them. They understood the graph and were

interested in clicking the high and low graphs.

Discussion and Proposed Changes:

Based on the discussion and feedback, we decided to remove yellow button in Fig 7.1.

We needed something which indicated too high of a speed vs too low of a speed. We are

considering substituting it with a Hare and Tortoise button to indicate the speed of the class (Fig

8.1).

Also, we needed a clear distinction between the “?” button and red arrow in Pic 7.1 to

give our users more clarity of function of the ask questions button. We think it is worth

considering replacing the “Ask” button with the “Ask New Question” button.

For the task of viewing report, we think it is important to substitute the simple dot

representing a question (see Fig 7.3) with a symbol like Fig 8.2.

Appendix I Testing Script

Rough bullet­pointed script of how we interacted with each participant.

­ Welcome them to the test and explain that we are working on a product that is used

during lectures to help professors and students better engage in the lecture. They’ll be

testing the student interface followed by the professor interface. Get them to sign

consent form.

­ Student interface:

­ Give them initial “gauge” view as Tina starts the “lecture” on Stanford Co­Op

history. Tina will explain how she will be using the product to get further

engagement from the audience.

­ Wait for gauge interaction depending on how fast/slow Tina is going

­ When the user switches to question view,

­ Flip in submit screen on ASK press

­ Put on sticky note of their new question

­ Professor interface:

­ Prompt them to give the first 10 minutes of their most reason section

­ Tina will become the student for this portion

­ Computer will “bubble up” gauge measurements and questions to user

­ Stay out of their way as they present and see if they answer the questions

­ Computer will show pre­prepared “report” view if they use report button

Appendix II Notes

Test 1

Student:

understood which button meant only because we mentioned speed up slow down

thought arrows were too similar to upvote

rabbit vs tortoise? something that shows fast vs slow

understood how to ask questions

ask new instead of just ask ­ makes it seem like you can pick one of given

questions and click ask

read first question of the pre­written questions and only answered that one

first tried to ask a question in person before interacting with app (one on one situation)

understood upvoting

Professor:

understood the purpose of the meter

helpful as weighted average of a lot of people where you can’t tell by expression whether

on average people are confused

clicked on points but did not immediately think to do so

would look at the extremes on the graph ­ high and low understanding

Test 2

Student:

pressed smiley face button when teacher asked is everything ok

understood the function of all buttons

did not press question button initially

thought the same thing about the ask button ­ that you press a question then press ask

vs pressing upvote

confused about how to return back to stoplight screen

Professor:

understood red meant slow down

yellow feels like slow down even though it means ok

green means good

sees green = good, yellow = a little slower, red = slow

liked the dots and questions but would not immediately think to click the dots

would press all the dots

Appendix III Additional Images

Fig 7.1 ­ The initial task screen for giving class speed feedback

Fig 7.2 ­ The screen to ask questions

Fig 7.3 ­ Report view for professor where hovering meant seeing a question

Fig 8.1 ­ Proposed Icon change for Too fast/Too slow

Fig 8.2­ Proposed change to “Question” symbol instead of dot in Report mode for

Professor

Appendix IV Consent Form The EnGauge application is being produced as part of the coursework for Computer Science course CS147 at Stanford University. Participants in experimental evaluation of the application provide data that is used to evaluate and modify the interface of EnGauge. Data will be collected by interview, observation and questionnaire. Participation in this experiment is voluntary. Participants may withdraw themselves and their data at any time without fear of consequences. And concerns about the experiment may be discussed with the researchers (Marie Vachovsky, Tina Vachovsky, Sloane Sturzenegger, Karna Choksi) or with Professor James Landay, the instructor of CS147: James A. Landay CS Department of Stanford University, landay@stanford.edu. Participant anonymity will be provided by the separate storage of names from data. Data will only be identified by participant ID. No identifying information about the participants will be available to anyone except the researchers and their supervisors. I hereby acknowledge that I have been given an opportunity to ask questions about the nature of the experiment and my participation in it. I give my consent to have data collected on my behavior and opinions in relation to the EnGauge application. I understand I may withdraw my permission at any time Name ______________________________________________ Date _______________________________________________ Signature____________________________________________ Witness name ________________________________________ Witness signature_____________________________________