[ID] Week 07. Modeling

Lecture 5

Modeling

Interface Design/ COM3156, 2017 Fall Class hours : Fri. 1-4 pm Lecture room : #210 Billingsley Hall, Main Campus 13th October

SURVEY RESULTS ID Class Team-Up

Lecture #5 COM_Interface Design 2

6 Teams


I II III IV V VI

Team Lead

Project Title

Members

Presentation Bullet Points

• Team Members & Project Title

– Expected roles

– Basic Project Ideas

• Research Components

– Who are (or could be) the users?

– Where/When do they use(or access/approach/meet/recognize)?

– Why will they return to it?

– Technological Keywords & Issues.


DESIGNING FOR THE DIGITAL AGE: HOW TO CREATE HUMAN-CENTERED PRODUCTS AND SERVICES CHAPTER 10. MAKING SENSE OF YOUR DATA: MODELING

Lecture #5


Introduction

• Model – a description that helps people

understand and communicate about

observed behavior.

– Bohr's model of the atom and Freud's

ego, superego, and id, for example, give

us frameworks for understanding the

complex ideas they stand for.

– Similarly, modeling the results of your

research will help you condense and

visualize information to understand

human behavior patterns, workflows,

and trends.


Introduction


Bohr's Model of the Atom Freud's ego, superego, and id

Synthesizing Stakeholder Findings

• Topics to cover


Figure 10.1. Overview of activities during the modeling phase.


• Topics to cover

– What kinds of users and customers do stakeholders think are most

important?

– What do they think the product is?

– What do they expect to ship and when?

– What presumed constraints exist, and which may be flexible?

– What should the project accomplish for the business?

– What do stakeholders think success is and what do they think will be

required to achieve it?

– What barriers to success do stakeholders expect?



• Topics to cover

– What brand values should the product or service communicate?

– How are the organization and the product or service positioned versus the

competition?

– What concerns do you have about the answers to the above questions and

what should you do about them?

– What disagreements about any of the above topics can you help resolve?

– What unrealistic expectations, if any, should you discuss with the project

owner?

– What else did you hear that might affect how you proceed?

– Who are the most influential stakeholders and what will you (and the design)

need to do to satisfy them?



• Preparing to communicate stakeholder findings

– Analysts will prepare information; executives will mostly consume it.

Because analysis takes considerable time and expertise, no matter how clear

the interface, stakeholders expect that the bulk of the work will continue to be

done by analysts who clean the data, select appropriate queries to run against

it, format it for easy consumption, and perhaps point out interesting issues.

Executives are the primary consumers of the data, but can only engage with it

through static reports today. The next release is expected to provide some

ability for executives to sort, filter, and format the post-analysis data on their

own. Stakeholders believe this is important because younger executives are

more accustomed to tweaking documents or slides for themselves.



• Preparing to communicate stakeholder findings

– Hopes for executive user engagement vary. We heard a wide range of

views regarding how central the tool would be to executives' work. Some

stakeholders expect executives to view reports every month or so, while

others envision daily use even when executives are out of the office.


Analyzing Customer and User Data

• Qualitative analysis

– Qualitative analysis is critical for design because it excels at

explaining why and how, as well as what. Human intuition does play a role;

there's nothing wrong with making intuitive leaps as long as you pause

and examine the data to see if it actually fits the structure your

subconscious has proposed. Good qualitative techniques make those

intuitive leaps easierand help you determine whether your leaps are

correct.




– On every project, you should also be looking at the data at varying levels of granularity.

Just as you can't understand an animal without understanding its ecosystem, or an

ecosystem without understanding the organisms that comprise it, you need to know the

individuals, as well as the whole data set, to understand what your results really mean.

– This is why it's best to start with single-case analysis, which is just what it sounds like:

focusing on understanding what you heard and saw with one individual at a time. This

helps ensure that you and your teammates not only understand what you saw and heard,

but also have a good idea of why each person thought and behaved as he did.

– Once you're well grounded in the individual cases, you can move on to cross-case

analysis, which involves grouping and comparing the individual cases to identify trends

and behavior patterns. This involves comparison of individuals in most instances, but you

may aggregate further to compare types of organizations (such as small versus large

companies) if you're designing enterprise software.





Figure 10.2. Overview of analysis process.



– TECHNIQUES COMMON TO SINGLE-CASE AND CROSSE-CASE AND ANALYSISIS (1/3)

• Deductive reasoning starts with an existing general principle or hypothesis and compares the data to it.

For example, if you assumed in your interview planning that having small children would give people

more reason to take and share photos, you would be looking at your interview notes to see if the

observed behavior supported your hypothesis. Clearly, it's important with deductive reasoning to avoid

bending the data to fit your hypothesis.

• Inductive reasoning, on the other hand, involves trying to derive a general principle from specific data.

If you saw in your data that serious photographers tend to delete a larger percentage of their images

than casual photographers do, you would look in the data for explanations of why that might be the case

and try to turn it into a general statement, such as, "Serious photographers tend to take many photos to

make sure they capture a scene as they 'see' it. Their quality standards are exacting, so they delete any

photos that don't meet them." The primary danger with inductive reasoning is that it's easy to get false

positives; just because every serious photographer in your sample owns a particular camera does not

mean that all serious photographers do.




– TECHNIQUES COMMON TO SINGLE-CASE AND CROSSE-CASE AND

ANALYSISIS (2/3)

• Displaying your data in various ways will also help you understand it, derive

insight from it, and communicate with your teammates and stakeholders about

it. While displaying your data is probably most useful in cross-case analysis, it

also helps you be honest with yourself in single cases; if you can't find a way

to express a process or relationship in concrete terms (whether visual or

textual), that means you probably don't understand it well enough to proceed.




– TECHNIQUES COMMON TO SINGLE-CASE AND CROSSE-CASE AND

ANALYSISIS (3/3)

• Finally, while it's not a good idea to force your data into a simile or metaphor,

you might find yourself using one to describe a process, place, role, or mind-

set in either single-case or cross-case analysis. If you do, stop and examine it,

because metaphor is a type of model we apply to understand and explain the

world around us; chances are, your subconscious has drawn some

connection that you need to examine.




– SINGLE-CASE ANALYSIS

• Single-case analysis is primarily about ensuring that you and your teammates have a

thorough (and shared!) understanding of what you've seen and heard. You probably don't

need to communicate about single cases with anyone outside the design team except as

examples to illustrate trends, so this part of the analysis can be entirely informal.

• To walk through a single case, review your field notes with your fellow interviewer(s),

discuss what you think the behavior and comments mean and why (with particular

attention to anything that was puzzling or unexpected), and note any disagreements or

alternate explanations. If possible, start this type of analysis on an informal basis

between interviews so you can use it to guide subsequent interviews. Ten or 15 minutes

between sessions and a few minutes at the end of an interview day are enough for an

experienced team to do this preliminary analysis; less experienced teams may need a

dedicated hour or two a day. Continue single-case analysis after the interviews are done.




– SINGLE-CASE ANALYSIS

• Reducing and organizing your data (coding)

• Summary notes

• Articulating models within a case




– Social scientists begin single-case analysis by categorizing each

comment or observation. They call this coding, but you won't hear many

designers refer to it this way because of the association with writing

software code. The formal version involves assigning a category to every

respondent statement or observed behavior. Check-coding, in which two

researchers individually code the data and then compare and merge their

work, is generally accepted as more accurate and objective than a single

researcher coding alone. Although it's far less formal, this is essentially

what happens in a design team meeting in which everyone reviews her

own notes and discusses her thoughts about an interview.




– This categorizing process is both deductive and inductive, in that there are

certain categories of information you would expect to find in your notes, while

the data itself may suggest other categories. These category codes and the

quantity of data can become hard to manage in large academic studies, so it's

common for ethnographers or anthropologists to use spreadsheets or

specialized software to help them sort individual quotes or observations by

code. The average design project involves less data and fewer categories, so

the organizing tool may be no more sophisticated than scribbles in the margin

of your notebook. For the most part, the codes you're likely to use on every

project mirror the questions you ask in research, such as:



• Goals

• Frustrations

• Skills

• Frequency

• Quantity

• Priority

• Interactions with others

• Mental models

• Demographics

• User physical characteristics



• Requisition

• Hard goods

• Services

• Vendor selection criteria

• Payment terms

• Delivery terms

• Negotiation

• Follow-up


Of course, there are always categories unique to each project, such as particular process steps or types of documents used. For an enterprise purchasing application, for example, your notes might also include the following categories:



Interview transcript Codes Participant: Or I end up traveling somewhere that someone else I know has traveled recently. Choice of destination

(driven by trusted guide)

Second interviewer: Why is that better? Participant: It's just so much easier when somebody can give you a few pointers to get started. I tend to travel in such a way that I have a couple destinations that I'd like to go to and maybe they helped me figure out, oh, you should go to this landmark because there's that, plus there's other stuff around it if you don't like that. And then I tend to ... "Ok, we're going to go somewhere." But I never really stick to going there, so if something else more interesting shows up or whatever, I like to be very flexible, but still a destination to at least get started towards.

Reliance on trusted guide

Part planned, part flexible

Primary interviewer: When you're traveling like that, how do you decide what other things to do? So if you planned to go one place, how do you find other things to do?

Participant: Yeah, I guess it depends on how you're moving about. So I had gone to Venice last November, and my friend and I had both looked in some tour books. So we went to Venice, which is a pretty small place, and so we had picked out a couple of things like, "Oh, we really need to go see this, and this." And we were there for an event. Again, this is like, I like going somewhere where there are people involved.

Tour books Part planned, part

flexible Choice of destination

(driven by business trip)

Primary interviewer: What was the event? Participant: Architectural Biennial show that was put on by one of the universities there. So, we knew that that event was going to take most of the time, and so we had a little standing list of things that when we had free time, we should do. Check out the major square there. Go to the major shopping street. There was a glass blowing ... island actually. An island with glass blowing workshops, which we did not make it to.


Second interviewer: But you wanted to? Participant: Yeah, it was definitely on the list. But I guess that's the thing. I don't like making the list solid because then I regret if we didn't get somewhere. But maybe there's something more interesting.




Interview transcript Codes Primary interviewer: So once you had some free time, and you wanted to get to one of these places, how did you find it?

Participant: We were looking at tourist maps in tourist books for the most part. Navigation (maps, books)

Primary interviewer: Guidebooks that you brought with you? Participant: Yeah, guidebooks. And attempting to ask directions. Navigation (books,

asking directions) Primary interviewer: Neither of you spoke the language? Participant: No, not really. Primary interviewer: How did that work out? Participant: We'd usually figure out where we were going eventually. One of the people who had ... so I went with a friend and there was another group of people who met us there. And when we finally connected with them, one of them spoke better Italian and had been to Venice before. Again, using people is the easiest way. So he would direct us. I think the hardest part became that you'd get somewhere, and then getting back was usually the hardest part.

Language Trusted guide Navigation


• Summary notes

– useful if you have many people doing separate interviews or if you're expected to share

interview details outside the design team. A summary of the interview segment above might

look more like this:


Choice of destination: Chooses destination opportunistically (if she's there on business) or because people she knows and trusts have visited and enjoyed it.

Planning: Favorite source is a trusted human guide who is from the destination or has visited it. Also looks at travel books and Web sites for ideas. May read fiction related to the area. Makes a tentative list of activities to ensure that she doesn't waste time deciding what to do, but assumes it's flexible.

Selecting activities: Uses her list as a starting point. Chooses other activities that look interesting once she gets there. Prefers activities involving people. May walk by an interesting place, see an ad, or hear about it from a local.

Your team should be able to illustrate the basic activity flow you observed and describe the criteria involved at each decision point.

Navigating: Uses local tourist maps and tour books to get around. Often asks locals for directions, but language can be a barrier. Uses hotel as primary reference point.



Figure 10.3. A design team using interview photos to help with single-case review.



– Modeling the contents of each interview can help you understand and interpret your

observations. After an effective interview, your team should be able to illustrate the

basic activity flow you observed and, when relevant, describe the criteria involved at

various decision points. Activity diagrams and decision trees like those illustrated

in Figures 10.4and 10.5 can be good ways to do this. Such diagrams include the kinds of

documents, objects, or pieces of information the respondent uses and what the

respondent does with them in what sequence. It's usually not critical to cover every tiny

detail.

– If activities are complex, you might try sketching out this kind of model toward the end of

a user interview and reviewing it with your respondent to make sure you haven't missed

or misunderstood anything. This is a good technique for novice interviewers, in any case.




Figure 10.4. This simple diagram illustrates the process a shopper goes through before making a purchase.

Figure 10.5. This diagram illustrates the process a purchasing agent goes through to choose a vendor. Note that he considers timeline first, then a combination of two factors (quality and past performance), with price coming last.



– The other thing you should be able to articulate is each

respondent's mental model of the world as it relates to your design

problem. This includes what the respondent calls various objects, how he

defines them, and how he views their relationships; this is often called

a taxonomy. For example, an individual shopper's taxonomy of house

wares might look like the one in Figure 10.6. Note how some items, such

as mixers, fit in multiple categories—categories in a taxonomy need not

be exclusive. Card sorting, described in Chapter 9, is a useful way to

extract a taxonomy, though it's often possible to do so implicitly during the

course of an interview.




Figure 10.6. An example taxonomy.



– A mental model is a bit more than a taxonomy, though; it also includes the way that

someone imagines a process or structure to work (as opposed to what actually happens).

– Note that the example activity diagram, decision tree, and taxonomy shown here aren't

particularly formal; they're just thinking and communication tools for members of a small

team with shared experiences, so there's no need to whip out Visio or concern yourself

with the rules of UML.

– Also note that being able to articulate these things for every interview doesn't

necessarily mean you need to spend the time to do so; a skilled team that's used to

working together probably doesn't. If you're not sure, you can try it for your first few

interviews; if you're able to do it easily and you're not finding disagreements among

interviewers, then it's probably not the best use of your time. If you're not able to outline

workflows and mental models or if you have significant disagreements, you need to

sharpen your interviewing skills.


https://products.office.com/en-gb/visio/flowchart-software

https://en.wikipedia.org/wiki/Unified_Modeling_Language



– CROSS-CASE ANALYSIS

– Cross-case analysis helps you identify behavior patterns and trends.

– For design, the richest form of cross-case analysis results in personas, which are

composite models of user behavior patterns (see Chapter 11).

– However, you will also want to identify trends and general issues in your data to

help yourself and stakeholders understand in general terms how processes work,

what kind of data you're dealing with, what potential users often struggle with, and

so forth.

– These themes become your user and customer findings, which lay the groundwork

for people to understand and accept the personas and requirements.




– CROSS-CASE ANALYSIS

– One manageable way to start is to select two or three respondents who strike you

as similar to one another, then discuss exactly what is similar or different about

them. Affinity diagrams and composite models are also common tools.

• Affinity diagrams

• Composite models




– The most common inductive

approach is to build an affinity

diagram.

– There are varying points of view

about the best way to develop an

affinity diagram, from whether it's

done in silence to how many colors

of sticky notes are required; there

are even software packages for

managing affinity diagrams.


Figure 10.7. Sticky notes make it easy to create and rearrange clusters.



– There's really no need to make a big production of it because the

essential concept is simple: Each interviewer gets a pile of index cards

or sticky notes and writes one respondent comment or behavior on each.

– The design team then clusters those notes with others that seem similar

and tries to derive a category, underlying issue, or concept that

describes each cluster




Data points in the cluster (individual sticky notes) Idea or category that ties them together (cluster of sticky notes)

Alice sends photos of her little boy to her parents at least once a week via e-mail Jorge e-mails pictures of the new baby to grandparents in Mexico

Sending via e-mail

Ben posts photos of the twins on Flickr every few days for his sister Alice posts pictures on her Web site for friends to see if they're interested

Posting to Web sites

Alice organizes photos by date Ben organizes by date, sometimes with an event name (like "Christmas") Dan assigns attributes to each photo that describe contents, lighting, or other aspects of the image Stacy stores photos in folders by year, then by month Ellen puts photos in folders by subject (birds, mammals, plants, etc.), then by subtopic (seagulls, songbirds, etc.)

Organizing

Table 10.1. Example affinity clusters.


• Composite models

– You may also find it useful

to develop composite

activity diagrams, decision

trees, taxonomies, or other

illustrations of behavior

across cases.

– These are most useful in

communicating about the

behavior of different

personas.


Figure 13.9. These slides and document pages illustrate several ways of expressing requirements.


• Quantitative analysis

– PREPARING YOUR QUALITATIVE DATA

– UNDERSTANDING QUANTITATIVE DATA

– GAINING INSIGHT QUANTITATIVELY



• PREPARING YOUR QUALITATIVE DATA

– Qualitative samples are usually much smaller than those in quantitative studies, so it's

important to represent your findings in a way that isn't misleading. Saying that 12 of your

16 respondents did a certain thing lets people decide for themselves whether they think

that's true of the general population; expressing that number as a percentage hides the

fact that it's a small sample and may imply that you did a quantitative study.

– This kind of primitive quantitative analysis may also point to the need for quantitative

data. If you're designing an e-commerce Web site and you find that most of your

interviewees spend very little money and a few interviewees spend a lot of money, a

quantitative study can help you determine whether it's more lucrative to focus on one or

the other.



• UNDERSTANDING QUANTITATIVE DATA

– Most people take the results of quantitative studies at face value, but this

can be misleading. For example, whenever there's an election coming up,

there's always a news story when one candidate or another is gaining in

the polls. The news agencies may report that 31% of likely voters favor

candidate A and 32% favor candidate B. Even disregarding the fact that

many people change their minds when they get to the ballot box, these

numbers may very well be wrong because the difference (1%) is less than

the potential for error in the sample.




– The standard error (sometimes described as the margin of error) is an

estimate of how far your answer is from the "real" answer due to sampling

error in a random sample. If the standard error in the poll is 3%, then what

the poll really says is that 28 to 34% of people favor candidate A and 29 to

35% favor candidate B, so either one could really be ahead by several

points.




– Standard deviation is a measure of confidence in the data; in other words, it describes

the probability that the data can be used to predict future results. A large standard

deviation means the measured values are spread far from the mean (also known as the

average), so the probability of accurate prediction is low. A small standard deviation tells

you that the measured values are clustered closely around the mean, so predictability is

high.

– Let's use income as an example. If you surveyed people with similar education and

experience in similar jobs at similar companies in one city, you'd have a low standard

deviation because the individual responses are within a narrow range, which means you

have a good chance of accurately predicting the salary of someone similar to your

respondents. If you used a truly random sample of people across an entire country or

region, then the values would vary much more, so you'd be much less able to make an

accurate guess at an individual's real salary.




Figure 10.8. In the figure on the left, the standard deviation is relatively small because nearly all of the values are in a narrow range. In the figure on the right, the values are more spread out, which means the chances of predicting an individual's salary are lower.



– Determining confidence for a normal distribution (the classic bell curve

where values cluster around the mean and taper off toward the high and low

ends) is easy: There's a 68% probability of a value being within the range

defined by the standard error and a 95% probability of a value being within

twice that range. In other words, if you measured an average income of

$100,000 and had a 5% error with a normal distribution of results, it would

indicate there's a 68% chance that someone's income is between $95,000 and

$105,000 and a 95% chance that it's between $90,000 and $110,000. With a

bimodal or other type of distribution, it gets more complicated, so you'll want

to call up an expert or pull out that statistics textbook.




For a value that is sampled with an unbiased normally distributed error, the above depicts the proportion of samples that would fall between 0, 1, 2, and 3 standard deviations above and below the actual value.



https://youtu.be/cgxPcdPbujI

The Normal Distribution and the 68-95-99.7 Rule

https://youtu.be/cgxPcdPbujI



Figure 10.9. In a normal distribution, values cluster around the mean and taper off on either side. In a bimodal distribution, there are two clusters; for example, you may find a certain behavior exhibited by the very young and the very old, but not by adults in their middle years.


• GAINING INSIGHT QUANTITATIVELY

– Cross-tab analysis

• Insight generally comes from comparing variables in a table, such as age versus

attitude or geography versus access to services. This is called cross-tabulation or

cross-tab analysis.

• The idea is to look for any apparent correlation between two or more

characteristics, such as age, income, or region versus time spent online.

• The common variances that are useful in interview planning—such as variation by

environment (geography, company type, or family structure), age, or experience—

are often good starting places.

• For example, in Table 10.2, there's nothing at all interesting about comparing the

behavior of men and women, but you can see some clear trends when you compare

the behavior of people with different primary reasons for taking photos in Table 10.3.




Photos taken in a day Times per year photos are taken

Photos shared at a time

Times per year photos are shared

Men 25 16 21 13

Women 27 14 17 12

Photos taken in a day Times per year photos are taken

Photos shared at a time

Times per year photos are shared

Watching children grow up

4 75 12 31

Capturing vacation and special events

35 6 43 6

Creating perfect, expressive images

276 24 2 3

Table 10.2. Men versus women.

Table 10.3. People with different reasons for taking photos.


• Graphs and charts

– People often use line or bar graphs to

represent aggregate data (such as the

relationship between education and income, or

number of customer support calls versus time

spent with a product) because their visual

nature makes it easier for some people to grasp trends.

– Scatterplots, which represent individual data

points in a graphical fashion, may point to

relationships that are particularly hard to see

in textual format. If you label each point, you

may realize that people of certain types are

clustered together and may be related in some

way you didn't anticipate.


Figure 10.10. An example scatterplot.


• Explanations and relationships

– As you bounce back and forth between types of analysis, you'll begin

formulating explanations for why your respondents behaved as they did

and how various aspects of their attitudes, goals, and behaviors are

related.

– Drawing meaning from your data is the whole point of analyzing it; mere

summary won't accomplish much.

– Sometimes you can draw those explanations from within a single

interview, but in most cases you'll gain more insight from comparing

respondents to one another.



• Explanations and relationships

– Any attempt at explanation should rely in part on the classic rules for

determining causality:

• A precedes B.

– If A is the cause, it exists or occurs before B. In the case of human behavior, this

would mean that a goal, attitude, or condition exists before the demonstrated

behavior exists.



• When A, always B.

– If A causes B, in theory, B should occur any time A occurs. In reality, human

behavior is very complex, so don't stop looking if it seems that B

doesn't always follow A; sometimes it doesn't happen because some other

condition exists. For example, it might seem that skilled photographers always

throw out any photo that doesn't meet their standards, but one respondent kept

some bad images of his son. Does that explode your theory? Not really; that

photographer kept those awful photos because they were the only documentation

of an important event in his child's life, so you can say, "When A, always B, except

when C."



• A plausible mechanism links A and B.

– If there is a believable connection between A and B and the other conditions are

met, it's reasonable to assume that A causes B. "Enjoyment of shopping leads to

more frequent shopping" makes sense, but "having blue eyes leads to more

frequent shopping" doesn't make sense, so it's probably a coincidence if your data

shows that people with blue eyes shop more than others.



• Preparing to communicate your user findings


Buyers chose their automobiles in widely varying ways, but most decisions

involved emotion to some degree.

Some people selected a car because they felt it expressed something they wanted to

convey about themselves, whether because they liked its design or identified with the attributes of the

brand. These buyers tended to be decisive and not very research oriented; one said she purchased a

Jaguar because she "liked the kitty on the hood." Most people said they choose their vehicles for

various practical reasons, such as cargo capacity, safety, fuel efficiency, or budget. Some of these

buyers said they may have started with a leaning toward one model for emotional reasons, but would

not have bought it if it did not meet their other criteria; they seemed to feel that emotional reasons

were not a valid basis for making such a decision. This group varied with respect to the amount of

research they did; research orientation did not appear to be related to any particular factor. A small

group, which mostly consisted of men under age 40, was very focused on performance characteristics,

such as engine size and handling.


• Preparing to communicate your user findings

– Common topics for a findings discussion include:

• User mental models, especially where they differ from current

implementations

• An overview of existing processes and major points of pain within them

• Trends, behavior patterns, and the factors that influence them

• User skills or characteristics, especially if they differ from expectations

• Comparison of customer and user needs


Homework


“Preparing and Ready for Modeling Report”

1

- Collecting Data - Recruit at least 5 people - Set a plan for

interview/observation - Execute on-site researches

- Analysis - Qualitative - Quantitative

- communicate your user findings and summarize your discussion.

Submission Due : 11: 59 pm Sun. 19th October

Mid-Term Presentation Guideline

• Date/Time/Duration

– Friday, 20th October 2017

– 1 – 4 pm

– 20 mins for each team [15 min Presentation/ 5 min Q&A ]

• Presentation Document Content

– System Concept Statement

– Data Collection

– Interview


Mid-Term Submission Guideline

• Post on your team blog.

– Due until 11:59 pm (Thur) 19th Oct.

– Embed the presentation file from slideshare.


[ID] Week 07. Modeling

Design

Transcript of [ID] Week 07. Modeling