Standford Poytner Project: Eye Tracking

STANFORD-POYTNER PROJECT - EYE TRACKING - INTRODUCTORY HIGHLIGHTS

There's lots to wonder about as Internet news readership grows markedly. Which provider sites, for instance, do readers consider "news" sites? Are they sites mainstream news purveyors would consider traditional?

Will Internet news readers give up serendipity? If they narrow their sources, will they stop encountering stories they hadn't intended to read? Will this cut down on their knowledge of the world?

Is it myth or fact that Internet news readers forego reading a broad spectrum in favor of personal-interest news only?

Web news providers gather statistics of what happens on their individual sites, but we wanted to learn how individuals behave across sites. What do they seek, what do they really read when they sit down at their computers and call up news? Do they read traditional suppliers of journalistic information, or mostly declaimers of opinions? Do they venture beyond narrow information sources?

We began this study four years ago, first videotaping Internet news readers in their homes and offices. We learned that they read both mainstream general news sources and traditional specialty news providers. They had often tried customized news, but given that up "because I might miss something I ought to know about." They read multiple news sites in about 30-minute sessions. They were news junkies, still reading newspapers, magazines, listening to radio news. Most of them had been reading online news for about one year or less.

Two years ago we wanted to learn more. Exactly which types of stories do Internet news readers look at most often? Do they read full articles or only headlines? How important are graphics and photos? How do they move around among sites?

A way to get more specific answers than one can from videotaping is to track eye movements. So we began another study. This one used eyetracking equipment that recorded where the eyes stopped to absorb information. That tells us what our subjects read. We also could track movement from site to site.

We had subjects use their own news bookmarks in as normal a fashion as possible, for as long as they wished. The eyes could be tracked as screens scrolled normally. This is the first such scrolling-screen eyetracking effort that we know about. We found that fewer subjects appear to be news junkies in the last two years compared to four years ago. At least this seems so as judged by their other news consumption habits: often they don't subscribe to daily papers; they don't do much TV news viewing; they do still listen to radio news programs; they have cancelled some magazine subscriptions.

Not many have cancelled newspaper subscriptions, but that's because some had given up subscribing before taking up online news reading. So online news has brought them back to the news reading fold. Most have been reading online for two or more years now. They still don't like customized news much, for the same reason as before: they might miss "something important."

What other questions did the team from Stanford University and The Poynter Institute ask of the data from 67 subjects in two cities? Here are a few highlights:

• Where do eyes go initially after firing up the first screenful of online news? To text, most likely. Not to photos or graphics, as you might expect. Instead, briefs or captions get eye fixations first, by and large. The eyes of online news readers then come back to the photos and graphics, sometimes not until they have returned to the first page after clicking away to a full article.

• Also contrary to much current belief, we found that banner ads do catch online readers' attention. For the 45 percent of banner ads looked at at all, our subjects' eyes fixated (definition) on them for an average one second. That is long enough to perceive the ad.

Graphics other than banners were looked at 22 percent of the time, and also received about a second's eye fixation. Sixty-four percent of photos were looked at on average about one-and-a-quarter second.

Whether text in and of itself really attracts eyes before artwork is difficult to conclude since, often, the text comes up before the graphics. Nonetheless, the provider's first chance to engage the reader is through text. Furthermore, the Stanford-Poynter eye tracking study does show a pattern in which text is sought out and either skimmed or read.

• Online news readers read shallow but wide, while at the same time pursuing selected topics in depth. We conclude this from several measurements:

1) More than three times as many briefs as articles were called up.

of 10

12/11/2001http://www.poynter.org/eyetrack2000/body.htm

2) Twenty four different categories of news as defined by section names were looked at. 3) An average six providers per session were viewed; in some cases, as many as 19 were viewed in one session. Sessions averaged 34 minutes.

The key measurement in judging that our subjects pursued selected topics in considerable depth is the vertical length to which subjects perused an article. Their eyes systematically went over more than 75 percent of the length of almost all those articles presented to them.

This may not be surprising. Most articles that subjects looked at had been purposely selected by them by clicking on a headline or brief. When reading a print newspaper, often serendipity motivates a reader to stop to read an article. That may lead to earlier abandonment.

Following are a few examples of what we learned from eye movements that told us that at least some reading (definition ) was taking place:

• We found that 30-year-olds were more likely to read local news than either 60-year olds or 20-year-olds. And 20-year-olds read more science and sports news than did other age groups. Virtually all ages read opinion articles in healthy proportion to their total article reading.

• Eighty percent of all participants read crime and disaster coverage. Females were very slightly more likely to read this category, but males were more likely to read more items. (These judgments are based on proportion of each gender to its number in the study: 30 females, 37 males.)

• Sports, surprisingly, was read equally by males and females - 70 percent of the total for each gender. But, no female read heavily in this category, while 11 percent of the men did. In fact, the heavy sports readers were likely to exceed the number of items read per person than in any other category.

• A higher proportion of women read local news than did men, and by a tiny margin also read more heavily in this category. Overall, 48 percent of all participants read local news.

• Somewhat more men than women read national news and by a small margin, also read more items. Overall, 67 percent of all participants did some reading of national news.

In another finding, we noted that interlacing news-provider (definition) sites is a strong news reading pattern online. That is, subjects often moved back and forth between sites rather than read them serially. This duplicates findings from the first round of this study made in 1997 with video cameras.

Forty percent of our subjects started a session with their local newspaper provider before moving on. And, of the nine most favored providers, seven were newspaper sites and two broadcast ones.

Not all "news provider sites" as defined by our subjects fit what news purveyors might consider mainstream general news sites. In fact, subjects called up non-general news sites twice as often as they did general news sites.

However, mainstream news sources have not been abandoned. Most of the specialty providers present news in a traditional manner that can be considered mainstream. Also, the aforementioned breakdown of favored providers demonstrates the strength of mainstream general news. Another indicator is the time subjects spent with each provider: the maximum was 20 minutes with a non-general news provider, while 46 minutes was the maximum spent with a general news provider.

The fact that Internet news readers are spending more time and activity with traditional news providers should be encouraging to newspaper editors who are in the Internet news world, too. Additionally, early indications seem to show that serendipity operates readers scroll Internet pages, just as it does when they turn the pages of print newspapers. Asked in the post-reading interview why they had chosen certain articles, a standard answer was: "I noticed the brief (or headline) and it seemed interesting."

Internet news readers are paying attention to the general world, too, not just to their personal interests. Two early indicators of this are the percentages of articles and briefs in the Opinions category (this includes opinions, editorials, letters to the editor, and columnists), which have some reading in them: 58 percent of briefs, and 97 percent of articles opened. Furthermore, 10 percent of our subjects deliberately chose to read foreign providers because they said they wanted get another perspective on the news.

We completed eyetracking data on 67 subjects in Chicago, Ill., and St. Petersburg, Fla., almost evenly split between

Readers and Heavy

Readers

Favored Providers

of 10


the two areas. The 67 subjects called up 211 unique providers for a total of 426 individual visits, counting each person's visits just once. Adding all visits together, participants visited or revisited sites 610 times. In total, they spent 40 hours looking at news on the Internet.

What else have we learned so far from these 67 subjects? Look at our Table of Contents to view what we viewed and to read the details.

Demographic

Data

of 10


THE STUDY

Two years ago Stanford University and The Poynter Institute researchers began collaborating to learn how frequent Internet news readers went about perusing news online. Key to the research would be tracking eye movement. Eyetracking tells us more precisely what the eyes take in than do survey questions that depend on recall. Exactly where do Internet news readers go to catch their news? Which stories do they read, which skim, which ignore? Do they read only headlines and briefs, or full articles? If they hyperlink to a related story, do they return to the original site? Learning the answers to questions such as these would, we hoped, begin to give us clues to Internet news reading behavior that could subsequently be correlated with civic action.

Beginning in December 1998 we ran a pilot study in the San Francisco-San Jose Bay Area to test the technical feasibility of recording eyetracking data while persons read Internet news online in real time. They used their own bookmarked sites in normal fashion, scrolling screens as they read, switching from site to site, entering forums if they liked, and so forth. After the bugs were worked out, we proceeded to Chicago and St. Petersburg, Fla., where we collected valid data on 34 and 33 subjects respectively for a total of 67. (We gathered data from 10 additional subjects, but they could not be calibrated.)

Subjects wore a lightweight SMI EyeLink eye tracking system that recorded all eye tracking data in a computer. This is a minimally intrusive, lightweight, head-mounted device. Small cameras attached to a mounting device illuminate eyes with very low levels of infrared light. They collect images 60 times per second. In addition to measuring the gaze-location of each eye, the EyeLink measures head position from a small camera attached to the computer's monitor. This camera determines where the subject is looking on the screen even as the head moves slightly. This ability was crucial to the study since subjects read news for long periods of time.

Specially written software developed by Stanford permanently captured both eyetracking movements and screen content to a computer for later viewing. Eye fixations were overlaid against the content. The length of time for each fixation and the order in which each occurred was also recorded.

Each screen viewed was subsequently coded in many different ways:

Here are some of the numbers we collected:

Demographic

Data

By format • Headlines boxes and individual headlines • Navigation boxes by shape • Articles • Briefs • Photos • Banner ads • Graphics (including ads and editorial graphics, not always easily distinguished from each other) • Related stories • Forums.

Format Categories

By topics • Crime & Disaster • Science & Medicine • Local • State • National • International • War • Lifestyle • Sports • Business & Technology • Opinions, Editorials, and Columns • Other

By action • Keyboard events (clicks, arrows, typing)

By visits and time spen t

• Providers • Sections • Pages (Urls)

of 10

12/11/2001http://www.poynter.org/eyetrack2000/toc/study.htm

We recruited our subjects through promo pieces in online newspapers local to the research sites. These ran in the Chicago Sun-Times in Chicago and the St. Petersburg Times in St. Petersburg. For the pilot test in the San Francisco Bay Area, they ran in the San Jose Mercury News.

Reading online news at least three times a week was the chief criterion in selecting subjects from those who responded to the promos. "Reading news" was defined as "reading a formatted news site similar to those provided by online newspapers or online broadcast sites. Beyond that, reading news is whatever you believe it to be."

Subjects were asked to bring their own news-provider bookmarks with them to the research site. These were uploaded to the site computer to recreate the user's own computer screen. Although a PC Windows based platform was used, some Mac users declared familiarity with Windows and were members of the subject group.

After fitting and calibrating the eyetracker, subjects were asked to read normally whichever news providers they usually read in the manner in which they usually read, for as long a session as was typical for them.

At the end of reading sessions, subjects were interviewed about their experiences during the session, as well as about their general media habits. By and large, subjects reported that reading news at the research site was substantially similar to their normal habits. Session times ranged from a minimum of 7 minutes to a maximum of 65. The average was 34 minutes.

Total number of subjects: 67

Total number of unique providers called up: 211

Total number of unique visits to providers: 426

Total visits to providers including revisits: 610

Total pages viewed: 5,963

Total unique pages viewed: 4,542

Total session hours during research: 40

Total eye fixations: 608,063

Total mouse clicks 24,530

Total boxes coded of all types: 9,441

of 10

12/11/2001http://www.poynter.org/eyetrack2000/toc/study.htm

FRONT PAGE ENTRY POINTS (INITIAL ANALYSIS)

Photos and graphics grab the eye first, right? Maybe wrong, for Internet news sites.

Text seems favored over artwork for front-page attention, a preliminary analysis of an eyetracking study appears to show. This conclusion comes from Stanford University and The Poynter Institute, collaborators on the first eyetracking study of Internet news reading on normally scrolling screens. (definition)

We found in our early analysis that more often than not, briefs or captions got the first eye fixations (definition) when the first page (definition) came up. Then the eyes came back to photos or graphics, sometimes after readers had clicked away to a full article before returning to the first page.

Let's show you an example (click on the titles to see the pages)

1. Initial Screen In this shot you see what came up first when subject 59 initiated his/her online news session: a fairly dramatic picture of the launch of a cruise missile in the Kosovo war, with a caption underneath; briefs on the war and on local reaction; vertical navigation listings in the left column and horizontally under the masthead; right column with lists of stories and a graphic.

2. Fixation Order The numbers superimposed over the pink blocks show the order in which fixation clusters (definition) occurred on this first, or front, page (definition). Follow the numbers and note the roaming pattern. Note, too, the dark "x" to the right of photo that indicates a mouse click for calling up a hyperlinked story. Density of color is not significant.

3. Returns to First Page Although the first fixation cluster does appear to left on the first photo (Fixation Order), contrast this with how the subject really looked at the photo after returning to the First Page. After subject clicked away to an article, then returns to the front page, note the groupings of eye fixations. In the first encounter with the front page (Fixation Orders), the first cluster landed in an area that more likely indicates attention on the left-hand column than the blank area of the photo. On the revisit, the reader's eye searched out the photo early on, then kept returning to it (note the numbering).

Now, to give you a fuller perspective, we'll show you the entire sequencing. (click on the titles to see the pages)

1. Initial Screen As outlined previously, in this shot you see what came up first when subject 59 initiated his/her online news session

2. Screen with Boxes Superimposed Over Content This shows coding boxes superimposed over content, with labels for each box.

3. Boxed Regions Without Content Here the coding outlines are shown without the background content, for easier viewing.

4. Logical Fixations The little red squares are individual fixations (definition). These fixations are about 200 milliseconds. To give you a measure of time, it takes about 250 milliseconds from seeing a light go on to pushing a button in response.

5. Fixation Order This shows the order in which the eyes fixated on the screen, as already described above. Density of color is not significant.

6. Fixations and Flat Right When fixations line up roughly from left-to-right, reading (definition) is presumed. Note red lines connecting such fixations.

7. Flat Right Coverage Order The large blocks of red show the order in which text was read on this front page (definition); the headline; the local brief; the caption under the photo; the general war brief.

of 10

12/11/2001http://www.poynter.org/eyetrack2000/toc/frontpg.htm

8. Story Reading Order We note from the numbers that the full article (datelined Belgrade) to which the reader clicked is read in sequential order. Density of color is not significant. (It may be necessary to scroll to the right to see the numbers)

9. Returns to Home Page On returning to the front page, the reader's eye searched out the photo early on, then kept returning to it, as described above. Note, too, that after scanning text for the first time the eye fixates on the graphic at top right column, as well as checking out the left navigation listing more thoroughly now.

Out of our 67 subjects, we picked every fifth to analyze their first pages in the above way. This gave us a random sampling of 14 from our pool. Three pages had no graphics or photos at all. Of the 11 remaining pages, 7 had either a graphic or photo, but these were not looked at until either the text was read or this page received a second visit. In the remaining 4 pages, a graphic was looked at first.

Of this last group, banner ads comprised three of the first-fixations clusters. One page with an initially looked-at banner ad also had a photo. While the banner ad was looked at right away, the photo was ignored until the subject returned to the page after having clicked away.

The remaining "looked-at-first" page contained a golf graphic. The subject looked at that first, then clicked on it to hyperlink to golfing information. An interesting sidelight is that a second subject also called up this page first. He/she, however, totally ignored this same graphic.

At this time, we have not carefully analyzed more front pages, and perhaps doing so will negate the early findings. However, quickly spot checking a few more pages appears to show similar results. Click on the headlines below to view two examples.

1. Brief Page The reader of this Brief Page ignores the graphics in right column and looks at the briefs sequentially.

2. Washington Post This subject pays no attention to the key element of the photo before fixating on text.

What to make of these preliminary findings? Perhaps that graphics and photos load more slowly than text. However, our subjects were connected to the Internet at high speeds through university or corporate networks. Or, perhaps that the smaller size of photos and graphics do not grab readers right away in the same way that larger artwork does in print newspapers.

However, overall, during the entire online reading session, graphics and photos did less well than text. Of all graphics, aside from banner ads and photos, available to a subject throughout a session, only 22 percent were looked at. Banner ads, which were broken out separately from other graphics, did somewhat better: 45 percent were looked at. And photos did best: 64 percent were looked at. These figures stack up against briefs and articles text, which were looked at 82 and 92 percent of the time, respectively.

We should also state that peripheral vision does take in information. Other eyetracking research has shown that some information not fixated on does enter memory, so we cannot guarantee that information was not absorbed outside our delineated areas. This

Articles text 92%

Briefs 82

Photos 64

Banner Ads 45

Graphics 22

of 10


might mean that graphics and photos are absorbed without eye fixations registering.

Nonetheless, whatever the explanation, it appears a provider's first and best chance to engage the reader is through text.

Aside from attention to photos and graphics on the first page, an interesting point we gleaned from examining the top screen (definition) is whether people scroll to continue reading briefs and headlines. They do.

We asked the question, "What's the percentage of briefs and headline boxes read on the top screen versus all that were available on subsequently scrolled screens?"

Out of 1,311 briefs available to our subjects in the top screens, 45 percent had some flat-right (definition), or reading behavior. This compares to a 56 percent ratio of some reading in the 2,951 total briefs available on all screens. A similar 10-point difference exists when we count top-screen headline boxes versus all headline boxes available. These figures, which we find unexpected, imply to us that reading attention continues beyond first screens as readers scroll.

of 10


FRONT PAGE ENTRY POINTS (UPDATED)

The next round of analysis is in and text remains the preferred point of entry over graphics among online news readers studied by Stanford University and The Poynter Institute.

This subsequent analysis probed more deeply than was possible in our first cut at the data released in May and examined the reading patterns of more of the study's subjects.

Among readers whose first glances on a page do include graphics, the most recent analysis shows they're more likely to fixate on banner ads or photographs than on information graphics or other forms of artwork.

The differences between text and graphics vary depending on how the data is measured, but the recent analysis confirms the general findings of preliminary results released in May.

That report caused a stir among many designers because, among other things, its text-before-graphics finding suggested a scenario of online reading that contrasts quite sharply with the generally accepted notion that graphics represent key entry points for readers of printed pages. (See the accompanying article by designer Alan Jacobson for a critique of our findings and methodology.)

To gain understanding of what draws readers to a page of online news, we used sophisticated camera equipment to capture readers' first three fixation-clusters, a term that describes usually overlapping glances that include the eye's precise point of focus and the immediately surrounding area. We looked at the first three fixation-clusters rather than just the first one in order to avoid a bias toward text that might be created on slower-loading pages.

In the latest analysis, we included a total of 168 pages containing both graphics and text, and examined how these pages were viewed by 66 of the study's 67 subjects. (Data from one subject was discarded because the results were unreadable.) In checking how many of the subjects' first glances included graphics, we checked the 504 fixation-clusters generated by the first three glances at the 168 pages.

We also checked to see how many of the 168 pages had graphic elements that were viewed among the first three glances. Finally, we examined the frequency with which readers included different forms of artwork -- banner ads, photographs or other graphics - in their first three glances.

The study's participants were recruited through notices published in the Chicago Sun-Times and the St. Petersburg Times. The subjects were experienced and regular users of the web who said they read news online at least three times a week.

The research was not designed to be predictive but as a beginning reality check of the ways regular online users view news within and across various sites. In effect, this is the message we're trying to send to the design community: Here's how 66 regular users of online news moved their eyes across the screens of their favorite news sites. What does this suggest about how we can most effectively design pages? What future research should be done to test and challenge this early reality check?

When we first began this research, we did not intend to focus primarily on the text vs. graphics question. We were expecting to learn which sections of front pages (e.g. top right, top left, etc.) drew the eyes first. Because we had most page elements boxed for identification, the eyetracking was able to show us specifically when the eyes focused on text vs. photos, banners, or graphics.

As we observed subjects moving into their selected pages, we observed a strong preference for text over graphics as entry points. To test that impression, we did a preliminary examination of approximately every fifth subject, or 14 of the original group. Of that group of 14, 11 viewed pages that included both text and graphics. We discovered that the first three fixation-clusters on seven of those pages included no graphic elements, with four of the pages including graphics among the first three glances. That represented a margin of nearly two to one favoring text entry points (64 percent) to graphics (36 percent).

By that same measure, the follow-up analysis of 66 subjects shows a slimmer margin, with 90 (54 percent) of the 168 pages including only text entry points among the first three fixation-clusters and 78 pages (46 percent) including graphic elements.

Our examination of all 504 fixation-clusters included among the first three glances at the 168 pages shows a stronger preference for text as an entry point, with just 112 (22 percent) glances at graphic elements included among the 504.

Because our research did not evaluate the quality of graphics or text displayed on the pages studied, it's impossible to determine what prompted the subjects to prefer text as starting points.

Entry point breakdown

of 10

12/11/2001http://www.poynter.org/eyetrack2000/toc/entrypoints.htm

Of the 78 pages generating initial glances at graphics, we found that 48 pages with one of the first glances going to graphics and the other two to text. On 26 pages, we found readers glancing at two graphic elements and one text element. On four pages, all three of the initial glances included graphic elements.

We found that 14 of the 66 subjects (22 percent) included no artwork among their first three glances at the pages they chose to view.

The preliminary findings released in May included only the home page of the first news provider read by the 14 subjects checked. For this follow-up analysis, we examined initial glances at the front pages of up to the first four news providers for each of the subjects.

In drawing conclusions from the research, it should be noted that eyetracking research has shown that some information absorption takes place beyond the area considered within an eye fixation-cluster.

So it is possible that artwork is perceived even if there is no direct fixation on it.

of 10

12/11/2001http://www.poynter.org/eyetrack2000/toc/entrypoints.htm

CATEGORIES

Number of visits to pages: 5483

Number of visits to providers: 610

Number of visits to sections: 1371

Format Categories Total number coded

Articles 750

Briefs 2951

Headline boxes 1096

Banner ads 1127

Other graphics 3413

Photographs 751

Individual Headlines 3168

Related stories 297

Topic Categories Articles Briefs

Crime & Disaster 102 302

Science & Medicine 40 84

Local 34 145

State 23 65

National 63 221

International 32 175

War 57 210

Lifestyle 125 589

Sports 72 479

Business & Technology 88 341

Opinions, Editorials & Columns

102 299

Other 12 41

Page 1 of 1

12/11/2001http://www.poynter.org/eyetrack2000/toc/study-categories.htm

DEMOGRAPHIC INFORMATION ON OUR SUBJECTS

Location

Florida 34

Illinois 33

TOTAL 67

Gender

Male 37

Female 30

Ages

20s 17

30s 24

40s 13

50s 9

60s 3

70s 1

Ethnicity

Caucasian 51

Asian 1

Hispanic 1

Black 5

Pacific 1

Native American 1

Mixed 2

Unknown 5

Education

High School 9

Associates 4

Bachelors 31

Masters 8

JD 3

PhD 1

Unknown 11

Occupation

Technical 10

Administration 17

Education and Media 10

Sales 6

Social Work 6

Top Executive 3

Blue Collar 3

Homemaker 1

Page 1 of 3

12/11/2001http://www.poynter.org/eyetrack2000/toc/study-demos.htm

Student 4

Unknown 7

Married

Yes 35

No 32

Unknown 0

Children in the Home

Yes 8

No 58

Unknown 1

U.S. Citizen

Yes 62

No 4

Political Party

Democrat 29

Republican 14

Reform 0

Independent 13

Libertarian 2

Green 0

None 7

Unknown 2

Home Hardware

Mac 4

PC 45

PC & Mac 4

PC & Other 1

None 1

Unknown 11

TOTAL 56

Office Hardware

Mac 3

PC 49

PC & Mac 1

PC & Other 0

None 1

Unknown 13

TOTAL 54

Where Read

Home 22

Page 2 of 3


Work 19

Both 18

Missing 8

Connect Speed

Dial-up 34

ISDN 0

T1 22

Unknown 6

Browser

Explorer 26

Netscape 30

Both 4

Netscape, AOL 1

Netscape, Explorer, AOL 1

Other 3

Unknown 2

Income

< $25,000 18

< $50,000 31

< $100,000 13

> $100,000 1

Unknown 4

An additional 10 subjects conducted reading sessions, five in Florida and five in Illinois; however, their eye tracking data were uninterpretable, so they are not included in any reports of results. There is no reason, demographic or otherwise, to believe that a systematic bias led to the failure of these sessions.

Page 3 of 3


Standford Poytner Project: Eye Tracking

Documents

Transcript of Standford Poytner Project: Eye Tracking