Online Market Research

© 2001 Ann Schlosser, University of Washington Business School

Agenda

• What accounts for differences in actual and forecasted numbers?

– Survey panels vs. server log files

– Response and sampling biases

• Online market research methodologies

– VR/Simulated environments

– E-mail surveys

– ISP’s data warehouse (unobtrusive)

– Quasi-experimental design

• Using the results

Who Has a Reliable Estimate of a Site’s Audience?

Internal Server

Third-Party Ratings FirmsThird-Party Ratings Firms

For pageviews, ratings-firm numbers were anywhere from 85% less than to 300% more than that reported by the site’s server.

Comparing Audience Measurement Firms Using Panel Data

COMPANY U.S PANEL SIZE

RECRUITMENT METHOD

INCENTIVE

Media Metrix

55,000 Random-digit dialing, follow-up mail

$50 annual maximum, giveaways, sweepstakes

Nielsen NetRatings

65,000 Random-digit dialing $50 U.S. savings bonds every six months

PC Data 120,000 Random-digit dialing, advertising

$40 annually, sweepstakes

ComScore 1.4 million

Advertising, partnered with pollsters

Provides faster Internet-page download service

The Trouble With Online Panels

Some Benefits of Panels

• Level of detail– Geography– Unique visitors

(different people accessing the site)

– Captures PC activity

Internal Server Records are Imperfect

• Tallies computer (not human) access– Over counts due to:

• Same user with home and work access

• Spiders and other bots

• ISPs that indiscriminately assign users to IP addresses

– Under counts due to: • Different users of same computer

• Cached pages– Especially problematic if done by ISPs and other content

aggregators

Some Factors Influencing the Reliability of Data

• Response biases

• Sampling biases

• Incentives

Impact of the Internet on Market Research

Kannan et al, 1998

Virtual Shopping Market Research

Comparing Electronic With Mail Surveys

E-mail Mail

Response rate 6%-73% 27%-56%

Response speed 7-8 days 7.5-13 days

Completion Less More

Bad address notification 10 minutes Weeks

Percent of bad addresses 24.5% 2%

Weible & Wallace (1998)

Collecting Data Through ISPs while Protecting Privacy

ISP login/street address Anonymous ID,Geocode,Append Demog.

FoveonCollection (now Plurimus)

login Anonymous ID DataWarehouse

street address

= security key

3rd Party

50 participating ISPs and 3.5 million Web users

Raw Material: Collector Lines

23540 12:04:30 com.egghead.www /store2/ent/eggs_ordstat.browse23540 16:30:57 com.egghead.www /store1/ent/images/space.gif23540 12:04:34 com.egghead.www /media/bnr/B_EggheadSpecials_0823540 12:04:42 com.egghead.www /store2/ent/images/space.gif23540 16:30:57 com.egghead.www /media/images/bdot.gif23540 16:30:11 com.egghead.www /media/images/but_previous.gif23540 16:30:11 com.egghead.www /store1/ent/eggs_shop.additem?s23540 16:30:55 com.egghead.www /store1/ent/eggs_shop.additem?s23540 16:30:32 com.egghead.www /media/images/portal_egg-header23540 16:30:57 com.egghead.www /media/images/recalculate.gif23540 16:30:08 com.egghead.www /store1/ent/eggs_prod.browse?se23540 16:30:07 com.egghead.www /store/ent/eggs_prod.browse?sec23540 16:31:03 com.egghead.www /media/images/but_proceed-order23540 12:04:34 com.egghead.www /media/images/but_nav_auction[323540 12:04:34 com.egghead.www /media/images/but_search.gif25150 12:21:41 com.yahoo.rd /results/a/?http://www.altavist25150 14:41:43 com.yahoo.rd /results/a/?http://www.altavist26004 22:07:53 com.yahoo.rd /search/navbar/top/*http://ink.

FID Date/Time Host URL

Classification of Behavior

23540 OCT16:200012:04:30 com.egghead.www2 /store2/eggs_shop.additem

•Duration•Time of Day•Calendar Day

•www.egghead.com•“Internet Computers”•NAICS Code

•“Shopping Cart”

•Geography (BG ID)•Demography

Geocoding/Appending

CalculationSite Classification

URL Classification

FID Date/Time Host URL

Log File Data

User Start Date/ End Date/ Packets Bytes Host URLID Time Time To/From To/From

GBF vs. GIRF:Reasons for Dot-Com Failures

Poor revenue, cost investment and profit models 61%

No competitive advantage 44%

No consumer benefit 21%

Organization, implementation and execution problems 17%

Channel conflict 10%

Ineffective fulfillment process 7%

Source: BCG

Various Performance Metrics

• 65% abandon shopping carts

• Overall average buyer conversion rate < 4%

• Low customer conversion, upselling and loyalty, even at best sites– At 10 leading e-commerce sites

• 20% converted from visitors to buyers

• 10% upsold

• 5% returned within 6 months

Retention Measures

KEY RESEARCH QUESTIONS

– Are you reaching the right audience?

– Do they remember your website?

– Does it enhance the brand’s image?

– Does it increase sales?

• What is the objective(s) of your website?

Web Site Report

Page Views by Hour

Site Navigation Summary Report

Disentangling the effect of your website

• A quasi-experimental design is the best means of isolating the impact of a website from the numerous external factors which may impact attitudes and opinions

Online Marketing Traditional Marketing

website

Peer influence Press influence

Pre-existing attitudes

Methodology

=Website effect

Experimental Effect

First time users

Experimental Group

- Repeat visitors

Control Group

User accesses website www.your.website

JavaScript sampling procedure

Visitor continuessurfing, unaware of the sampling script

Not Sampled

Survey 1

SampledReturned to

the page

originallyrequested

Survey 2

E-mailinvitation

User accesses website www.your.website

JavaScript sampling procedure

Visitor continuessurfing, unaware of the sampling script

Not Sampled

ONLINE QUESTIONNAIRES

• What should you ask?– Questions relating to your website’s objectives

– Audience profile questions

– Satisfaction questions

– Relationship questions

– Intent questions

– Comparative questions

Type of Information

Available7.65

Type of Information

Available7.65

Detail of Information

Available6.98

Detail of Information

Available6.98

The Layout of

the Site7.84

The Layout of

the Site7.84

Navigation Aides on the Site

6.75

Navigation Aides on the Site

6.75

Speed of Loading Pages7.10

Speed of Loading Pages7.10

Overall Overall SatisfactionSatisfaction

I=.000 I=.533 I=.39 I=.272 I=.000

Customer Satisfaction ModelVisit Again

Visit Again

Recommendto others

Recommendto others

Spend more time

Spend more time

Download Information

Download Information

I=.120I=.120I=.320I=.420

Customer Satisfaction Model

0.00

0.10

0.20

0.30

0.40

0.50

0.60

5.00 6.00 7.00 8.00 9.00 10.00

CRITICAL IMPROVEMENT STRENGTH

LOWER LEVERAGE AREALOWER PRIORITY

Detail of information

available

Navigation aides on the

site

Layout of the site

Type of information available Speed of loading the pages

SATISFACTION WITH FEATURE

IMP

AC

T O

N O

VE

RA

LL

SA

TIS

FA

CT

ION

Strategic Improvement Matrix