1

Ira Haimowitz

Director / Team Leader, Data Innovation

INFORMS NYC, February 2006

Ira Haimowitz

Director / Team Leader, Data Innovation

INFORMS NYC, February 2006

Marketing-Driven Measurement of Internet Sites and Online Media

Marketing-Driven Measurement of Internet Sites and Online Media

2

Ira Haimowitz Personal BackgroundIra Haimowitz Personal BackgroundIra Haimowitz Personal Background

Ph.D. Computer Science, M.I.T. 1994Artificial intelligence for medical diagnosis

General Electric R&D 1994-1998Data mining for financial services, retail

Software for “film-less radiology”

At Pfizer U.S. Pharmaceuticals 8 years:Director/ Team Leader, Data Innovation

President, Pharmaceutical Management Science Assn.www.pmsa.net

Ph.D. Computer Science, M.I.T. 1994Artificial intelligence for medical diagnosis

General Electric R&D 1994-1998Data mining for financial services, retail

Software for “film-less radiology”

At Pfizer U.S. Pharmaceuticals 8 years:Director/ Team Leader, Data Innovation

President, Pharmaceutical Management Science Assn.www.pmsa.net

3

Measurement of TV ad impactMeasurement of TV ad impact

Optimization of media spending vs. goalsOptimization of media spending vs. goals

ResultsResults

Redesigned web site helps people explore content in-depthRedesigned web site helps people explore content in-depth

4

First Click

Seco

nd C

lick

Before After

Content Explored Deeply in Re-designed SiteContent Explored Deeply in Re-designed Site

5

Motivations

Prerequisites

Data mining

Marketing-driven measurements

Evaluating media

Motivations

Prerequisites

Data mining

Marketing-driven measurements

Evaluating media

OutlineOutline

6

Complete multimedia DTC tracking analytics

Feedback to marketers in terms they understand“Is the web site working?”

“Who are the customer segments visiting?”

“What is the quality of traffic?”

“What are the predominant paths to goals?”

Continual web site improvementSystemic analytics for continuous discovery and improvement

Complete multimedia DTC tracking analytics

Feedback to marketers in terms they understand“Is the web site working?”

“Who are the customer segments visiting?”

“What is the quality of traffic?”

“What are the predominant paths to goals?”

Continual web site improvementSystemic analytics for continuous discovery and improvement

MotivationsMotivations

7

No personally identifiable informationHIPAA compliant

Results viewed in aggregate

Analysis – no DB scoring

Strict opt in policy

No personally identifiable informationHIPAA compliant

Results viewed in aggregate

Analysis – no DB scoring

Strict opt in policy

DisclaimerDisclaimer

8

Print, TV, Radio Promotion

Search Engines

Banner Ad

Prescription or Sample Package

RxDrug.com

Request for Information

On-line Survey

Loyalty Program

SourceSource Web SiteWeb Site GoalsGoals

Data Mining:Is the Website Really Working?Data Mining:Is the Website Really Working?

Informative Page

9

Hits, visits/sessions, page views, top pagesDo not capture behavior

Minimal insight for improvement

Largely measure the quantity from a promotion, not the quality of lead

Hits, visits/sessions, page views, top pagesDo not capture behavior

Minimal insight for improvement

Largely measure the quantity from a promotion, not the quality of lead

The Beef With Traditional MetricsThe Beef With Traditional Metrics

10

RelevantRelevantDiscrete EventsDiscrete Events

InsightsInsightsAssociationsAssociations FunnelsFunnels

Actionable AnalyticsActionable Analytics

Marketing GoalsMarketing Goals

Market SegmentsMarket Segments

DimensionsDimensionsSite Im

provement

Site Improvement

11

PrerequisitesPrerequisites

Truly integrated approach to relationship marketing across all media

Goal-oriented designEngineered for measurement

Cross-functional, Interdisciplinary teams (Statisticians, Marketers, DB Managers, Customer Response Managers, Creative Agencies)

Truly integrated approach to relationship marketing across all media

Goal-oriented designEngineered for measurement

Cross-functional, Interdisciplinary teams (Statisticians, Marketers, DB Managers, Customer Response Managers, Creative Agencies)

12

Discovering knowledge

that is actionable

automatically

from large databases

Multidisciplinary nature

Artificial intelligence

Data warehousing

Statistics, operations research

Overview of Data MiningOverview of Data Mining

DataDataSourcesSources

SurveyResponses

Business Rules

Web Logs

Media Spend

Ext. Data

DataDataPreparationPreparation

Extraction

Cleansing

Enrich

Aggregate

Impute

Transform

Extraction

Cleansing

Enrich

Aggregate

Impute

Transform

Data AnalysisData Analysisand Data Miningand Data Mining

ReportsReports

MiningMining

ReportingReportingOLAPOLAP

DeviationsDeviationsTrendsTrends

SegmentationSegmentationClusteringClusteringSimulationSimulation

OptimizationOptimization

Changed Market Conditions

Changed Market Conditions

The Data-Mining ProcessThe Data-Mining Process

Insights andDeploymentInsights andDeployment

Redesign site or Change CampaignRedesign site or

Change Campaign

Assistance: Alexander Linden, Gartner GroupAssistance: Alexander Linden, Gartner Group

14

Data Preparation for Web MiningData Preparation for Web MiningPreparing web analytical data mart :

Cleaning up web logs / tags

Dividing into sessions

Determine relevant time span of analysis

Fields related to behavior– Referring page– First few pages visited– Indicators of visiting certain sections– Indicators of achieving goals

Other Data Sources Registration database

Survey results

Media spending

Preparing web analytical data mart :Cleaning up web logs / tags

Dividing into sessions

Determine relevant time span of analysis

Fields related to behavior– Referring page– First few pages visited– Indicators of visiting certain sections– Indicators of achieving goals

Other Data Sources Registration database

Survey results

Media spending

15

Data Mining Yields More Relevant On-Line MetricsData Mining Yields More Relevant On-Line Metrics

Segment distributions Do site visitors correspond to marketing objectives?

AssociationsCan we “cross market” different aspects of our site?Are there unforeseen correlations?

Path AnalysisAre visitors taking the paths we intended?

Predictive modelingWhat are the key drivers of reaching goals?

Segment distributions Do site visitors correspond to marketing objectives?

AssociationsCan we “cross market” different aspects of our site?Are there unforeseen correlations?

Path AnalysisAre visitors taking the paths we intended?

Predictive modelingWhat are the key drivers of reaching goals?

16

Web Site Business SegmentsWeb Site Business Segments

Unbranded Interest

Unbranded Interest Branded

InterestBranded Interest

NeitherNeither

16%16%

37%37%47%

Interest

RegistrantsRegistrants

Non-RegistrantsNon-Registrants

Registrants

95%

5%5%

17

Traffic behaviorTraffic behavior

Content Areas ViewedContent Areas Viewed

Goal PagesGoal Pages

C1C1

CkCk

Web Data Mining: Behavior-Based SegmentationWeb Data Mining: Behavior-Based Segmentation

C2C2

18

Customer transactional DBCustomer transactional DB

Association Rules: Retail Association Rules: Retail

Customer ID Product123 Cola123 Pretzels123 Chips134 Diapers134 Cola134 Bandaids134 Apples245 Pretzels245 Cola367 Pretzels367 Hotdogs367 Cola…

Rule:

If Customer buys Cola, then also buys Pretzels

Confidence = 35%

Support = 6%

Rule:

If Customer buys Cola, then also buys Pretzels

Confidence = 35%

Support = 6%

Re-arrange shelf space to leverage associationsRe-arrange shelf space to leverage associations

19

Customer transactional DBCustomer transactional DB

Association Rules: Web Sites Association Rules: Web Sites

Rule:

If Session includes Condition page,

then session also includes screening page

Confidence = 35%

Support = 6%

Rule:

If Session includes Condition page,

then session also includes screening page

Confidence = 35%

Support = 6%

Session ID Page URL 123 Condition_home.htm123 See_doctor.htm123 Branded_drug_home.htm134 Side_effects.htm134 Branded_drug_home.htm134 See_doctor.htm134 Screening.htm245 See_doctor.htm245 Condition_home.htm367 See_doctor.htm367 Screening.htm367 Condition_home.htm…

Re-arrange links to leverage visitor associationsRe-arrange links to leverage visitor associations

20

Path AnalysisPath Analysis

Home

Product.htm

Condition.htm45%

25%

Screening.htm

Comorbid.htm

35%

30%

PI.htm

Brochure.htm

10%

20%

Thousands of paths, few desirableThousands of paths, few desirable

21

Visualizing Associations and SequencesVisualizing Associations and Sequences

Page associations

Session sequence

22

Total Sessions: 100% (10,000)

Material Request: 10.7% (1,070)

Form for Materials 8.5% (850)

Materials Receipt: 0.7% (70)

Total Sessions: 100% (10,000)

Material Request: 10.7% (1,070)

Form for Materials 8.5% (850)

Materials Receipt: 0.7% (70)

Funnels: Drop-offs to Goal Pages Funnels: Drop-offs to Goal Pages

Material Request Conversion FunnelMaterial Request Conversion Funnel

Simplify pathway to widen the funnelSimplify pathway to widen the funnel

23

Optimizing On-Line Media TacticsOptimizing On-Line Media Tactics

How to allocate a fixed on-line media budget?

Banner Ads, Sitelets, Search Engine Optimization, Content Integration, Sponsorship, etc…

How to allocate a fixed on-line media budget?

Banner Ads, Sitelets, Search Engine Optimization, Content Integration, Sponsorship, etc…

What is the volume of visitors each tactic sends to the site(s)?

What is the value of the visitors once they arrive?

24

Value of a Web Site VisitorValue of a Web Site VisitorRates of meaningful marketing behaviors

Goal page completion

Registrant

Desirable segment

Likelihood of NRx

Measure promotional cost per behavior for each campaign tactic

Rates of meaningful marketing behaviorsGoal page completion

Registrant

Desirable segment

Likelihood of NRx

Measure promotional cost per behavior for each campaign tactic

25

Extending to Integrated Media AnalyticsExtending to Integrated Media Analytics

Use web site to measure the effectiveness of a complete DTC campaign

TV, Print, Banner Ads, Patient Samples

Challenge: identify visitors to site as from specific campaign

Subdirectories-specific to campaign

Isolated media expansion

Use web site to measure the effectiveness of a complete DTC campaign

TV, Print, Banner Ads, Patient Samples

Challenge: identify visitors to site as from specific campaign

Subdirectories-specific to campaign

Isolated media expansion

26

Modeling Websites as Markov Processes: Research with Bronx Science Intel studentsModeling Websites as Markov Processes: Research with Bronx Science Intel students

Transition MatrixTransition Matrix

Click VectorsClick Vectors

27

SummarySummary

Data Mining of DTC Web sites enable:complete multimedia DTC tracking analytics

feedback to marketers in terms they understand

Process improvement framework is critical

Software vendor selection is only part of the effort

Discoveries have impact

Data Mining of DTC Web sites enable:complete multimedia DTC tracking analytics

feedback to marketers in terms they understand

Process improvement framework is critical

Software vendor selection is only part of the effort

Discoveries have impact