Internet Measurement Progress Internet Measurement Progress and the Futureand the FutureAn MRC OverviewAn MRC Overview

Presented to the CNA11/26/08

MRC Background – BriefMRC Background – Brief

Tracking the Development of GuidelinesTracking the Development of Guidelines

Newspaper SpecificsNewspaper Specifics

Other MattersOther Matters

11

AgendaAgenda

2

Auditing UpdateAuditing Update

• Understanding Sources of Measurement Differences

MRC BackgroundMRC BackgroundPainlessly brief…Painlessly brief…

3

MRC Mission StatementMRC Mission Statement

• To secure for the media industry and related users audience measurement services that are valid, reliable and effective;

• Set Standards; and

• Conduct audits to verify compliance with Standards.

4

History and Mission of the MRCHistory and Mission of the MRC

• Congressional hearings held in 1963-64• System of self-regulation established• Founded as the “Broadcast Rating Council”• Not-for-Profit entity• Voluntary process for syndicated

measurement services to:– Supply complete information to the MRC– Comply with MRC Minimum Standards– Conduct the service as represented to clients– Submit to annual audits– Pay for the audit costs (internal & external)

• Confidential process

5

Industry ParticipationIndustry Participation

• 117 Member Organizations• Media Organizations

– Broadcasters – radio and television– Cable MSOs and networks– Internet organizations– Out-of-home media– Print organizations

• Advertising Agencies • Advertisers• Trade Associations

– Voting – CAB, IAB, MMA, MPA, NAA, NAB, RAB, TVB

– Liaison (non-voting) – AAAA, ANA, ARF

6

MRC’s Internet OrientationMRC’s Internet Orientation

• Setting a Strong Foundation– Guidelines, Transparency, Education

• Auditing, validation and accreditation of each method– “Raise all boats” approach– No measurement orientation favored– Goal – enhance trust and understanding of users

• Research required in several areas– Standards application– Measurement accuracy

• MRC Internet Committee – key assessment role– Fact-based, insofar as possible– Based on CPA independent audit– Neutral but aggressive in seeking best

practices– Seeking quality and full audience

representation7

Auditing UpdateAuditing UpdateCensus and Panel…Census and Panel…

Accuracy; Consistency; Transparency

Panel Measurement StandardsPanel Measurement Standards

• Approach• Audit Status• Progress and Learning

Note: Quantcast recently requested an audit for certain census measurement methods, which we hope will be expanded to its full methodology.

9

Overall ApproachOverall Approach

• Research Assessment (Audit Committee and Measurement Service)– Focus on methodological

validity– Non-probability sample

implications– Panel maintenance– Panel turnover– Panel coverage and

accuracy• Work vs. Home, etc.

– Persons identification– Fatigue– Comparison and

Reconciliation to Validated Census-Based Measurements

• Auditing(CPA Procedures)– Explained in following

slides:• Enumeration survey• Calibration panel• Large panel• Meter testing• Back-office processing

and reporting

10

Panel Audit StatusPanel Audit Status

• comScore MM– Pre-audit complete

• Confidential

– Audit committee formed– Audit scope and research

plans defined• Presented to audit

committee

– Research Plan work in process

– Meter Evaluation – Complete

– Enumeration Process Audit – Nearing Completion

– Large Panel Work -- Initiated

• Nielsen//NetRatings– Pre-audit complete

• Shared with Internet Committee

– Audit committee formed– Audit scope and research

plans defined• Presented to audit

committee

– Much of 1st phase of research plan executed

• Presented to audit committee; follow-ups needed and planned

– Meter Evaluation – Complete

– Data Collection and Editing – Complete

– Panel Auditing – Initiated11

Progress and Learning Progress and Learning

• Both Vendors Now Fully Cooperative– Please continue to encourage them!

• We Believe Large Panel Standard Errors Can Be Estimated– Jack Knife Replication (done at NNR)

– Weighting Controls and Sample Sizes need attention

• Turnover Plan A Likely Necessity– Turnover Analysis (some work done at NNR)

• Reported Levels and Coverage– Study approach agreed upon, work in-process

12

Census (Ad-Serving) Audit Status Census (Ad-Serving) Audit Status – MRC Audits– MRC Audits• Accredited

– AOL– Atlas DMT– CNET Networks– DoubleClick (DFP, DFA)– Edmunds.com– Eyeblaster– Forbes– Google (Impressions)– MSN– Turner– Univision.com– WDIG– Weather.com– Yahoo!– 24/7 Real Media

• In-Process– Broadband Enterprises– DoubleClick (DE)– USA Today– WPNI

13

Understanding Sources of Understanding Sources of DifferencesDifferences

•Geography

•User Types

•Demos

UniverseUniverse

•Naming

•Aggregation

ContentContent

•Usage Types

•Applications

•Devices

BehaviorBehavior

Sources of Counting Differences: Sources of Counting Differences: Definitional IssuesDefinitional Issues

15

• Users vs. Computers• Cookie Deletion

• Latency and Ad Delivery• Auto-Refresh

• Internal Traffic• In-Session Gaps

• New Types of Usage• International Traffic

Site-CentricSite-Centric

• Panel Representation• Meter Coverage

• Capturing All Access Locations• Metering All Computers

• User Data Collection• User In-Session Identification

• Compliance• Sampling Error

User-CentricUser-Centric

• Operational Robustness• Sound Internal Controls

• Handling Ambiguous Data• Disclosures

CommonCommon

Sources of Counting Differences: Sources of Counting Differences: Potential Quality IssuesPotential Quality Issues

16

Tracking the Development of GuidelinesTracking the Development of GuidelinesKey Points…Key Points…

Accuracy; Consistency; Transparency

IAB Measurement GuidelinesIAB Measurement Guidelines

18

Impression GuidelinesImpression Guidelines

Click GuidelinesClick Guidelines

Audience GuidelinesUniques, Time Spent

Audience GuidelinesUniques, Time Spent

2004 -

2007

The IAB’s Measurement Guidelines program began in 2004 with its first impression guidelines release.

Although new technologies will always create a need for additional measurement guidelines, the IAB’s overall plan continues in motion with Click Guidelines and Audience Guidelines being finalized.

Next On the Horizon…

• Internet– Partner: IAB

• Process Addendum (2008 release)– Important for Agencies and ASP-Based Publishers

• Clicks (being finalized)• Audience Reach Measurement (being

finalized)• Internet Enabled Gaming (in-process)

• Mobile– Partners: MMA, IAB, GSMA, ABCe

Basic PrinciplesBasic Principles

• Client-initiated Counting– Rejection of server-side counting– Adoption of beacon (and 302, etc.) at client side (as

close as possible to final delivery)

• Robot and Spider Filtration– Creation of IAB/ABCe international spiders & robots

list– Behavioral filtration guidelines were specified

• Caching– HTTP header controls and random number

assignments

• Internal Traffic– IT and .com employees are focus– Segregation, if material

• Disclosures– Goal is transparency– Description of measurement methodology– Reporting tool

20

Other Impression AdditionsOther Impression Additions

• Broadband– Post cache, revenue event counting– Encourages development of additional

measurement metrics

• Rich Media– Standardize 3rd and 4th party counting –

critical area

• Rich Internet Applications– First count per existing impression

guidance– 2nd count linked with strong user activity

• If absent, default to auto-refresh counting

– Disclose “in focus” where possible

21

Click Measurement GuidelinesClick Measurement Guidelines

Key Components:• Click Measurement Definitions• Impression-Click-Referral Cycle

– Initiated click (HTTP get)– Re-directed (measured) click (HTTP re-direct)– Received click (HTTP get)– Resolved click (HTTP 200 OK)

• Filtration Procedures– Control analyses for invalid clicks– Filtration of general activity

• Auditing Guidelines• General Reporting Parameters• Description of Methodology• Click Process Schematic

22

Audience Reach Measurement Audience Reach Measurement GuidelinesGuidelines

Focus – “Uniques”• Unique Hardware

– Cookies

– Cookied Browsers

– Devices

• Unique People– Proper terms defined and will be required– Linkage to People-Sourced data required

• Time Spent and Sessions– Ancillary needs in determination of Uniques,

so defined in guidelines

23

Newspaper SpecificsNewspaper Specifics

Suggested ActionsSuggested Actions

Stay Informed:• IAB

– Emerging Guidelines– All Available at IAB U.S. Website

• MRC– Many International Organizations

Participate in MRC

Legitimize and Validate Your Own Measurements:– Evaluation and Adopt Guidelines– Seek to Reduce Counting Differences and

Validate Accuracy25

Suggested ActionsSuggested Actions

Apples to Apples• Maximize “Combined Audience” focus in

your marketing:– Seek to understand unique traffic to each

medium and duplicated users• Internal and External Research• Content Consumed

– Leverage total audience (unique and duplicated traffic – with separate customized approaches) fairly while considering ad content

• Think of content consistency as a strategy in the appropriate content locations – Frequency2 (daily and medium)

• Maximize effectiveness of ad-type linkages• Consider Emerging Trends – e.g., Strong

Video Focus in Interactive, Ajax, etc.

26

Suggested ActionsSuggested Actions

Other Items• Use appropriate preambles in your

research to allow respondents to distinguish hard-copy from Internet readership.– Internal– Ensure external audience measurement

organizations do so also

• Account for “Pushed” Internet content properly– Count upon access

27

Other ItemsOther ItemsFor Your Consideration…For Your Consideration…

Other MRC ActivitiesOther MRC Activities

• Reconciliation Project– Ad Serving to Publisher– Census to Sample

• New Initiatives and Testing– Internet/Print Confusion– Nielsen Internet/TV Integration – Live Test– Arbitron PPM

• Standard Errors and Variability– Promote Further Understanding– Push for Better Tools– Internet?

• Seeking to Establish Industry Dialogue– On-Line Ad-Effectiveness Studies – Ground Rules

• AAAA, IAB, etc.

29

Other ResourcesOther Resources

• MRC Website (www.mediaratingcouncil.org):

– Accreditation Status• What Products are Accredited/Audited• Products In-Process

• Status-Update – NOL, CS, Quantcast– Educational MRC Staff Papers

• POV – Non-Probability Sampling Validation

• POV – Internet Measurement Alternatives

– Standards-Related Materials• MRC Minimum Standards

• Data Integration Guidelines

• Contact MRC if You Have Questions…Or if you’d like to get involved.

30

George W. IvieGeorge W. Iviegivie@mediaratingcouncil.org

David GunzerathDavid Gunzerathdgunzerath@mediaratingcouncil.org

Thank You!Thank You!