Pros and Cons of Measuring Advertising Effectiveness

Reasons to Measure Reasons to Measure Ad EffectivenessAd Effectiveness

Avoid costly mistakes Evaluate alternative

strategies Increase efficiency of

advertising in general Determining if objectives

are achieved

Reasons Not to MeasureReasons Not to MeasureEffectivenessEffectiveness

Cost of measurement Problems with research Disagreement about

what to test Objections of creative

personnel Lack of time

Evaluating Creatives*

Concept Testing: Assessment of potential creatives

Communication Research: A Look at the Advertising Creative

Copy Testing: Assessment of creative achievement

* The word creative when used as a noun refers to an actual advertisement

Concept Testing A concept is similar to the actual ad, but more

succinct without a lot of selling and posturing The concept states in clear terms the selling

premise, unique position or benefit of the advertised product or service

We test the concepts in order to determine which of a set of concepts would be the most useful in accomplishing our objective

See handouts to see the difference between a concept and the corresponding creative

Types of Concept Tests New Product Concept Tests

Generate concepts for multiple new product possibilities and test the concepts to assess which new product has the most potential for success

Benefit Positioning Concept Tests Generate multiple concepts for one product to test

which concept has the most potential for generating desired outcome, generally purchase intent

Each concept will focus on a different benefit (position)

Example: concept test for a new PDA concept 1 focuses on ease of use concept 2 focuses on technical capability

General Outline for Performing Concept Testing

Generate Concepts Generate Questionnaire Administer the Test Organize, analyze, interpret data

Concept Generation

Concepts are typically presented on 8.5”X11” cardstock or board, and are sometimes called “white card” tests

Card contains: Headline Body copy Illustrations

White Card Headline This is NOT the same as an advertising headline A succinct summary of the main selling proposition or

product benefit Leads into the body copy Gives consumer a clear understanding of the ideas

that are to be presented in the body copy Use simple, single-minded declarative statements

“The new GeoNET PDA integrates many wireless technologies so you can do more when you travel.”

Concept Body Copy Be direct and clear, using short paragraphs

and active, declarative statements Do not use:

Advertisingese; do not embellish Industry jargon that is unfamiliar to the audience Overwritten or long concepts

If new product concept, you should also Identify product category membership List all primary benefits Identify differentiating attributes

Concept Body CopyThe new GeoNET PDA integrates a wireless phone, GPS

receiver, and Blackberry wireless technology with 512MB of internal memory and coast to coast Web access.

With all of these features in one convenient device, working while traveling is easier than carrying multiple devices to accomplish the same tasks.

The GeoNET PDA is available with either Palm OS or Windows Mobile for even more flexibility

Suggested retail prices: $299 with black and white display; $449 with 16-bit color display

Concept Generation Considerations Using Brand names is generally accepted

as concept perceptions are often impacted greatly by brand name presence

Pricing information should be present If not in the concept itself, integrate it into the

questionnaire Price does not need to be explicitly stated, but if

it is, it should accurately reflect the actual expected price

Illustrations and format between tested concepts should be as identical as possible

Generate Questionnaire Demographic and brand usage questions Diagnostic questions pertaining to specific

characteristics of the concept Core questions

Communication of the main idea Believability of the main idea Uniqueness of the main idea Personal relevance of the main idea Purchase Intent Purchase frequency Reasons for purchase intent and purchase

frequency

Concept Test Administration

Target definition Research methodology Questionnaire construction Concept exposure Data analysis and interpretation

Data Organization, Analysis and Interpretation Split sample by respondent characteristics

(quota groups) and assess differences between concepts

Split sample by concepts and assess differences between quota groups

Crosstabs on core questions by purchase intent, brand usage, and communications

Correlation matrix between believability, uniqueness, and relevance

Group Practice Time

Concept Testing Handout Read through the description of the

study and purpose of the research The actual concept and a variety of

results tables are provided for you Answer the 4 questions at the end of

the handout using the results provided

Communication Research After concept testing, we have identified

the concept, that is, the main idea that is to be communicated in the advertisements

Now we must develop the actual communications, the ads, and determine if we are communicating our ideas effectively

Basic goal is to determine how consumers react to the ad and why they are reacting that way

General Outline for Communication Research

Create test ads Specify Respondent Characteristics Select Methodology Generate Questionnaire Analyze and Report findings

Creating the Test Ads Print ads should be created to the semi-comp stage,

meaning it is full size, with color and art in place and scaled to size May not be exactly how ad will look in print, but

should be close Radio ads can be scratch tracks, meaning that all

aural elements are there, but recording may not be professional quality yet

TV ads may be storyboarded as long as the ad comes across to the audience in the correct way May use animatics, photoboards, or photomatics if

needed to get across important visual cues In all cases, copy should be precise

Generate the Questionnaire Brand usage, demography, etc. Core questions to assess:

Whether the respondent accurately processed the information presented

Respondent attitudes toward the communication, i.e. the message was well received Should probe cognitive and emotional

responses Respondent attitudes toward the ad itself

Questions specific to the particular creative

Analysis and Interpretation of Communication Research Data

Typically standard tests (t-tests, chi-square tests) are all that are needed to describe the results

Relationships between variables can be interesting, but is more difficult and requires some creativity on the part of the researcher since no hypotheses are developed for testing

Copy Testing Communication research is performed during creative

development and is diagnostic in nature Copy testing takes place after creative developments

using actual ads, and is evaluative in nature Independent marketing research companies specialize

in copy testing, and have established a set of norms upon which go/stay decisions are made

The majority of copy testing is done on television advertising, although options exist for print, radio, and outdoor advertising as well Television advertising is far more expensive to create

and run than other media, hence the abundance of tv copy testing

Four key decisions to make Place: in home viewing or out of home Naturalness: forced versus unforced

exposure Number of exposures before data collection Key measures:

Recall Message communication Attitude change Purchase intention Ad attitude Et cetera

Typical structure of Television Copy Test Show a sample of 200 respondents a 30 minute program with test

ad imbedded Program is broadcast on an unused cable channel so respondents

may view the program in their homes The day after viewing, respondents are contacted and asked to

name the commercials they remember seeing during the program If unable to name the commercials, they are prompted with correct and

incorrect answers This tests both aided and unaided recall

Purchase intent can be ascertained with a pre-post exposure measurement Prior to watching, respondents are asked what brand they are likely to

purchase next time After exposure, respondents are asked what brand they would prefer to

win Asking the same preference question in a different context masks the

purpose of the question to ensure accurate responses

Other Key Measures Intrusiveness: measure of the ads ability to break through

the clutter and earn a place in the consumer’s mind Unaided recall – correct brand Proved unaided recall Proved brand aided recall

Communication: identifies the ideas that consumers got out of the ad The Lead Idea is the idea that is most often cited by

respondents Persuasion: ad’s ability to generate favorable brand

attitudes Purchase intent Standard brand attitude measure

Commercial Reaction Standard ad attitude measures

Typical Structure of a Magazine Copy Test 125 respondents are recruited through mall intercepts They are asked questions about media and product

usage They are provided a fake magazine with test ad

imbedded and asked to quickly flip through and give initial reactions

Ask respondents what ads they remember seeing Have them look through magazine again, for a longer

time this time Now ask recall, persuasion and communication

questions Direct respondents to the test ad and ask them to

read it Ask customized communication and diagnostic

questions

Typical Measures in Print Copy Testing

Recall/Recognition Idea Communication Brand Attitude Ad Attitude Purchase Intent Stopping Power

Noted Readers: only remember seeing it Associated Readers: read part of it Read Most Readers: read more than half of it

Copy Testing Notes Methodology should be as close to

“real world” as possible Using multiple exposures and

collecting different data after each exposure allows identification of long term effects

Responses are more valid when exposure occurs without respondent knowledge of participation

When Using Outside Firms Research Services Some measures between companies may

have similar names, but measure completely differently. Understand the method completely so you can truly ascertain the meaning of the data

Know what the “norms” are, and be sure that they are current and appropriate to the product

Ensure that measures used by research firm are reliable and valid; if firm does not share, or claims to not know, reliability and validity diagnostics, be wary of results