CHAPTER 15

Evaluating an Integrated Marketing Program

What should be evaluated?

Short-term outcomes (sales, redemption rates)Long-term results (brand awareness, brand loyalty)Product specific awareness (new product or

extension, company)Awareness of the overall companyAffective responses (liking the company, and

positive brand image)

Evaluation

Pretest measures Concurrent measuresPost test measures

Concept testingCopytestingEmotional reaction testsCognitive NeuroscienceOn-line evaluation techniquesBehavioural techniquesRecall testsRecognition tests

Message Evaluation Techniques

Difficulties in Evaluating Advertising

Influence of other factors on behaviourDelayed impact of advertisingConsumers change their mind in the storeWhether or not the brand is in evoked setGoal of ad may be to build brand equity, not

increase sales so what you intend to measure is important to clarify.

Advertising Tracking Research

Track ads after launch Nielsen IAG Millward Brown

Monitors Brand performance Advertising effectiveness

Specific time testContinuous tracking

Concept Testing

Aimed at content of a marketing piece.What is evaluated?

Verbal messages Message and meaning Translations for international copy Product placement in ads Value of a sales promotional efforts

Focus groups may be usedConcept testing instruments:

Comprehension and readability tests Reaction tests

Copytesting

Used when marketing piece is finished or in final stages

Portfolio test – print communicationTheater test – television/mass media

communicationMethods used:

Focus groups Can be mall intercept

Copytesting

Copytesting can be used to determine if viewers comprehend this ad and what their reaction to it is.

Copytesting

Criticisms of copytesting Some agencies do not use them Can stifle creativity Focus groups may not be a good judge

Support of copytesting Issue of accountability -

http://www.youtube.com/watch?v=6aXN8qaxPek Majority support copytesting because clients

want support for ad decision

Testing procedure should be relevant to objectives.Researchers should agree on how the results will be

used in advance.Multiple measures should be used.The test should be based on some model or theory of

human response to communication.Testing procedure should allow for more than one

exposure. In selecting alternate ads to include in the test, they

should be at the same stage in the process as the test adThe test should provide controls to avoid biases.Sample used for the test should be representative of the

target sample.Testing procedure should demonstrate reliability and

validity. Source: Based on PACT document published in the Journal of Marketing, (1982) ,Vol. 11, No. 4, pp. 4-29.

F I G U R E 1 5 . 2Copytesting principles of Positioning Advertising Copytesting (PACT)

What’s Happening?

http://www.tvb.ca/pages/BroccoliCase_htm

Emotional Reaction Tests

Used for material designed to solicit emotions.Difficult to measure emotions with questions.Warmth Meters are usually used as am instrument

Sample Graph from a Warmth Meter30-Second TV Advertisement

Sample Ad RatingWarmth Meter

Start 10 seconds 20 seconds 30 seconds

Target Audience

Total Audience

Ad section that elicited negative emotions

Cognitive Neuroscience

Measures brainwave activityBetter indicator of respondent’s true reactionDoes not rely on self-reporting, like emotional

reaction testsCompanies experimenting with neuroscience

EmSense, NeureoFocus, Sands Research Frito-Lay

Power of cognitive neuroscience Reveals physiological reactions Understand how information is being processed

Online Evaluation Metrics

Click-throughs primary measurementLength of engagementDwell timeRedemption ratesResponse ratesSalesWeb chatter

SalesRedemption ratesResponse rates to offersTest marketsPurchase Simulation tests

F I G U R E 1 5 . 4

Behavioural Measures

Changes in salesTelephone inquiries. Response cards.Internet inquiries.Direct marketing responses.Redemption rate of sales promotion offers.

Coupons, premiums, contests, sweepstakes

F I G U R E 1 5 . 6

Responses to Marketing Messages that can be Tracked

Test Markets

Used to assess: Advertisements (different versions in different

markets) Consumer and trade promotions Pricing tactics New products

Cost effective method of evaluation prior to large-scale launch.

Resembles actual situation.Design test market to model full marketing plan.Length of test market is a concern.Competitive actions must be considered

Bias in purchase intention questionsSimulated purchase testsResearch Systems CorporationDoes not rely on opinions and

attitudes

Purchase Simulation Tests

Recall Tests

Day-after recall (DAR)Unaided recallAided recallFactors affecting recall:Respondent’s age affects recall scoresMedium used also plays a factor

Product name or brandFirm nameCompany locationTheme musicSpokespersonTaglineIncentive being offeredProduct attributesPrimary selling point of communication piece

Recall Tests - Items tested

Sample Recall Test 30-Second TV Advertisement for Pet Food

0.0%

5.0%

10.0%

15.0%

20.0%

25.0%

Brand name Theme music Spokesperson Tagline Incentive ProductAttribute

Test Ad Competitor A Competitor B

Recall DecayMagazine Ad vs. Television Ad

Source: Magazines Canada’s Research Archive

100% 100%

86%

65%75%

43%

0%

20%

40%

60%

80%

100%

DAR Two days later Eight days later

Magazine Television