2003 National Conference on Tobacco or Health2003 National Conference on Tobacco or Health Page 1Page 1

Assessing the Relative Impact of a Media Ad Campaign

Michael Willmorth, Ph.D.

Clearwater Research, Inc.

Selina Carver

Idaho Department of Health

and Welfare

December 11, 2003

Background

Idaho Tobacco Prevention and Control Program (ITPCP)• Bureau of Health Promotion, IDHW• Prevent initiation of tobacco use, eliminate

exposure to ETS, promote quitting among youth and adults, and eliminate disparities in the population

es/drake (ESD) Clearwater Research, Inc.

Policy

R & E

Media

2003 National Conference on Tobacco or Health2003 National Conference on Tobacco or Health Page 2Page 2

Research History

ITPCP’s targeted surveillance and evaluation efforts• Baseline survey of teens and parents

in 2000• Anti-tobacco marketing survey of

teens and parents in 2001• Counter-marketing media campaign

evaluation surveys of teens in 2002 and 2003

2003 National Conference on Tobacco or Health2003 National Conference on Tobacco or Health Page 3Page 3

2002 Teen Media Ad Campaign

Targeted to teens (aged 12–17) Messages

– If you smoke, quit– Don’t start

Channels– Radio– Television– Billboards

Flights: September 2001–May 2002

2003 National Conference on Tobacco or Health2003 National Conference on Tobacco or Health Page 4Page 4

2001–2002 ITPCP Radio Ads

Facts A full chorus sings “Buy A Pack Today” as teen announcer presents facts about smoking.

5th Guy Interviewer talks with boy who dates girls who smoke.

Singing Pollution Secondhand smoke is compared to car emitting annoyingly loud music.

Just a Pinch Cowboy describing pleasures of smokeless tobacco progressively loses parts of his mouth.

Lucky Rick Smokeless tobacco user describes surgery to remove parts of his tongue and jaw because of mouth cancer.

Joe DeBoer Smokeless tobacco user describes frightening experience with leukoplakia.

Critique Student describes essay he wrote on “the last thing he read”: the Surgeon General’s Warning.

Look at Me Girl who smoked and wanted to get the attention of boys now gets unwanted attention because of hole in throat.

When You Smoke Series of teen voices describe other uses of chemicals found in cigarette smoke.

2003 National Conference on Tobacco or Health2003 National Conference on Tobacco or Health Page 5Page 5

2001–2002 ITPCP TV Ads

Grapes US ban on Chilean grape imports. 10 times as much cyanide in single puff.

Baseball English Throat cancer dashed hopes of aspiring major league umpire.

Otolaryngologist Man with throat cancer must visit doctor for life to check for recurrence.

Careful Tim Chemical spill worker is concerned that cigarette smoke contains benzene.

Surgery Surgery to remove mouth tumor becomes more radical than expected.

Safe Alternative Man recalls TV ads that claimed snuff was a save alternative to smoking.

Doesn’t Kill Teens show tobacco users who are still alive, but seriously harmed.

Emergency Room Man recalls devastating news that wife had brain lesions from lung cancer.

46 Man muses that wife who died from smoking was middle aged at 23.

Building Toluene, a chemical in cigarette smoke, is also used to make explosives.

Jar Formaldehyde, a chemical used in embalming, is also in secondhand smoke.

2003 National Conference on Tobacco or Health2003 National Conference on Tobacco or Health Page 6Page 6

2001–2002 ITPCP Billboards

I love you, Mom … hate your smoke

Black and white close-up photo of girl’s face

Chemotherapy scares me, Scout

Marlboro Man-type character speaking to horse

I really need to quit Marlboro Man-type character talking into phone (walkie-talkie)

Half off “Rick Bender, age 37. He didn’t lose his life to tobacco. He lost half his jaw.” (“Half Off with Tobacco”)

2003 National Conference on Tobacco or Health2003 National Conference on Tobacco or Health Page 7Page 7

2002 Study Objectives

Measure the effectiveness of ITPCP media campaign based on campaign objectives and exposure.

Provide research-based conclusions that can guide message development and delivery to Idaho teens.

Evaluate impact of current media campaign on teen behavior.

2003 National Conference on Tobacco or Health2003 National Conference on Tobacco or Health Page 8Page 8

2002 Study Design Overview

Data collection• Telephone interview with 630 Idaho teens (12–17)• Field period: April 24–June 13, 2002

Sample• Probability sample (list-assisted RDD)• Stratified proportionately by media market • ±3.9% C.I. At 95% C.L. for entire state• Response rate: 49.0% CASRO, 63.9% upper bound

2003 National Conference on Tobacco or Health2003 National Conference on Tobacco or Health Page 9Page 9

Research Questions

What impact has the campaign had on the population of teenage smokers' propensity to quit or reduce smoking?

Has the campaign sparked conversation for the teenage population in Idaho?

How effective were particular ads in influencing smoking behavior? How convincing was each one? How sellable was the idea in each one?

What ads received by Idaho’s teenage population have been the most effective (whether or not a part of the ESD/ITPCP media ad campaign)?

What messages are the most beneficial for effecting change in smoking behavior? What works and what doesn’t?

What audience is most receptive to the current media ad campaign?

2003 National Conference on Tobacco or Health Page 102003 National Conference on Tobacco or Health Page 10

Data Collection Methodology

Telephone interviews • CATI• 10 minute interviews (on average)

Instrument design: screening questions• Determine eligibility of household• Obtain permission from parent/guardian to interview

teen• Enumerate number of teens in household• Random selection of teen (with replacement as refusal

avoidance)

2003 National Conference on Tobacco or Health Page 112003 National Conference on Tobacco or Health Page 11

Questionnaire Topics

Demographics (including media use) Social activities Use of tobacco (self, family, friends) Quit attempts Tobacco-related beliefs (including ad-related

messages) Campaign and noncampaign ads (recall,

awareness, effectiveness) Conversations about tobacco/ads

2003 National Conference on Tobacco or Health Page 122003 National Conference on Tobacco or Health Page 12

Recall Frequency Hearing Ad

Over the past 12 months, how frequently have you heard a radio commercial or ad telling you about the risks of smoking and tobacco? Would you say …

– Not at all– Less than once a month– 1 to 3 times a month– 1 to 3 times a week– Every day or almost every day– More than once a day

2003 National Conference on Tobacco or Health Page 132003 National Conference on Tobacco or Health Page 13

Unaided Recall

Please describe one of the anti-tobacco radio ads you have heard over the past 12 months. (SELECT UP TO 3)

– Facts– 5th Guy– Singing Pollution– Just a Pinch– Lucky Rick– Joe DeBoer– Critique– Look at Me– When You Smoke– OTHER (SPECIFY)

2003 National Conference on Tobacco or Health Page 142003 National Conference on Tobacco or Health Page 14

Prompted Recall

Now, I will describe for you an ad that might or might not have been playing on the radio in your area. [READ DESCRIPTION] Over the past 12 months, how many times have you heard this ad? Would you say …

– Not at all– Once– 2 to 4 times– 5 to 10 times– More than 10 times

Followed up with 3 agree/disagree assessments– Ad was convincing– Ad exaggerated the problem– Ad made me more aware of the risks of smoking and tobacco

2003 National Conference on Tobacco or Health Page 152003 National Conference on Tobacco or Health Page 15

Coding Data

Interviewers coded open-ended responses into existing categories or recorded verbatim descriptions

New categories were created for “other” ads– Consultation with ESD– Internet searches

Analysts reviewed open-ended responses– Checked and corrected interviewer coding– Recoded “other” ads into new categories

New variables (“yes”/”no”)– Any ITPCP ad– Only ITPCP ads– No ITPCP ads

2003 National Conference on Tobacco or Health Page 162003 National Conference on Tobacco or Health Page 16

Most Recalled Radio Ads

1. When you Smoke (54) 11. “Anti-Drug” (9)

2. Bad health effects (39) 12. Health stats (7)

3. Lucky Rick (38) 13. 5th Guy (6)

4. “Infect Truth” (34) 14. “Tobacco is whacko” (5)

5. Likely TV ad (32) 15. Prenatal risks (5)

6. Other (29) 16. Singing Pollution (4)

7. Critique (12) 17. Lights (Infect Truth) (4)

8. Facts (10) 18. “Tobacco smokes you” (4)

9. Just a Pinch (9) 19. Cow farts (4)

10. Look at Me (9) 20. Teens: smoking is bad (4)

2003 National Conference on Tobacco or Health Page 172003 National Conference on Tobacco or Health Page 17

Most Recalled TV Ads

1. Ratman (Infect Truth) (101) 11. Smoking Thru Her Throat (19)

2. Squadron (Infect Truth) (54) 12. Doesn’t Kill (18)

3. Surgery (53) 13. Safe Alternative (17)

4. Other (53) 14. Chemicals in smoke (13)

5. “Infect Truth” (46) 15. Teen refuses cigarette (11)

6. Doorhanger (Infect Truth) (31) 16. Grounded (Anti-Drug) (10)

7. Body Bags (ALF) (28) 17. Careful Tim (9)

8. Dog Walker (Infect Truth) (24) 18. Ammoniade (Infect Truth) (9)

9. Baseball English (20) 19. Parents talk to kids (9)

10. Otolaryngologist (20) 20. Anti-Drug, sports (8)

2003 National Conference on Tobacco or Health Page 182003 National Conference on Tobacco or Health Page 18

Most Recalled Billboards

1. Chemotherapy Scares Me (85) 11. Likely TV ad (6)

2. Half Off (54) 12. Diseased organs and mouth (6)

3. Other (42) 13. Tobacco is whacko (6)

4. I Really Need to Quit (24) 14. Smoking Thru Her Throat (5)

5. I Love You, Mom (16) 15. Cigarette chemicals (5)

6. “Infect Truth” (16) 16. “Anti-Drug” (3)

7. I Miss My Lung, Bob (15) 17. Man with hole in throat (3)

8. Bob, I’ve Got Emphysema (10) 18. Pregnant woman (3)

9. Surgeon General’s Warning (8) 19. Casualty count (2)

10. Mind If I Smoke? (7) 20. Camel (2)

2003 National Conference on Tobacco or Health Page 192003 National Conference on Tobacco or Health Page 19

Calculating “Impact”

“Impact” = percentage of unaided mentions accounted for by ITPCP ads

Lower bound = % only ITPCP ads / % any antitobacco ad

Upper bound = % any ITPCP ad / % any antitobacco ad

“Ballpark” point estimate = average of lower and upper bound

2003 National Conference on Tobacco or Health Page 202003 National Conference on Tobacco or Health Page 20

Relative Impact of Campaign

% only

ITPCP

% any

ITPCP

% any

anti-Tob

Lower

Bound

Upper

Bound Ballpark

Radio 19.3 20.9 50.5 38.2 41.4 40

TV 12.4 19.1 80.9 15.3 23.7 20

Billboard 22.6 25.4 46.2 48.9 55.0 52

Combined 1.7 46.2 90.3 1.9 51.2 27

2003 National Conference on Tobacco or Health Page 212003 National Conference on Tobacco or Health Page 21

Recommendations

Use to quantify (in a “ballpark” way) how components of a media ad campaign fit into the larger system of antitobacco messages in which it operates

Limitations– Very restricted definition of “impact”– Relies completely on unaided recall, so

interpretation is dependent on time of measurement

2003 National Conference on Tobacco or Health Page 222003 National Conference on Tobacco or Health Page 22

Thanks!Michael Willmorth, Ph.D.Michael Willmorth, Ph.D.

Clearwater ResearchClearwater Research Senior Study DirectorSenior Study Director

2136 North Cole Road2136 North Cole RoadBoise, ID 83704Boise, ID 83704

(208) 376-3376 1-800-727-5016(208) 376-3376 1-800-727-5016

http://www.clearwater-research.comhttp://www.clearwater-research.com

2003 National Conference on Tobacco or Health Page 232003 National Conference on Tobacco or Health Page 23