2003 National Conference on Tobacco or Health2003 National Conference on Tobacco or Health Page 1Page 1
Assessing the Relative Impact of a Media Ad Campaign
Michael Willmorth, Ph.D.
Clearwater Research, Inc.
Selina Carver
Idaho Department of Health
and Welfare
December 11, 2003
Background
Idaho Tobacco Prevention and Control Program (ITPCP)• Bureau of Health Promotion, IDHW• Prevent initiation of tobacco use, eliminate
exposure to ETS, promote quitting among youth and adults, and eliminate disparities in the population
es/drake (ESD) Clearwater Research, Inc.
Policy
R & E
Media
2003 National Conference on Tobacco or Health2003 National Conference on Tobacco or Health Page 2Page 2
Research History
ITPCP’s targeted surveillance and evaluation efforts• Baseline survey of teens and parents
in 2000• Anti-tobacco marketing survey of
teens and parents in 2001• Counter-marketing media campaign
evaluation surveys of teens in 2002 and 2003
2003 National Conference on Tobacco or Health2003 National Conference on Tobacco or Health Page 3Page 3
2002 Teen Media Ad Campaign
Targeted to teens (aged 12–17) Messages
– If you smoke, quit– Don’t start
Channels– Radio– Television– Billboards
Flights: September 2001–May 2002
2003 National Conference on Tobacco or Health2003 National Conference on Tobacco or Health Page 4Page 4
2001–2002 ITPCP Radio Ads
Facts A full chorus sings “Buy A Pack Today” as teen announcer presents facts about smoking.
5th Guy Interviewer talks with boy who dates girls who smoke.
Singing Pollution Secondhand smoke is compared to car emitting annoyingly loud music.
Just a Pinch Cowboy describing pleasures of smokeless tobacco progressively loses parts of his mouth.
Lucky Rick Smokeless tobacco user describes surgery to remove parts of his tongue and jaw because of mouth cancer.
Joe DeBoer Smokeless tobacco user describes frightening experience with leukoplakia.
Critique Student describes essay he wrote on “the last thing he read”: the Surgeon General’s Warning.
Look at Me Girl who smoked and wanted to get the attention of boys now gets unwanted attention because of hole in throat.
When You Smoke Series of teen voices describe other uses of chemicals found in cigarette smoke.
2003 National Conference on Tobacco or Health2003 National Conference on Tobacco or Health Page 5Page 5
2001–2002 ITPCP TV Ads
Grapes US ban on Chilean grape imports. 10 times as much cyanide in single puff.
Baseball English Throat cancer dashed hopes of aspiring major league umpire.
Otolaryngologist Man with throat cancer must visit doctor for life to check for recurrence.
Careful Tim Chemical spill worker is concerned that cigarette smoke contains benzene.
Surgery Surgery to remove mouth tumor becomes more radical than expected.
Safe Alternative Man recalls TV ads that claimed snuff was a save alternative to smoking.
Doesn’t Kill Teens show tobacco users who are still alive, but seriously harmed.
Emergency Room Man recalls devastating news that wife had brain lesions from lung cancer.
46 Man muses that wife who died from smoking was middle aged at 23.
Building Toluene, a chemical in cigarette smoke, is also used to make explosives.
Jar Formaldehyde, a chemical used in embalming, is also in secondhand smoke.
2003 National Conference on Tobacco or Health2003 National Conference on Tobacco or Health Page 6Page 6
2001–2002 ITPCP Billboards
I love you, Mom … hate your smoke
Black and white close-up photo of girl’s face
Chemotherapy scares me, Scout
Marlboro Man-type character speaking to horse
I really need to quit Marlboro Man-type character talking into phone (walkie-talkie)
Half off “Rick Bender, age 37. He didn’t lose his life to tobacco. He lost half his jaw.” (“Half Off with Tobacco”)
2003 National Conference on Tobacco or Health2003 National Conference on Tobacco or Health Page 7Page 7
2002 Study Objectives
Measure the effectiveness of ITPCP media campaign based on campaign objectives and exposure.
Provide research-based conclusions that can guide message development and delivery to Idaho teens.
Evaluate impact of current media campaign on teen behavior.
2003 National Conference on Tobacco or Health2003 National Conference on Tobacco or Health Page 8Page 8
2002 Study Design Overview
Data collection• Telephone interview with 630 Idaho teens (12–17)• Field period: April 24–June 13, 2002
Sample• Probability sample (list-assisted RDD)• Stratified proportionately by media market • ±3.9% C.I. At 95% C.L. for entire state• Response rate: 49.0% CASRO, 63.9% upper bound
2003 National Conference on Tobacco or Health2003 National Conference on Tobacco or Health Page 9Page 9
Research Questions
What impact has the campaign had on the population of teenage smokers' propensity to quit or reduce smoking?
Has the campaign sparked conversation for the teenage population in Idaho?
How effective were particular ads in influencing smoking behavior? How convincing was each one? How sellable was the idea in each one?
What ads received by Idaho’s teenage population have been the most effective (whether or not a part of the ESD/ITPCP media ad campaign)?
What messages are the most beneficial for effecting change in smoking behavior? What works and what doesn’t?
What audience is most receptive to the current media ad campaign?
2003 National Conference on Tobacco or Health Page 102003 National Conference on Tobacco or Health Page 10
Data Collection Methodology
Telephone interviews • CATI• 10 minute interviews (on average)
Instrument design: screening questions• Determine eligibility of household• Obtain permission from parent/guardian to interview
teen• Enumerate number of teens in household• Random selection of teen (with replacement as refusal
avoidance)
2003 National Conference on Tobacco or Health Page 112003 National Conference on Tobacco or Health Page 11
Questionnaire Topics
Demographics (including media use) Social activities Use of tobacco (self, family, friends) Quit attempts Tobacco-related beliefs (including ad-related
messages) Campaign and noncampaign ads (recall,
awareness, effectiveness) Conversations about tobacco/ads
2003 National Conference on Tobacco or Health Page 122003 National Conference on Tobacco or Health Page 12
Recall Frequency Hearing Ad
Over the past 12 months, how frequently have you heard a radio commercial or ad telling you about the risks of smoking and tobacco? Would you say …
– Not at all– Less than once a month– 1 to 3 times a month– 1 to 3 times a week– Every day or almost every day– More than once a day
2003 National Conference on Tobacco or Health Page 132003 National Conference on Tobacco or Health Page 13
Unaided Recall
Please describe one of the anti-tobacco radio ads you have heard over the past 12 months. (SELECT UP TO 3)
– Facts– 5th Guy– Singing Pollution– Just a Pinch– Lucky Rick– Joe DeBoer– Critique– Look at Me– When You Smoke– OTHER (SPECIFY)
2003 National Conference on Tobacco or Health Page 142003 National Conference on Tobacco or Health Page 14
Prompted Recall
Now, I will describe for you an ad that might or might not have been playing on the radio in your area. [READ DESCRIPTION] Over the past 12 months, how many times have you heard this ad? Would you say …
– Not at all– Once– 2 to 4 times– 5 to 10 times– More than 10 times
Followed up with 3 agree/disagree assessments– Ad was convincing– Ad exaggerated the problem– Ad made me more aware of the risks of smoking and tobacco
2003 National Conference on Tobacco or Health Page 152003 National Conference on Tobacco or Health Page 15
Coding Data
Interviewers coded open-ended responses into existing categories or recorded verbatim descriptions
New categories were created for “other” ads– Consultation with ESD– Internet searches
Analysts reviewed open-ended responses– Checked and corrected interviewer coding– Recoded “other” ads into new categories
New variables (“yes”/”no”)– Any ITPCP ad– Only ITPCP ads– No ITPCP ads
2003 National Conference on Tobacco or Health Page 162003 National Conference on Tobacco or Health Page 16
Most Recalled Radio Ads
1. When you Smoke (54) 11. “Anti-Drug” (9)
2. Bad health effects (39) 12. Health stats (7)
3. Lucky Rick (38) 13. 5th Guy (6)
4. “Infect Truth” (34) 14. “Tobacco is whacko” (5)
5. Likely TV ad (32) 15. Prenatal risks (5)
6. Other (29) 16. Singing Pollution (4)
7. Critique (12) 17. Lights (Infect Truth) (4)
8. Facts (10) 18. “Tobacco smokes you” (4)
9. Just a Pinch (9) 19. Cow farts (4)
10. Look at Me (9) 20. Teens: smoking is bad (4)
2003 National Conference on Tobacco or Health Page 172003 National Conference on Tobacco or Health Page 17
Most Recalled TV Ads
1. Ratman (Infect Truth) (101) 11. Smoking Thru Her Throat (19)
2. Squadron (Infect Truth) (54) 12. Doesn’t Kill (18)
3. Surgery (53) 13. Safe Alternative (17)
4. Other (53) 14. Chemicals in smoke (13)
5. “Infect Truth” (46) 15. Teen refuses cigarette (11)
6. Doorhanger (Infect Truth) (31) 16. Grounded (Anti-Drug) (10)
7. Body Bags (ALF) (28) 17. Careful Tim (9)
8. Dog Walker (Infect Truth) (24) 18. Ammoniade (Infect Truth) (9)
9. Baseball English (20) 19. Parents talk to kids (9)
10. Otolaryngologist (20) 20. Anti-Drug, sports (8)
2003 National Conference on Tobacco or Health Page 182003 National Conference on Tobacco or Health Page 18
Most Recalled Billboards
1. Chemotherapy Scares Me (85) 11. Likely TV ad (6)
2. Half Off (54) 12. Diseased organs and mouth (6)
3. Other (42) 13. Tobacco is whacko (6)
4. I Really Need to Quit (24) 14. Smoking Thru Her Throat (5)
5. I Love You, Mom (16) 15. Cigarette chemicals (5)
6. “Infect Truth” (16) 16. “Anti-Drug” (3)
7. I Miss My Lung, Bob (15) 17. Man with hole in throat (3)
8. Bob, I’ve Got Emphysema (10) 18. Pregnant woman (3)
9. Surgeon General’s Warning (8) 19. Casualty count (2)
10. Mind If I Smoke? (7) 20. Camel (2)
2003 National Conference on Tobacco or Health Page 192003 National Conference on Tobacco or Health Page 19
Calculating “Impact”
“Impact” = percentage of unaided mentions accounted for by ITPCP ads
Lower bound = % only ITPCP ads / % any antitobacco ad
Upper bound = % any ITPCP ad / % any antitobacco ad
“Ballpark” point estimate = average of lower and upper bound
2003 National Conference on Tobacco or Health Page 202003 National Conference on Tobacco or Health Page 20
Relative Impact of Campaign
% only
ITPCP
% any
ITPCP
% any
anti-Tob
Lower
Bound
Upper
Bound Ballpark
Radio 19.3 20.9 50.5 38.2 41.4 40
TV 12.4 19.1 80.9 15.3 23.7 20
Billboard 22.6 25.4 46.2 48.9 55.0 52
Combined 1.7 46.2 90.3 1.9 51.2 27
2003 National Conference on Tobacco or Health Page 212003 National Conference on Tobacco or Health Page 21
Recommendations
Use to quantify (in a “ballpark” way) how components of a media ad campaign fit into the larger system of antitobacco messages in which it operates
Limitations– Very restricted definition of “impact”– Relies completely on unaided recall, so
interpretation is dependent on time of measurement
2003 National Conference on Tobacco or Health Page 222003 National Conference on Tobacco or Health Page 22
Thanks!Michael Willmorth, Ph.D.Michael Willmorth, Ph.D.
Clearwater ResearchClearwater Research Senior Study DirectorSenior Study Director
2136 North Cole Road2136 North Cole RoadBoise, ID 83704Boise, ID 83704
(208) 376-3376 1-800-727-5016(208) 376-3376 1-800-727-5016
http://www.clearwater-research.comhttp://www.clearwater-research.com
2003 National Conference on Tobacco or Health Page 232003 National Conference on Tobacco or Health Page 23
Top Related