Post on 24-Jul-2015
22
Summary
What you’ll learn in this guideWe’ll show you how to use surveys to test marketing claims by taking the guesswork out of your big decisions.
Our guide includes:
• Tips for getting key metrics
• Best practices on writing claims testing surveys
• A real-world example of how our claims testing framework can be applied to your brand
Apple Watch™ marketing claim key findings • We compared two of Apple’s® marketing claims on its Apple Watch™
against each other:
o We found that the claim with a style and personal preference focus scored higher in clarity, a key metric.
• Identifying what resonates with a target market allows for more targeted advertising and potentially higher likeability and purchase intent:
o Taking a further look, we were able to segment the market into two distinct groups—early adopters and those who wait to purchase new products.
• The differences for early adopters, Apple’s® target market, and later stage adopters, were significant:
o Early adopters found the claim with a style and personal preference focus more convincing, unique, and memorable.
o They also listed text messaging as the most important for a smart-watch to have, significantly higher than any apps their later adopter counterparts listed.
Created by and for the use of SurveyMonkey Audience. This study was not commissioned or endorsed by Apple® or the other companies measured in the study. Apple and Apple Watch are trademarks of Apple Inc.
33
Table of Contents
Marketing Claims Testing Overview5
Market Segmentation16
Taking It to the Next Level19
The Smartwatch Category9
12Apple Watch™ Study: Marketing Claims Test
44
Methodology
What?SurveyMonkey Audience fielded a Marketing Claim Test study on two Apple® marketing claims to demonstrate how to successfully test marketing claims.
How?The survey was fielded through the SurveyMonkey panel. Respondents represented the general U.S. population. A total sample size of 417 was used for this study.
When?We fielded the survey from 5/1/15–5/4/15.
66
5 benefits of testing claims with surveys
Test various claims quickly and easily
A survey allows you to present a wide range of ideas to your target market, and get quantitative feedback fast.
Filter out the ineffective claims
Let your target market decide what works and what falls flat by asking them directly to investigate what customers want.
Refine your good ideas
If you test a bunch of concepts through a survey, you can quickly see which ones are rising to the top.
Test with diverse audiences
It’s much easier to use surveys than expensive focus groups to reach different demographic groups in your target market and segment their responses.
Save money
After you’ve launched your first concept testing survey, take a look at the results and see if any ideas tanked. Now imagine having spent months and thousands of dollars developing those ideas.
77
How to determine marketing claim successThe quickest way to determine a winning claim is to use a standard scoring model. This is an example scorecard that incorporates several key success metrics depending on the type of claim you’re testing and how you define a successful claim.
A B
Attributes Tested
Purchase Intent Statistical Significance
% %
Appeal Statistical Significance
% %
Uniqueness Statistical Significance
% %
Relevance Statistical Significance
% %
Average Score % %
*Z-test scores at 95% confidence interval
Created by SurveyMonkey Audience
Top 2 Box: The percentage of respondents who answered the two most positive answer options
Step 1: Define your key success metrics.
Step 5: Determine which claims performed better by using stat testing, a calculation that shows if two numbers or scores have a significant, or big enough, difference. This shows if one concept truly resonates with consumers more than the others.
Step 3: Use the top 2 answer choices for each question to more easily compare how each claim performed.
Step 2: Set the number of metrics depending on the claim you’re testing and how you define success.
Step 4: Compare the average scores of each claim to determine which one performed best based on the success metrics you chose.
88
Key success metrics
Here are several key metrics you can incorporate into a claim test:
Whether your claim motivates respondents to buy
If your claim is perceived to be for a high quality brand/product
The overall appeal of your claim is a strong indicator of purchase intent
How clearly your claim messaging is understood by respondents
A gauge of how well your target audience remembers your claim
How different one claim is from the others
How trustworthy your claim is perceived to be
A way to determine if your claim resonates with respondents
Purchase Intent:
Perception of Quality:
Appeal:
Message Clarity:
Recall: Uniqueness:
Believability:Relevance:
10
Apple’s® entering a new market...
Do you own any of the following devices?
0%
50%
An activity/fitness tracker (e.g. Fitbit, Garmin, etc.)
A smartwatch (Apple Watch, pebble, etc.)
No, I don’t own either of these devices
20%
23%
3%
75%80% ...but they’ve got high category recognition
100% of respondents knew what a smartwatch is:
“A smartwatch is a computerized wristwatch with touchscreen
display and functionality that is enhanced beyond timekeeping.
Many smartwatches run mobile apps, while a smaller number of
models run a mobile operating system.”*
10
*Definition provided to respondents.
1111
And Apple® has the highest brand recognition
Based on your current knowledge, which of the following companies make smartwatches?
0%
80%
90%
Apple® Other
10%
20%
30%
40%
50%
60%
70%
23%
93%
4%
100%
Android™
18%
LG®
13%
SAMSUNG®
40%
Sony®
13%
Google®
18%
Motorola®
11%
Pebble®
11%
1313
Apple® claims test summary
Background:Apple® released its Apple Watch™ for pre-orders on April 10, 2015. We used SurveyMonkey Audience to test the effectiveness of two of the marketing claims found on the Apple® website.
Goal:Put two in-field claims head to head to see which one performs better.
Key insights: • We found that the claim with a style
and personal preference focus scored higher in clarity, a key metric.
• We then segmented the market into two distinct groups to see which claim resonated best with the target market.
1414
The 2 claims
Claim 1: Person and technology relationship focus
“Our most personal device yet. Apple Watch represents a new chapter in the relationship people have with technology. It’s our most personal product yet, because it’s the first one designed to be worn.”
Claim 2: Style and personal preference focus
“There’s an Apple Watch for everyone. A device you wear is vastly different from one you keep on a desk or carry in your pocket. It’s more than a tool. It’s a true expression of your personal taste. So we designed Apple Watch to reflect a wide range of styles and preferences. Because we want you to love wearing it as much as you love using it.”
vs
1515
Apple Watch™ scorecardWe found that the claim with a style and personal preference focus (Claim 2) scored higher in all key metrics:
Claim Test Scorecard (Top 2 Box Scores)
Claim 1: Person and technology relationship focus
(A)
Claim 2: Style and personal preference focus
(B)
Attributes Tested n=417 n=417
Purchase Intent Statistical Significance
Top 2 Box = Extremely likely + Very likely 11% 12%
Believability Statistical Significance
Top 2 Box = Extremely appealing + Very appealing 36% 37%
Likeability Statistical Significance
Top 2 Box = Extremely unique + Very unique 33% 37% (A)
Clarity Statistical Significance
Top 2 Box = Extremely relevant + Very relevant 33% 41%
Average Score 28% 32%
*Z-test scores at 0.95 confidence interval
1717
Segmenting responses reveals additional data
Taking a further look, we segmented the market into two distinct groups: early adopters and those who wait to purchase new products.
Identifying what resonates with the target market allows for more targeted advertising and higher likeability and purchase intent.
“I usually try new products before other people do.”
Strongly agreeAgree
Early adopters
Strongly disagreeDisagree
Later-stage adopters
1818
Identifying what resonates with the target marketThe differences for early adopters, Apple’s® target market, were significant:
They found Claim 2 clearly more convincing, unique, and memorable. They also listed text messaging and phone calls as the most important for a smartwatch to have, higher than any apps their later adopter counterparts listed.
0% 0%Confusing Text
MessagingPhone Calls
Calendar Activity tracking
Camera remote
Music Maps InstagramConvincing Irritating Unique Memorable
10% 5%
20%10%
30%15%
40%
20%50%
25%60%
13%
11%
16%15%
22%
1%
7%15%
21%22%
16%17%
2%
6%6%
1%
37%
Late
Late
Early
Early
55%
23%
18%
38%
56%
35%
51%
Later stage adopters found it more irritating Later stage adopters selected
activity tracking as their top choice
2020
Going beyond your first testClaims testing goes beyond asking respondents to rank and rate claims against one another.
Take it a step further: • Ask respondents open-ended
questions to get the "Why?" behind their answers.
• Track your performance over time by setting up recurring surveys.
Need some help selecting metrics? We can help.
• SurveyMonkey can provide you a list of all the available key metrics you can test and get respondents to rate claims on.
The possibilities are endless:
• Package testing, logo testing, name testing, claims testing...the list goes on.
• Our concept testing framework spans a variety of categories—all you have to do is provide us the stimuli, and we can make that concept assessment happen. Contact us to get started
21
Bring Your Best Ideas to Market with Claims TestingLearn more about testing your marketing claims by
contacting SurveyMonkey Audience today:
Contact us to get started
© 2015 SurveyMonkey. SurveyMonkey, the SurveyMonkey logo, SurveyMonkey Audience and the SurveyMonkey Audience logo are trademarks of SurveyMonkey Inc. and its affiliates. Other company and product names may be trademarks of their respective owners.