Multivariate Testing & Conversion Tweaking - PubCon 2009
-
Upload
efficient-frontier -
Category
Business
-
view
2.326 -
download
0
description
Transcript of Multivariate Testing & Conversion Tweaking - PubCon 2009
![Page 1: Multivariate Testing & Conversion Tweaking - PubCon 2009](https://reader034.fdocuments.in/reader034/viewer/2022051610/548231aab4af9f960d8b470c/html5/thumbnails/1.jpg)
01.16.08
Multivariate Testing &Conversion Tweaking
Jason CooperDirector, Value Added Services
jason_cooper
November 11, 2009
![Page 2: Multivariate Testing & Conversion Tweaking - PubCon 2009](https://reader034.fdocuments.in/reader034/viewer/2022051610/548231aab4af9f960d8b470c/html5/thumbnails/2.jpg)
Performance-Based Design
2
We can no longer assume that what we think as designers is best for users.
Nov. 1996Oct. 1997Nov. 1999Oct. 2000
Nov. 2002Nov. 2003Nov. 2004
Nov. 2006
Today
![Page 3: Multivariate Testing & Conversion Tweaking - PubCon 2009](https://reader034.fdocuments.in/reader034/viewer/2022051610/548231aab4af9f960d8b470c/html5/thumbnails/3.jpg)
• Technology on the web is maturing
• Broadband penetration
• Down economy has made getting “more for less” a top goal
• Availability of free technology
How Did We Get Here?
3
![Page 4: Multivariate Testing & Conversion Tweaking - PubCon 2009](https://reader034.fdocuments.in/reader034/viewer/2022051610/548231aab4af9f960d8b470c/html5/thumbnails/4.jpg)
• Growing awareness of and interest in testing
• Greater ROI through post-click optimization
• Hunger for more data
What Does It Mean?
4
+
![Page 5: Multivariate Testing & Conversion Tweaking - PubCon 2009](https://reader034.fdocuments.in/reader034/viewer/2022051610/548231aab4af9f960d8b470c/html5/thumbnails/5.jpg)
• To find the granular synergies that persuade visitors you need complex statistical analysis
• Available tools remove the complexity so you can focus on what to test not how to test
• Variety of options and costs
The Technology
5
![Page 6: Multivariate Testing & Conversion Tweaking - PubCon 2009](https://reader034.fdocuments.in/reader034/viewer/2022051610/548231aab4af9f960d8b470c/html5/thumbnails/6.jpg)
• Simple Answer = EVERYTHING
• Challenge is knowing where to start
What Should You Test?
6
• First impressions• User-experience • General layouts • Radical redesigns• Better for A/B
When we think like designers
• Specific messaging• User-experience • Individual elements• Variable alternatives• Better for multivariate
When we think like marketers
What’s sticky? What resonates?
![Page 7: Multivariate Testing & Conversion Tweaking - PubCon 2009](https://reader034.fdocuments.in/reader034/viewer/2022051610/548231aab4af9f960d8b470c/html5/thumbnails/7.jpg)
Specific Tweaks
7
Call to ActionCall to Action
HeadlineHeadline
Brand PresentationBrand Presentation
Hero Shot / Main ImageHero Shot / Main Image
Product ImageProduct Image
ButtonsButtons
IncentivesIncentives
“The Process”“The Process”
![Page 8: Multivariate Testing & Conversion Tweaking - PubCon 2009](https://reader034.fdocuments.in/reader034/viewer/2022051610/548231aab4af9f960d8b470c/html5/thumbnails/8.jpg)
Test What Matters
Worst
+15%
+30%
![Page 9: Multivariate Testing & Conversion Tweaking - PubCon 2009](https://reader034.fdocuments.in/reader034/viewer/2022051610/548231aab4af9f960d8b470c/html5/thumbnails/9.jpg)
The Data Inputs
9
![Page 10: Multivariate Testing & Conversion Tweaking - PubCon 2009](https://reader034.fdocuments.in/reader034/viewer/2022051610/548231aab4af9f960d8b470c/html5/thumbnails/10.jpg)
“Plans based on average assumptions are wrong on average.”
-from The Flaw of Averages by Sam Savage
Interpreting the Data
10
![Page 11: Multivariate Testing & Conversion Tweaking - PubCon 2009](https://reader034.fdocuments.in/reader034/viewer/2022051610/548231aab4af9f960d8b470c/html5/thumbnails/11.jpg)
• It depends on your business and the tools you have available
How Should You Segment Your Data?
11
Age
Gender
Language
Country
Connection Speed
Paid Search
Natural Search
Social Media
Affiliates
Keywords
Research Mode
Consideration
Motivated
Repeat Visitor
Loyal Customer
![Page 12: Multivariate Testing & Conversion Tweaking - PubCon 2009](https://reader034.fdocuments.in/reader034/viewer/2022051610/548231aab4af9f960d8b470c/html5/thumbnails/12.jpg)
• Qualitative data can provide flavor, texture, and nuance to quantitative data
• Typically quantitative data can’t tell you “why”
Combining Data Types
12
+ = MeaningfulSegments
+ = UnderstandThe “Why”
![Page 13: Multivariate Testing & Conversion Tweaking - PubCon 2009](https://reader034.fdocuments.in/reader034/viewer/2022051610/548231aab4af9f960d8b470c/html5/thumbnails/13.jpg)
• Slice, dice, and compare data any way you can to find relevant relationships
• Look for cause and effect
• Data on your competitors can provide the missing piece of the puzzle
• Utilize 3rd party tools to add more “color” to your data
Find the Relationships
![Page 14: Multivariate Testing & Conversion Tweaking - PubCon 2009](https://reader034.fdocuments.in/reader034/viewer/2022051610/548231aab4af9f960d8b470c/html5/thumbnails/14.jpg)
Twitter Spectrum
![Page 15: Multivariate Testing & Conversion Tweaking - PubCon 2009](https://reader034.fdocuments.in/reader034/viewer/2022051610/548231aab4af9f960d8b470c/html5/thumbnails/15.jpg)
Facebook Lexicon
![Page 16: Multivariate Testing & Conversion Tweaking - PubCon 2009](https://reader034.fdocuments.in/reader034/viewer/2022051610/548231aab4af9f960d8b470c/html5/thumbnails/16.jpg)
• Google’s “41 shades of blue”
• Involve all of your business units
• Set aside a portion of your budget for testing
• Data can breed creativity—allows you to take risks
Build a Culture of Testing
16
![Page 17: Multivariate Testing & Conversion Tweaking - PubCon 2009](https://reader034.fdocuments.in/reader034/viewer/2022051610/548231aab4af9f960d8b470c/html5/thumbnails/17.jpg)
• Doubling your conversion rate is far easier (and cheaper) than doubling your traffic
• Get to know your data (all of it) intimately so you can uncover relationships
• Segment and combine your data for more meaningful interpretation
• Test everything until you know what matters
• Develop a culture of testing
• Evolve through continuous testing and improvement cycles—become data-driven
Take Aways
17
![Page 18: Multivariate Testing & Conversion Tweaking - PubCon 2009](https://reader034.fdocuments.in/reader034/viewer/2022051610/548231aab4af9f960d8b470c/html5/thumbnails/18.jpg)
01.16.08
Thank [email protected]
Jason CooperDirector, Value Added Services
jason_cooper
November 11, 2009