Ektron Synergy: Fail Faster with Multivariate Testing
-
Upload
tom-wentworth -
Category
Technology
-
view
924 -
download
2
description
Transcript of Ektron Synergy: Fail Faster with Multivariate Testing
![Page 1: Ektron Synergy: Fail Faster with Multivariate Testing](https://reader038.fdocuments.in/reader038/viewer/2022103016/5552894db4c905115b8b5337/html5/thumbnails/1.jpg)
Fail Faster with Multivariate Testing
Tom Wentworth| VP, Web Solutions | @twentworth12
![Page 2: Ektron Synergy: Fail Faster with Multivariate Testing](https://reader038.fdocuments.in/reader038/viewer/2022103016/5552894db4c905115b8b5337/html5/thumbnails/2.jpg)
In this session, you’ll learn:
• The Importance of website testing
• The difference between A|B and multivariate testing
• Testing best practices
• Creating A|B and multivariate Tests in Ektron
![Page 3: Ektron Synergy: Fail Faster with Multivariate Testing](https://reader038.fdocuments.in/reader038/viewer/2022103016/5552894db4c905115b8b5337/html5/thumbnails/3.jpg)
Optimize Landing Pages with Website Testing
• Test copy, images, pricing, call to action, layout
• Track Revenue, Leads, Subscriptions, Registrations, Donations
• A|B vs. Multivariate
![Page 4: Ektron Synergy: Fail Faster with Multivariate Testing](https://reader038.fdocuments.in/reader038/viewer/2022103016/5552894db4c905115b8b5337/html5/thumbnails/4.jpg)
When to use A|B vs. Multivariate
![Page 5: Ektron Synergy: Fail Faster with Multivariate Testing](https://reader038.fdocuments.in/reader038/viewer/2022103016/5552894db4c905115b8b5337/html5/thumbnails/5.jpg)
• A|B Testing– Test Broader Page Concepts– “Low Hanging Fruit”– Lower Traffic Requirements– Difficult to Control External Factors
• Multivariate Testing–More Learning–More Complexity, Planning, Traffic
![Page 6: Ektron Synergy: Fail Faster with Multivariate Testing](https://reader038.fdocuments.in/reader038/viewer/2022103016/5552894db4c905115b8b5337/html5/thumbnails/6.jpg)
Starting with A|B Testing
A B(Winner: A – 44% more Registrations)
![Page 7: Ektron Synergy: Fail Faster with Multivariate Testing](https://reader038.fdocuments.in/reader038/viewer/2022103016/5552894db4c905115b8b5337/html5/thumbnails/7.jpg)
Evolve with Multivariate Testing
B
A
D
E
C
![Page 8: Ektron Synergy: Fail Faster with Multivariate Testing](https://reader038.fdocuments.in/reader038/viewer/2022103016/5552894db4c905115b8b5337/html5/thumbnails/8.jpg)
“There are two possible outcomes: if the result confirms the hypothesis, then you've made a measurement. If the result is contrary to the hypothesis, then you've made a discovery.”
- Enrico Fermi
![Page 9: Ektron Synergy: Fail Faster with Multivariate Testing](https://reader038.fdocuments.in/reader038/viewer/2022103016/5552894db4c905115b8b5337/html5/thumbnails/9.jpg)
A|B Testing at Take Charge America
Challenger Champion
38% - 4% over lift
![Page 10: Ektron Synergy: Fail Faster with Multivariate Testing](https://reader038.fdocuments.in/reader038/viewer/2022103016/5552894db4c905115b8b5337/html5/thumbnails/10.jpg)
![Page 11: Ektron Synergy: Fail Faster with Multivariate Testing](https://reader038.fdocuments.in/reader038/viewer/2022103016/5552894db4c905115b8b5337/html5/thumbnails/11.jpg)
Landing Page Optimizationat Lloyds TSB
• Full range of banking and financial services
• 5th largest bank in Europe
• Results:– 90% increase in visitors– 150% increase in leads– 30% increase in visitor
to registered lead conversion rate
![Page 12: Ektron Synergy: Fail Faster with Multivariate Testing](https://reader038.fdocuments.in/reader038/viewer/2022103016/5552894db4c905115b8b5337/html5/thumbnails/12.jpg)
Testing the Three C’sContent, Copy, Call-to-Action
![Page 13: Ektron Synergy: Fail Faster with Multivariate Testing](https://reader038.fdocuments.in/reader038/viewer/2022103016/5552894db4c905115b8b5337/html5/thumbnails/13.jpg)
Testing the Three C’s
• Content– Test Images, layout, forms, rich media,
social widgets
• Copy– Strong, Clear, Direct Headline
• Call To Action– Be Specific, Obvious, and Singular
![Page 14: Ektron Synergy: Fail Faster with Multivariate Testing](https://reader038.fdocuments.in/reader038/viewer/2022103016/5552894db4c905115b8b5337/html5/thumbnails/14.jpg)
Select the Right Pages to Test
• A single, specific and “easy to measure” goal
• A page with a poor conversion rate• Enough conversions to test in a
reasonable amount of time
![Page 15: Ektron Synergy: Fail Faster with Multivariate Testing](https://reader038.fdocuments.in/reader038/viewer/2022103016/5552894db4c905115b8b5337/html5/thumbnails/15.jpg)
Know When to Hold Them…
Resist the Temptation to End Early
![Page 16: Ektron Synergy: Fail Faster with Multivariate Testing](https://reader038.fdocuments.in/reader038/viewer/2022103016/5552894db4c905115b8b5337/html5/thumbnails/16.jpg)
Don’t stop running a test too early
or too late
• Too early risks promoting the wrong version
• Too late wastes time that could be spent on new tests
![Page 17: Ektron Synergy: Fail Faster with Multivariate Testing](https://reader038.fdocuments.in/reader038/viewer/2022103016/5552894db4c905115b8b5337/html5/thumbnails/17.jpg)
Rinse. Lather. Repeat
Drive a culture of continuous Improvement
![Page 18: Ektron Synergy: Fail Faster with Multivariate Testing](https://reader038.fdocuments.in/reader038/viewer/2022103016/5552894db4c905115b8b5337/html5/thumbnails/18.jpg)
Design for Continuous Improvement
![Page 19: Ektron Synergy: Fail Faster with Multivariate Testing](https://reader038.fdocuments.in/reader038/viewer/2022103016/5552894db4c905115b8b5337/html5/thumbnails/19.jpg)
Building your FirstMultivariate Experiment in Ektron
(Demo)
![Page 20: Ektron Synergy: Fail Faster with Multivariate Testing](https://reader038.fdocuments.in/reader038/viewer/2022103016/5552894db4c905115b8b5337/html5/thumbnails/20.jpg)
\“Continuous Improvement is Better than Delayed Perfection”
- Mark Twain
![Page 21: Ektron Synergy: Fail Faster with Multivariate Testing](https://reader038.fdocuments.in/reader038/viewer/2022103016/5552894db4c905115b8b5337/html5/thumbnails/21.jpg)
Additional Resources
• Ektron Whitepaper – Fail Faster with Multivariate Testing– http://www.ektron.com/literature/whitepapers/fail
_faster_with_multivariate_testing.pdf• Sample Size Estimation Tool
– http://www.websharedesign.com/tools/sample-size-estimation-tool/
• Tim Ash “Seven Deadly Sins of Landing Page Design” Webinar– http://www.youtube.com/watch?v=erdEZvOq6wo
• Occam’s Razor Blog by Avinash Kaushik– http://www.kaushik.net/avinash/