Million $ Button Optimization - Which Test Won TLE 2014
-
Upload
michal-parizek -
Category
Data & Analytics
-
view
1.536 -
download
1
description
Transcript of Million $ Button Optimization - Which Test Won TLE 2014
Case Study:
avast! – Million $ Button Optimization
Michal Parizekavast!
Senior E-Commerce Specialist
#TLE2014The Live Event@parezem
London
#TLE2014The Live Event@parezem©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
1988
Founded in Czech Republic
Switched toFreemium
2001 2014
Users: 219,000,000Valued: $1,000,000,000
in 1988 AVAST was founded in Czech Republic by two researches Eduard Kucera and Pavel Baudis. It had good early 90s, but then the competition got fiercer. A major decision to switch to a freemium model saved the business. Since 2001 the business has been growing and nowadays AVAST protects more than 219M users and it is valued as a $1B company.
#TLE2014The Live Event@parezem©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Paid version
Trialversion
Website
Other
Free version
Breakdown of sales
We can see two main streams of revenue for AVAST: sales which come from the Free version (customers who decide to switch to a premium protection) and sales which come from the Paid version (subscription renewals and further monetization). My focus is on the retention business (=Paid version).
Breakdown of retention
sales
RENEW NOW
#TLE2014The Live Event@parezem©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
If we look at the retention sales in more detail we could see the key sales elements. The most of the sales is done by an expiration messaging (small and big pop-ups & expiration email). The other element is the “Renew now” button which makes around 25% of all consumer retention sales. This button is a center point of this case study.
#TLE2014The Live Event@parezem©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
RENEW NOW
The “Renew now” button sits in the program GUI. It appears once a customer is in an expiration period (starting 30 days before an expiration date) and stays there until a customer renews her protection.
Champion
#TLE2014The Live Event@parezem©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
ChallengerRENEW NOW RENEW NOW
In the autumn of 2013 we tried a different approach in the “Renew now” button sales flow. Champion (default) version meant the button opened a standard expiration big pop-up which a customer knew very well. The challenger version meant a new landing page which contained a clear comparison table of all three consumer products.
#TLE2014The Live Event@parezem©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
RENEW NOW RENEW NOW
Sales per View:
+ 13.4%
WinnerChampion Challenger
The challenger worked much better. We could see that customers who clicked on the “Renew now” button preferred to do a more informed decision – to see where their current solution stands and to see all the features of all options available. The champion version contained only different length of subscription period.
#TLE2014The Live Event@parezem©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
There is the comparison landing page in more detail. A customer can see all the features and the prices of all options. This entire new landing page which was responsible for around 25% of sales was a starting point of a 4-month long optimization project which in fact still continues (in the time presenting the case study).
#TLE2014The Live Event@parezem©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
William PoundstonePriceless: The Myth of Fair Valuehttp://bit.ly/price-less-book
I’d like to start the optimization part with an experiment William Poundstone describes in his book Priceless. They’ve approached students with two beers: $1.80 & $2.50. 80% of students picked the more expensive beer. Later they’ve approached students again and this time the menu contained also $1.60 beer. Facing those 3 options 80% of students picked the $1.80 beer..
Champion
#TLE2014The Live Event@parezem©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Challenger
2% 77% 21%Once we read the experiment we just had to try it! Our new landing page looked like the second round of that experiment. Almost 80% chose the middle option – Internet Security. We hoped that by removing the cheapest options the majority could also pick the most expensive option – Premier.
Champion
#TLE2014The Live Event@parezem©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Challenger
2% 77% 21% 75% 25%
Sales per View:
- 1.2%
Loser
Well, it did not happen.. We could see a slight increase of Premier’s share, but there was a decrease of a conversion rate. We believe there were three main reasons why it failed: 1) The prices are higher than in the beer experiment, more than 10x. 2) Don’t forget, it is retention business, customers had at least a 1-year experience with the middle product. 3) Decoy effect of the cheapest option.
#TLE2014The Live Event@parezem©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
William PoundstonePriceless: The Myth of Fair Valuehttp://bit.ly/price-less-book
Anyway, the Poundstone’s experiment continued. They’ve approached the students for the third time – to the main two beers they’ve added a super premium beer – a $3.40 one. 10% of students picked it and the share of $2.50 beer also increased.
#TLE2014The Live Event@parezem©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
2% 77% 21%
Champion Challenger
We had to try this one as well. However we had a crucial issue – we did not have a super premium product.. So we made it up. We bundled our premium product (Premier) with one of our attached products SecureLine VPN. And we called an Ultimate Security Pack! (I know sounds cheesy)
#TLE2014The Live Event@parezem©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
2% 77% 21% 73% 18% 9%
Champion Challenger
Sales per View:
- 1.7%
Loser
And it failed again. We persuaded almost 10% of customers to pick the super premium product. But! The share of Premier did not increased and again we could see a drop in a conversion rate.
#TLE2014The Live Event@parezem©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Champion Challenger
We moved our attention from the landing page to the sales flow behind it. Important thing to say is that customers can buy a 1-year subscription from landing page. What we wanted to try was to implement an interstitial element which offers customers a good deal of 2-year subscription. The message was simple: “Two are better than one!”
#TLE2014The Live Event@parezem©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Champion Challenger
Sales per View:
+ 20.9%
Winner
And the interstitial element worked fantastically! We could see a 21% increase in Sales per View. A lot of customers agreed to the 2-year subscription. It is interesting that this upsell did not work in our pop-up expiration flow in the past. It seems to be working only when the sales element is opened on a customer’s intention. Then they are happy to consider also other options.
#TLE2014The Live Event@parezem©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Champion Challenger
Then we tested the interstitial element itself. Champion version is the more straight forward message – “Take it or leave it”. The challenger version provides more options (also the 3-year is available) and it is also more transparent – you can see all prices. Plus there is a recap of your choice with key features listed.
#TLE2014The Live Event@parezem©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Champion Challenger
Sales per View:
- 9.8%
Loser
The more transparent version failed.. In USA.
#TLE2014The Live Event@parezem©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Champion Challenger
Sales per View:
+ 10.2%
Winner
But it succeeded in the rest of the world. It is not the first time a more straightforward and simpler design works great in the USA. On the other hand a different design (more transparent, more information) usually works better in Europe and our other key markets.
#TLE2014The Live Event@parezem©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Champion Challenger
In the more transparent version we felt pre-selecting a 2-year option will be better for sales. Obviously we were not sure so we A/B tested it.
#TLE2014The Live Event@parezem©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Champion Challenger
Sales per View:
- 13,3%
Loser
And it proved to be really better. When we pre-select certain option in a sales dialog we can see that more customers actually pick this option. And the pre-selection in our case does-not have an impact on a conversion rate.
#TLE2014The Live Event@parezem©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Champion Challenger
Last thing what we’ve tested so far was a 1PC/3PC switcher. We have already managed to successfully upsell customers to 2-year subscriptions. Now we wanted to upsell them to a 3PC version which they can use to secure more PCs.
#TLE2014The Live Event@parezem©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Champion Challenger
Sales per View:
+ 13.6%
Winner
We have results only from France so far but is seems to be working very well. It has a positive impact not only to AOV but also on a conversion rate as they seem to appreciate more options (like in the case of the interstitial element).
Total Optimization Score
#TLE2014The Live Event@parezem©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Incremental monthly sales
+ $200k2013M12 2014M01 2014M02 2014M03
“Renew now” Button LP - Sales per View
18%
In those 4 months of a continual optimization we could get 18% lift in Sales per View (“Renew now” landing page) – it is across all markets and all consumer paid products. It resulted in a massive lift of sales. Monthly retention sales increased by $200k incrementally.
• If a sales element is opened on customers’ intention there is a higher chance to upsell
• Always test your hypothesizes, don’t just trust case studies
• Decoy product effect works• Pre-selection influences purchase behavior• US/Europe purchase behavior differences
#TLE2014The Live Event@parezem©2014, WhichTestWon.com a division of Anne Holland Ventures Inc. All rights reserved.
Takeaways
To wrap it up there are few key takeaways we’ve learned.
Contact InfoMichal Parizekavast!Senior E-Commerce Specialist+420 604 927 [email protected]
#TLE2014The Live Event@parezem
London