Negative Lifts: How a 24% loss produced a 141% increase in conversion

Negative Lifts: How a 24% loss produced a 141% increase in conversion

Educational Funding Provided by:

#webclinicFunding provided by:

Join the conversation on Twitter

#webclinic


Today’s team

Dr. Flint McGlaughlinManaging Director

Jon PowellSenior Research Manager


Background and test design

Background: RegOnline is event management software that lets users create online registration forms and event websites to manage their events.

Goal: To increase number of completed leads on home page.

Primary research question: Which page will achieve the greatest addressable lead rate?

Approach: A/B multifactor split test

Experiment ID: RegOnline Homepage TestLocation: MarketingExperiments Research LibraryTest Protocol Number: TP1427

Research Notes:

�


5

Homepage Step 1Experiment: Control

•Testimonials

•Lead form

•Screenshots


6

Experiment: Control

Homepage Step 2


7

Experiment: Treatment Homepage Step 1

• Clarified the Value Proposition of the offer to emphasize Free Access.

• Reduced form fields in first step to capture more partial leads.

• Reduced friction by matching flow to thought sequence.


8

Experiment: Treatment

Homepage Step 2


9

Experiment: Treatment

Homepage Step 3


10

Experiment: Side-by-sideControl Treatment


11

Versions Conversion Rate Rel. diff

Control – Two-step home page 2.3% -

Treatment – Three-step home page 1.7% -24.5%

24.5% Decrease in Conversion The Treatment generated 24.5% less completed leads

What you need to understand: In spite of having clearer value and reducing the amount of form fields in the first-step the control still out-performed the treatment.

Experiment: Results


12

Background: RegOnline is event management software that lets users create online registration forms and event websites to manage their events.

Goal: To increase number of completed leads on home page.

Primary research question: Which process will yield a higher conversion rate?

Approach: A/B multifactor split test

Experiment ID: RegOnline Homepage TestLocation: MarketingExperiments Research LibraryTest Protocol Number: TP1428

Research Notes:

�Experiment 2: Background and test design


13

Experiment 2: Side-by-sideControl Treatment


14

Versions Conversion Rate Rel. diff

Control 0.3% -

Treatment 0.5% 89.8%

89.8% Increase in Conversion The Treatment generated 89.8% more completed leads

What you need to understand: Using what we learned in previous tests, we directed the flow of the visitor’s thought sequence through the value before revealing the price. This is what gave us our lift.

Note: After further testing, we were finally able to increase conversion on the homepage by 141%

Experiment 2: Results


15

F Key Principles

A. The goal of a test is not to get a lift, but rather to get a learning

B. To achieve the maximum amount of learning, your tests should be designed around two key elements:

1. A research question

How did we turn this loss into a lift?


16

Test Protocol Section 1

1. Question

1. < Final research question… >

u = 2q + t + m+ 2v + i ©

Part 1: The research question


Definition: The research question

The Research Question:

A research question is a behavioral question expressed in a factorial split so it must begin with “which.”


18

Example #1:

Not this..

What is the best price for product X?

But this…

Which of these three price points is best for product X?

Example #2:

Not this..

Why am I losing customers in the last step of my checkout process?

But this…

Eliminating which form element best reduces customer drop-off?

Examples: The research question


19

How would you refine the following research questions??1. What is the best headline for my landing page?

2. How many objectives should I have on my homepage?

The research question


20

F Key Principles




2. A theory question

What we discovered


Definition: The theory question

The Theory Question:

The theory question is a broader question about your customer that is derived from asking “WHY” of the research question.

The answer to the theory question should contribute to a wider understanding of your customer.

Our researchers call this the Customer Theory.


22

Chart: Marketers don’t validate results


23

Which? Why? What?

Research Question Theory Question

Theory Question: Behavior reflects identity


24

Value Proposition Question:

If I am your ideal customer, why should I buy from you rather than your competitors?

Which? Why? What?




25

€Value Proposition Question:


Which? Why? What?



Ideal Customer


26

€Value Proposition Question:


Which? Why? What?



Ideal Customer


27

€Ideal Customer

Value Proposition Question:


Which? Why? What?




28

Example: The theory question

Why?

Which headline will generate a higher response?

A. Save $100 on product X

B. Product X comes with a lifetime guarantee

Research Question

What does my ideal customer prefer?

A. Cost savings

B. Product quality

Theory Question


29

Example: The theory question

Why?

Which call to action will generate a higher response?

A. Get your product X now

B. Learn more about product X

Research Question

Where is my ideal customer in the thought sequence?

A. Ready to buy?

B. Ready to learn more?

Theory Question


30

24%

Homepage

Theory Question: How we turned our loss into a win


31

24%

Homepage

Why?



32

24%

Homepage


Why?

548%

SEO Landing Page


33

24%

Homepage


Why?

548%

SEO Landing Page

Why?


34

24%

Homepage


Why?

548%

SEO Landing Page

Why?

90%

Homepage


35

24%

Homepage


Why?

548%

SEO Landing Page

Why?

90%

Homepage

Why?


36

24%

Homepage


Why?

548%

SEO Landing Page

Why?

90%

Why?

27%

Homepage


37

24%

Homepage


Why?

548%

SEO Landing Page

Why?

90%

Why?

27%

Homepage

141% Cumulative Lift


38

201%

Test results are interpreted and a second test was created based on

the analyst’s observations

The theory question applied


39

201%




2%

Again, test results are interpreted and the next

round of testing is started for this page


40

201%




2%

Again, test results are interpreted and the next

round of testing is started for this page

29%

Test is again interpreted and transferrable principles are applied to other offer pages


41


451%

302%

603%257% 28%

• The discoveries and insights about customer motivation from the three prior tests were applied to other landing pages and used to optimize PPC campaigns.

• The purposeful effort to identify and selectively apply these transferrable insights led to widespread optimization gains .


42

Summary: Putting it all together

F Key Principles




2. A theory question


43

Get a Free website grade from HubSpot

Get a FREE detailed report of how your site is doing compared to your peers and competitors in:

-Generating Traffic-Search Engine Optimization-Social Engagement

Visit the link below for more…

Check Out HubSpot’s Website Grader

bit.ly/wsgrader

http://bit.ly/wsgrader

http://bit.ly/wsgrader


44

Testing Strategy Sessions


45

Strategy Session #1: QuoteScout.com


46



47



48

Strategy Session #2: Warmup.com


49



50



Strategy Session #3: Clinics-trials-events.com


54

Strategy Session #3: Clinics-trials-events.com


55

Strategy Session #4: Cooksdirect.com


View the original presentation free

You can view the full audio and visual presentation at MarketingExperiments.com

View the original presentation >>

http://www.marketingexperiments.com/site-optimization/negative-lifts.html

http://www.marketingexperiments.com/site-optimization/negative-lifts.html

Negative Lifts: How a 24% loss produced a 141% increase in conversion

Business

Transcript of Negative Lifts: How a 24% loss produced a 141% increase in conversion