Landing Page Optimization:

How to get better ROI on the traffic you’re already sending to your site

Nov. 29, 2012

#WTwebinar

• Daniel Burstein, Director of Editorial Content MECLABS/MarketingSherpa Twitter: @DanielBurstein

• Adam Lapp, Associate Director, Optimization and Strategy MECLABS/MarketingSherpa Twitter: @AdamLapp

• Kirk Ramble, Optimization Consultant Webtrends Optimization Solutions

Introductions

#WTwebinar

Follow today’s conversation

#WTwebinar

#WTwebinar

Webtrends provides optimization solutions to increase conversion and revenue

• Landing page optimization• Site optimization• Mobile/Social optimization• A/B and MVT testing• Visitor segmentation• Content targeting• Experience and expertise

#WTwebinar

MarketingSherpa is a research and publishing organization serving the marketing community

• MarketingSherpa’s annual research cycle provides knowledge for continuous marketing improvement

Research Background

• 2,677 qualified survey responses

• In 10 major industry verticals

• Key marketing insights on:

• Website optimization • Optimization ROI• Optimization Strategies• Testing and Analytics• Optimization Strategy Integration

• Key success stories

#WTwebinar

Research Background

• 2,673 qualified survey responses

• Over 190 charts with analytical commentary

• Key marketing insights on:

• Optimization tactics • C-level ROI and budgeting

perspectives • Testing and Analytics• Optimization Challenges

• Key success stories

#WTwebinar

#WTwebinar

Today, we will discuss…

• The case for LPO, information you can use to help you secure budget approval

• The challenges with statistical validity, to help you avoid making overconfident and perhaps erroneous assumptions based on misleading numbers

• How you can use LPO to keep up with your ever-changing customers in an ever-changing marketplace

#WTwebinar

The Case for LPO Validity ChallengesOptimization in a Changing Marketplace

2

3

1

Landing Page Optimization

3 Keys to successful online testing

#WTwebinar

“Business exists to supply goods and services to customers, rather than to supply jobs to workers and managers, or even dividends to stockholders.

– Peter Drucker

“

The Internet as a Research Lab

The Internet has become the most efficient means of gathering business intelligence BEFORE a major online (or offline) campaign.

Behavioral Experimentation

Opinion Research

Marketing Intuition

Level 1

Level 2

Level 3

The Decision Resolution

#WTwebinar

#WTwebinar

The Case for LPO: Why Test?

• 47% of marketers use optimization testing to inform customer theory

• More than half of marketers fail

to fully deploy optimization within their organizations

Q. Does your organization use website optimization and/or testing to draw conclusions about your customer base?

#WTwebinar

“How to get the entire Web IT, managers and copy writers fired? They are 5 years behind the rest of the other retailers. What they do on the website is totally subpar. It really is sad. Many dollars lost every day!”

- Benchmark Study Participants

Marketer Insights: Overcoming challenges to LPO

#WTwebinar

CMO or business unit head makes

the decision23%

Marketing department

decides based on published best

practices16%

Marketing department

decides collaboratively

36%

Other4%

Validated test result determines

the decision21%

Source: ©2011 MarketingSherpa Landing Page Optimization Benchmark Survey Methodology: Fielded Feb ruary 2011, N=2,673

Who called the shots in 2010?“Only one-fifth of organizations surveyed made decisions based on validated test results.” – 2011 MarketingSherpa Landing Page Optimization Benchmark Report

#WTwebinar

LPO without testing (based only on best practices)

33%

LPO, including testing

36%

No LPO31%

Source: ©2011 MarketingSherpa Landing Page Optimization Benchmark Survey Methodology: Fielded Feb ruary 2011, N=2,673

State of LPO in 2010

#WTwebinar

Getting started in LPO doesn’t have to be difficultEven with very small and simple changes you can gain great insight about your customer and receive dramatic results.

Control Stock image of customer service rep

Image of well-known company founder.

Treatment

35%IN CONVERSION

#WTwebinar

Huge wins are possible from very small changes

Control Stock image of customer service rep

Moved primary CTA to center of page

Treatment

280+%CONVERSION LIFT

Control Treatment

#WTwebinar

Small changes together make a big impact• Presented information as tools or modules• Adjusted price and free trial messages• Revised copy to be more benefit-based

12+%CONVERSION LIFT

Control Treatment

#WTwebinar

The Case for LPOValidity ChallengesOptimization in a Changing Marketplace

2

3

1

Landing Page Optimization

3 Keys to successful online testing

20 #WTwebinar

Validity Challenges: How marketers validate test results

21 #WTwebinar

Level of ConfidenceSample SizeSignificant

Difference

Validity Challenges: Elements of statistical significance

#WTwebinar

Validity Challenges: Significant Difference

*…except when it’s not

11% is more than 10%*

#WTwebinar

Validity Challenges: Sample Size

“Well, you’re alive today even though you didn’t have one of those fancy car seats.” – My Mom

“Compared with seat belts, child restraints…were associated with a 28% reduction in risk for death.”

– Michael R. Elliott, PhD; Michael J. Kallan, MS; Dennis R. Durbin, MD, MSCE; Flaura K. Winston, MD, PhD

n=2

n=7,813

#WTwebinar

Validity Challenges: Sample Sizes

Factors in determining Sample Size• Test complexity (number of versions being tested) • Conversion rate• Performance difference between variations • Confidence level• But – too short a test may not be as valid as it looks, especially if

distribution of time is a factor

Be realistic about what kind of test your site can support

#WTwebinar

Validity Challenges: Levels of Confidence

26 #WTwebinar

The math – 5 times out of every 100 that I do the coin-flip experiment, I expect to get a difference between my two samples that's AT LEAST as big as this one - even though there is NO ACTUAL difference...

Imagine an experiment… • Take one FAIR coin. (i.e., if flipped times, would come out heads 50%). • Flip the coin ‘n’ (many) times and record # Heads (e.g., say 60 times) • Then do it over and over again; same # flips.

Proportional to #-times it comes out

with that many Heads

Validity Challenges: Levels of Confidence

27 #WTwebinar

• Most common is 95% (i.e., 5% chance you’ll think they’re different when they’re really not)

• There is no ‘magic’ to the 95% LoC. • Mainly a matter of ‘convention’ or agreement. • The onus for picking the ‘right’ level for your test is on YOU.

• Sometimes the tools limit you

• 95% is seldom a “bad” choice.

• Higher = Longer test• Bigger difference needed for validity

• Decide based on… • Level of risk of being wrong vs. cost of prolonging the test.

Confidence Interval Limits

Validity Challenges: Levels of ConfidenceHow do I decide on the right level?

#WTwebinar

Validity Challenges: You can’t trust data in isolation

• History Effect - Something happens in the outside world that causes flawed data in the test

• Instrumentation Effect- When a test variable is affected by a change in the measurement instrument

• Selection Effect- Occurs when we wrongly assume some portion of the traffic represents the totality of the traffic

29 #WTwebinar

Background: Consumer company that offers online brokerage services

Goal: To increase the volume of accounts created online

Primary research question: Which page design will generate the highest rate of conversion?

Test Design: A/B/C/D multi-factor split test

Experiment ID: (Protected)Location: MarketingExperiments Research Library

Research Notes:

�Validity Challenges: Experiment

30 #WTwebinar

• Heavily competing imagery and messages

• Multiple calls-to-action

ROTATING BANNER

Control

Experiment: Control

31 #WTwebinar

• Most of the elements on the page are unchanged, only one block of information has been optimized

• Headline has been added

• Bulleted copy highlighted key value proposition points

• Chat With a Live Agent CTA removed

• Large, clear call-to-action has been added

ROTATING BANNER

Treatment 1Experiment: Treatment 1

32 #WTwebinar

• Left column remained the same, but we removed footer elements

• Long copy, vertical flow

• Added awards and testimonials in right-hand column

• Large, clear call-to-action similar to Treatment 1

ROTATING BANNER

Treatment 2

Experiment: Treatment 2

33 #WTwebinar

• Similar to Treatment 2, except left-hand column width reduced even further

• Left-hand column has a more navigational role

• Still a long copy, vertical flow, single call-to-action design

ROTATING BANNER

Experiment: Treatment 3

34 #WTwebinar

ControlTreatment 1

Treatment 2

Treatment 3

Experiment: Side-by-side

35 #WTwebinar

What you need to understand: According to the testing platform we were using, the aggregate results came up inconclusive. None of the treatments outperformed the control with any significant difference.

No Significant Difference None of the treatment designs performed with conclusive results

Test Designs Conversion Rate Relative Diff%

Control 5.95% -

Treatment 1 6.99% 17.42%

Treatment 2 6.51% 9.38%

Treatment 3 6.77% 13.70%

Experiment: Results

36 #WTwebinar

3.00%

5.00%

7.00%

9.00%

11.00%

13.00%

15.00%

17.00%

19.00%

ControlTreatment 3

• However, we noticed an interesting performance shift in the control and treatments towards the end of the test.

• We discovered that during the test, there was an email sent that skewed the sampling distribution.

Conv

ersi

on R

ate

Test Duration

Treatment consistently is beating the control

Control beats the treatment

Experiment: Validity Threat

37 #WTwebinar

Test Designs Treatment Relative Diff%

Control 5.35% -

Treatment 1 6.67% 25%

Treatment 2 6.13% 15%

Treatment 3 7.03% 31%

What you need to understand: After excluding the data collected after the email had been sent out, each of the treatments substantially outperformed the control with conclusive validity.

Experiment: Results

increase in conversion The best treatment outperformed the control by 31%31%

#WTwebinar

Testing Until You Achieve Stabilization

#WTwebinar

Selection Effect: The portion is not the whole of your traffic

Selection Effect: The effect on a dependent variable, by an extraneous variable associated with different types of subjects not being evenly distributed between experimental treatments.

Examples

• Channel profile does not match customer profiles

• Uneven distribution of traffic from sources among treatments

• Self selection (bias)

#WTwebinar

History Effect: Outside world causes flaws

History Effect Definition: The effect on a dependent variable by an extraneous variable associated with the passing of time.

Experiment

We conducted a 7-day headline test for a site that provides search and mapping services to a nationwide database of registered sex offenders.

Treatments

1. Child Predator Registry (control)

2. Predators in Your Area

3. Find Child Predators

4. Is Your Child Safe? lessCTR than all other treatments52%

#WTwebinar

History Effect: Outside world causes flaws

BUT OUR DATA WAS FLAWED

During the test period, the nationally syndicated NBC television program Dateline aired a special called “To Catch a Predator.”

This program was viewed by approximately 10 million individuals, many of them concerned parents. Throughout this program sex offenders are referred to as “predators.”

#WTwebinar

Instrumentation Effect: Test affected by tools

Instrumentation Effect: The effect on the dependent variable, caused by a variable external to an experiment, which is associated with a change in the measurement instrument.

Examples

• Short-duration response time slowdowns • E.g., due to server-load, page-weight, page-

code problems

• Splitter malfunction

• Inconsistent URLs

• Server downtime

#WTwebinar

The Case for LPOValidity ThreatsOptimization in a Changing Marketplace

2

3

1

Landing Page Optimization

3 Keys to successful online testing

#WTwebinar

Background: Company is a publisher of electronic marketing information and offers related services.

Goal: Increase registrations for a free email newsletter.

Primary research question: Which sign-up page will yield the highest conversion rate?

Approach: A/B/C Multivariate test involving changes in headline, credibility indicators, and images according to optimization best practices.

Experiment ID: (Protected)Location: MarketingExperiments Research LibraryTest Protocol Number: #TP1092

Research Notes:

�Experiment: Background

#WTwebinar

Original Page

• We radically redesigned this page based upon common best practices in landing page optimization.

Experiment: Control

#WTwebinar

• Clearer headline emphasizes the value proposition.

• “Featured Clients” list emphasizes value and reduces anxiety.

• Bolded key terms make body copy easier to read and scan.

• Body copy uses quantitative benefits

• Costumer testimonials reduce anxiety.

• Anti-spam seal reduces anxiety.

Treatment 1

Experiment: Treatment 1

#WTwebinar

• Headline is quantitative to emphasize the value proposition.

• Added more testimonials.

• Customer logos directly in the eye path

• Added personal feel with images and hand-written signature.

• “Tell me where to send…” language used.

Treatment 2Experiment: Treatment 2

#WTwebinar

Treatment 1 Treatment 2Control

Which of these marketing campaigns had the highest conversion rate?

Experiment: Side by Side

#WTwebinar

Vs.

Original Treatment 1

Conversion Rate = 14.26% Conversion Rate = 6.74%

53%

Experiment: Results

#WTwebinar

Treatment 2

52%

Experiment: Results

Vs.

Original

Conversion Rate = 14.26% Conversion Rate = 6.84%

#WTwebinar

Was this test a failure?

‒ No, because we learned something important about our customers.

#WTwebinar

Background: Company is a publisher of electronic marketing information and offers related services.

Goal: Increase registrations for a free email newsletter.

Primary research question: Which sign-up page will yield the highest conversion rate?

Approach: A/B multi-factorial test with a minimalist strategy of reducing page elements.

Experiment ID: (Protected)Location: MarketingExperiments Research LibraryTest Protocol Number: #TP1115

Research Notes:

�Experiment: Background

#WTwebinar

• Common landing page best practices failed to improve conversion on this original page.

• If adding elements to increase the value proposition decreased conversion, maybe the traffic to this page was already highly motivated?

• Maybe visitors didn’t need to see more value and more credibility indicators…

Original

Experiment: Control

#WTwebinar

• Maybe visitors just needed a simple, easy process.

• Much of the copy on this page is removed, leaving simple form submission fields.

• No real selling points are included in this design.

Treatment

Experiment: Treatment

#WTwebinar

Landing Page CTR

Original 12.09%

Treatment 21.54%

Relative Difference: 78%

78% Increase in Conversion Treatment 1 increased conversion by 78%

Experiment: Results

#WTwebinar

What you need to understand: We can no longer rely on speculation, marketer intuition, or even “best practices” for our marketing efforts. We must test because what worked for your colleague might not work for you.

And more importantly, we must design tests that provide insight about our customers.

X X

Experiment: Results

#WTwebinar

The Importance of a Good Hypothesis

Structure your hypothesis:

“If I ___ (do this) ___, then ___ (this) ___ will happen.”

#WTwebinar

The Webtrends Optimization Process

#WTwebinar

• Daniel Burstein, Director of Editorial Content MECLABS/MarketingSherpa Twitter: @DanielBurstein

• Adam Lapp, Associate Director, Optimization and Strategy MECLABS/MarketingSherpa Twitter: @AdamLapp

• Kirk Ramble, Optimization Consultant Webtrends Optimization Solutions

Any Questions?

#WTwebinar

Related Resources

• Get a free copy of Marketing Sherpa’s Landing Page Optimization Report

• Get the Webtrends whitepaper on Landing Page Optimization

• Contact Webtrends when you’re ready to chat: www.webtrends.com

North America:1-877-932-8736

Europe, Middle East, Africa:+44 (0) 1784 415 700

Australasia; Australia, New Zealand, South Pacific:+61 (0) 3 9935 2939

Save 25% with special code ENG13WEBINAR