Ton Wesseling Which Test Won London 2014 Keynote #tle2014 A/B-testing

Post on 08-May-2015

2.277 views 0 download

description

The slides of my conversion rate optimization talk on how we won so many Which Test Won awards - data & psychology - I've showed 25 cases in this presentation - thanks for being there and/or reading the slides now. If you're not able to get the PDF version - hit me on twitter http://twitter.com/tonw , I'll follow back and DM the slides! The wheel register code is "boston" (all small), not "Boston"

Transcript of Ton Wesseling Which Test Won London 2014 Keynote #tle2014 A/B-testing

London  

Ton Wesseling Online Dialogue

Chief Optimization Officer

#TLE2014 The Live Event @TonW

Learn  how  to  win  10+  WhichTestWon  A/B-­‐Test  Awards  

N o w w i t h f r e e c o n t e n t i n s i d e :

ü  2 5 c o n v e r s i o n c a s e s !

O N L I N E D I A L O G U E G u i d i n g y o u t o d i g i t a l m a t u r i t y

3 years in a row N u m b e r 1 a g e n c y i n t h e N e t h e r l a n d s

i n o u r fi e l d o f p r a c t i c e ( a n a l y t i c s & c o n v e r s i o n )

#TLE2014 The Live Event @TonW ©2014,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.      

Univé:  top  3  car  insurance  in  the  Netherlands  

Start  cases  

“Foot in the door” Get your visitors to interact asap

+23% sales

#TLE2014 The Live Event @TonW ©2014,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.      

Van  der  Valk  Hotels  (large  Dutch  hotel  chain)  

Start  cases  

“Chunking” Divide content in diminishing chunks of 3 or 5 pieces of information

+14% sales

#TLE2014 The Live Event @TonW ©2014,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.      

Online  Dialogue:  Conversion  Summit  landingpage  

Start  cases  

+29% interactions

Data Driven

Data Driven

Data Driven

Data Driven

Data Driven

Data Driven

Data Driven Data Driven

Data Driven

Data Driven

Data Driven

Data Driven

Data Driven Data Driven

Data Driven

Data Driven

Data Driven

Data Driven

Data Driven @ To n W

Data Driven

Persuasion Psychologist

@ B a r t S

Persuasion Psychologist

Persuasion Psychologist

Persuasion Psychologist

Persuasion Psychologist

Persuasion Psychologist

Persuasion Psychologist

Persuasion Psychologist

Persuasion Psychologist Persuasion Psychologist

Persuasion Psychologist

Persuasion Psychologist

Persuasion Psychologist

Persuasion Psychologist

Data Driven Persuasion Psychology

= C R O M a g i c

+

#TLE2014 The Live Event @TonW ©2014,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.      

Kras.nl  (TUI):  homepage  effect  on  sales  

Award  cases  

“Ambiguity Aversion” Make it crystal clear what your customer will get

+20% sales

#TLE2014 The Live Event @TonW ©2014,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.      

fonQ.nl:  Ecommerce  –  site  search  sales  improvement  

Award  cases  

“Carrot & Stick” Keep showing customers what they’re buying through the whole process

+49% sales

#TLE2014 The Live Event @TonW ©2014,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.      

Freo.nl:  liN  in  loan  applicaOons    

Award  cases  

“Sequencing” CRO is a reciprocal dialogue: Take the concepts that are active in your customers brain as your starting point

+19% applications

W h y i s t h i s w o r k i n g ?

W W W = Wo r l d W i d e Wa l h a l l a "f o r

A n a l y t i c s & B r a i n Te s t i n g

b r a i n s … B e c a u s e : y o u a re c o n v e r t i n g

W h a t i n fl u e n c e s t h a t b r a i n ?

b r a i n s … Yo u a re c o n v e r t i n g

W h a t i n fl u e n c e s t h a t b r a i n ?

b r a i n s … Yo u a re c o n v e r t i n g

D a t a D r i v e n P e r s u a s i o n P s y c h o l o g y

C re a t i n g c o n v e r s i o n b o o s t i n g & w i n n i n g A / B t e s t p l a n s =

#TLE2014 The Live Event @TonW ©2014,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.      

This  is  the  method  we  use  

Fact  &  Act  

&  

                   Tell              Combine                  Analyze  

                   Test                  Create                  Analyze              Filter  

#TLE2014 The Live Event @TonW ©2014,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.      

Let’s  start:  Filter  &  Analyze  

Fact  &  Act  

&  

                   Tell              Combine                  Analyze  

                   Test                  Create                  Analyze              Filter  

#TLE2014 The Live Event @TonW ©2014,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.      

Freo.nl:  more  liN  in  loan  applicaOons    

Filter  &  Analyze  

“The paradox of Choice” Offer a minimum of 2 and a maximum of 5 choices

+14% applications

#TLE2014 The Live Event @TonW ©2014,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.      

MoneYou.nl:  upliN  in  new  saving  accounts  

Filter  &  Analyze  

“In Gaze Placement” Put your most persuasive content in your customers’ gaze

+23% accounts

#TLE2014 The Live Event @TonW ©2014,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.      

Freo.nl:  survey  to  understand  visitor  needs  

Filter  &  Analyze  

“Foot between the lips” Directly get a forced click

Significant more applications Survey inv i ta t ion vs . no survey inv i ta t ion

winner

#TLE2014 The Live Event @TonW ©2014,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.      

And  now:  Create  &  Test  

Fact  &  Act  

&  

                   Tell              Combine                  Analyze  

                   Test                  Create                  Analyze              Filter  

#TLE2014 The Live Event @TonW ©2014,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.      

Autonomy:  Van  der  Valk  conversion  liN  

Create  &  Test  

Autonomy:  “Let  your  customer  think  he’s  free  in  his  choices”  

+25% sales

#TLE2014 The Live Event @TonW ©2014,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.      

But..  :  VakanOeveilingen.nl  (aucOon  website)  

Create  &  Test  

Urgency:  “When  we’re  in  a  hurry,  we  love  to  be  guided”  

Significant more submits, but..

Less registrations

#TLE2014 The Live Event @TonW ©2014,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.      

Direct  Priming:  our  newsleYer  headline  test  

Create  &  Test  

Direct  Priming:  “RepePPon  leads  to    quicker  reacPons”  

(Title first article)

(Title last article)

+42% clicks Outgoing clicks from within the newsletter by

members who opened the newsletter

#TLE2014 The Live Event @TonW ©2014,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.      

Choice  Paradox:  MoneYou  Call-­‐to-­‐AcOon  test  

Create  &  Test  

Choice  Paradox:  “We  love  a  few,  but  not  to  many  opPons”  

+16% accounts + 2 5 4 % c l i c k s ( o n C TA a r e a )

#TLE2014 The Live Event @TonW ©2014,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.      

Response  efficacy:  Kras.nl  travel  booking  form  

Create  &  Test  

Response  efficacy:  “We’re  more  likely  to  act  when  we  believe  it  will  actually  have  the  desired  effect”  

+18% sales

#TLE2014 The Live Event @TonW ©2014,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.      

Ambiguity  aversion:  Freo.nl  applicaOon  form  

Create  &  Test  

Ambiguity  Aversion:  “We  prefer  opPons  that  are  certain”  

+10% applications

#TLE2014 The Live Event @TonW ©2014,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.      

Visuel  cue:  Hotel  Schiphol  –scroll  promoOon  

Create  &  Test  

Visual  Cueing:  “Our  aTenPon  is  very  easily  influenced”  

-23% bouncerate + significant sales

#TLE2014 The Live Event @TonW ©2014,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.      

Framing:  MoneYou  email  &  landingpage  test  

Create  &  Test  

Equivalence  Framing:  “The  way  things  are  stated  or  portrayed,  highly  influences  our  choices”  

Significant more account opens

#TLE2014 The Live Event @TonW ©2014,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.      

Conceptual  Contrast:  job  posOng  offers  

Create  &  Test  

Ugly  Brother:  “The  way  things  are  stated  or  portrayed,  highly  influences  our  choices”  

Almost everyone chooses the maximum package

#TLE2014 The Live Event @TonW ©2014,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.      

Test  done?  Analyze!  

Fact  &  Act  

&  

                   Tell              Combine                  Analyze  

                   Test                  Create                  Analyze              Filter  

#TLE2014 The Live Event @TonW ©2014,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.      

Kras.nl:  from  most  sold  to  a  last  minute  list  

Analyze  

+58% conversions B u t n o s i g n i fi c a n t t u r n o v e r d i f f e r e n c e

#TLE2014 The Live Event @TonW ©2014,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.      

Banking:  what  is  your  interest  rate  posiOon?  

Analyze  

PosiOon  targeOng:  “We  like  basing  comparisons  on  just  a  few  aTributes”  

Winner when

number 1

Winner when not number 1

#TLE2014 The Live Event @TonW ©2014,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.      

Product  pages:  CTA  color?  

Analyze  

Goal-­‐directed  behavior  “We  need  other  persuasion  techniques  when  we’re  ‘geXng  things  done’  than  when  we  (want  to)  relax  

Monday winner Saturday winner

#TLE2014 The Live Event @TonW ©2014,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.      

Kras.nl:  new  &  returning  visitor  differences  

Analyze  

Returning visitor winner

New visitor winner

#TLE2014 The Live Event @TonW ©2014,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.      

Essent  Belgium:  landingpage  test  

Analyze  

Returning visitor winner

New visitor winner

#TLE2014 The Live Event @TonW ©2014,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.      

MoneYou  landing  page  redesign  

Analyze  

+27% applications B u t … ( s e e n e x t s l i d e )

#TLE2014 The Live Event @TonW ©2014,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.      

Let’s  dive  in  the  MoneYou  A/B-­‐test  data  

Analyze  

Cum. Conversion % per day" Difference in cum. Conversion %"Conversion difference

stabelizes"

Conversion difference stabelizes"

Why is the conversion percentage so high on

these 2 days?"

Why is the difference between A and B at first

negative and than positive?"

Hmm, euroclix.nl

#TLE2014 The Live Event @TonW ©2014,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.      

Affiliate  partner  delivering  external  moOvaOon  

Analyze  

Free money if you file an application

#TLE2014 The Live Event @TonW ©2014,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.      

MoneYou.nl  without  visitor  funding  affiliates  

Analyze  

Cum. Conversion % per day" Difference in cum. Conversion %"

Very stable differences"

(with conversions)"

Much shorter, difference in first days is caused by returning

visitors (which we filtered out after this analysis)"

+100% applications!

#TLE2014 The Live Event @TonW ©2014,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.      

Combine  &  Tell  

Fact  &  Act  

&  

                   Tell              Combine                  Analyze  

                   Test                  Create                  Analyze              Filter  

#TLE2014 The Live Event @TonW ©2014,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.      

Combine  &  Tell  

!"#$%&'("#)*+,-(./,"#) !"#$%&'(%)%"#*+'

CRO  becomes  Customer  Intelligence  

#TLE2014 The Live Event @TonW ©2014,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.      

Telegraaf.nl:  biggest  Dutch  newspaper  first  cookie  concept:  almost  100%  opt-­‐in  

Combine  &  Tell  

Defaults:  “We  tend  to  like  what  is  chosen  for  us”  

#TLE2014 The Live Event @TonW ©2014,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.      

Telegraaf.nl:  biggest  Dutch  newspaper  first  cookie  concept:  almost  100%  opt-­‐in  

Combine  &  Tell  

Reinforcements:  “We  want  to  hear  we’re  doing  good”  

#TLE2014 The Live Event @TonW ©2014,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.      

Based  on  customer  intelligence:  we  loved  to  be  challenged  

Combine  &  Tell  

We won J

#TLE2014 The Live Event @TonW ©2014,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.      

We  help  and  /  or  educate  Kras.nl:  They  won!  

Combine  &  Tell  

#TLE2014 The Live Event @TonW ©2014,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.      

We  teach:  Master  of  online  Persuasion  

Combine  &  Tell  

#TLE2014 The Live Event @TonW ©2014,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.      

Ever  growing  our  customer  knowledge  

3  years  of  being  a  partner  

ü   MoneYou  groei  over  jaren  heen  

#TLE2014 The Live Event @TonW ©2014,  WhichTestWon.com  a  division  of  Anne  Holland  Ventures  Inc.    All  rights  reserved.      

Just  do  it  

Digital Data Persuasion Psychology

How  to  create  winning  A/B  test  plans?  

F a c t & A c t

London  

Ton Wesseling Online Dialogue ton@onlinedialogue.com +3130 4100 170

#TLE2014 The Live Event @TonW

Contact  info  

one more th ing…

…we’ve loca ted ove r 200 pe rsuas ion techn iques…

w w w. W h e e l o f P e r s u a s i o n . c o m

w w w. W h e e l o f P e r s u a s i o n . c o m

T h e W h e e l o f P e r s u a s i o n

Temporary Access Code: “Boston” (valid until Tuesday / June 3th )

#TLE2014 The Live Event @TonW