ADVANCED A/B TESTINGBRUNO DILLEN

What you probably use today

LIMITATIONS OF A/B TESTING TOOLS

Novelty & primacy effect

Jumping to conclusions

Segmentation

Page templates

Page templates

Multi-page flow

Multiple goals

Then what?

Build your own!

BUILD YOUR OWN

How does it work1. Check if your visitor is a recurring visitor and has seen one of

your versions before (cookie)2. Decide on which A/B/x version to show to your visitor: if

recurring visitor check cookie versionif new visitor % split

3. Set cookie so this visitor will receive same version in later visits

4. Send version number to Google Analytics in custom dimension (or custom variable)

5. Display version according to our script 6. Set up custom segments in GA to analyze results

What are we testing?

The code

1. Check if your visitor is a recurring visitor and has seen one of your versions before (cookie)

2. Decide on which A/B/x version to show to your visitors: new visitor % split, recurring visitor coookie version

3. Set cookie so this visitor will receive same version in later visits

The code

4. Send version number to Google Analytics in custom dimension (or custom variable)

The code

5. Display version according to our script

The code

6. Set up custom segments in GA to track usage data

Result?

DECIDING ON WHAT TO TEST

Stop doing this...

... really?

+31%

+23%

+ 5%

Framework

Prioritisation model

StrategieRemarketingAdwordsCRMOn-siteSEOAffiliates

Prioritisation model

Where to look for inspiration?

Total page contribution = Unique Pageviews x Page Value

FAQ pages

SOME OF THE BEST CASES

AirBNB

AirBNB

AirBNB

(mobile) Navigation

Multi-page forms

Coolblue

Pinterest

02/05/2023 © Thes Days Y & R

A/B testing SEO

Example: title testing

version A: [product name] amigurumi pattern – Amigurumipatterns.netVersion B: Free [product name] amigurumi pattern

Results

Found project on Pinterest

A/B testing SEO: correct flow

1. Determine random division of pages from a certain template (database id even (=version A) / uneven (=version B) could be perfect).

2. Set up a custom dimension (or custom variable) in Google Analytics and send version A and B3. Let the website continue for at least a few weeks, so you can establish the baseline for your A

and B versions4. Implement the A/B test by updating title tags for version B only5. Wait and see if your custom dimension (version B) evolves differently to version A6. If version B sees increased landings through search engines, copy your best practice to

version A, wait, and see if you end up with an increase in version A as well

TAKEAWAYS

• Current A/B testing tools severely limit your options

• Improve your A/B testing significantly by creating your own tool

• Integrate with your central data source for analysis (Google Analytics)

• You can test pretty much anything: navigation, page templates, form funnels, social media clicks, SEO tactics, ...

• Keep testing to keep winning!

THANKS FOR YOUR ATTENTION

These Days Y&R Generaal Lemanstraat 47 box 2, 2018 Antwerp (Belgium) - T +32 (0) 3 286 44 50 - F +32 (0) 3 286 44 51 welcome@thesedays.com www.thesedays.com© These Days Y & R