THE TESTING CULTURE AT STUBHUB!

Bojan Ambrus!eCommerce Marketing Manager!

AGENDA!•  Introduction!•  Pre-culture change!

Our product roadmap!

•  Post culture change!Our new product roadmap and testing process!

•  Communicating change!•  Results, post culture change!

Lessons learnt!

INTRODUCTION!•  Who: StubHub!•  When: December 2011 UK

launch!•  What: A ticket marketplace

providing fans with a safe and convenient place to buy and an easy way to sell tickets!

•  Why test: Increase buy/sell flow conversion, revenue and engagement!

•  Test challenges: Perishable stock with a lack of control!

!

WHY CREATE A TESTING CULTURE?!

“THE TRUE METHOD OF

KNOWLEDGE!IS EXPERIMENT.” !

PRODUCT ROADMAP PRE CULTURE CHANGE!

Requirements and Planning • DEV, QA, DW

assigned •  Item is prioritised

against other projects

Build • Development activities

completed • Feature is released to

production

Measure • Metrics collected and

published • Look Back presented

Wishlist • Requirements

submitted • Project KPIs defined •  Item prioritised

Prioritised Backlog • Discovery team is

identified

Discovery • Usability, technical

feasibility and UX completed

• Analytics requirements and goals reviewed

Testing Process

TESTING PROCESS

"IF YOU CAN’T MEASURE IT, YOU CANNOT IMPROVE IT."!

PROCESS - HAVE DATA, NOT IDEAS!

WHERE DO GREAT TEST IDEAS COME FROM?!

Build a culture of test and learn through the use of...!Source Actions Analytics (Omniture, Google Analytics, etc.)

Compare KPIs, like conversion, RPV and exit rates across top trafficked pages and sections of the buy and sell flow.

Product roadmap Align testing to the product roadmap to validate direction and design for new features and functionality.

Qualitative research (UserTesting.com)

Use qualitative research including heat maps and online user testing.

Customer feedback (NPS, Foresee, Brand Awareness Tracker, CS Contacts)

Identify customer pain points throughout the business and develop tests to improve the experience through messaging, functionality or design.

Competitive reviews (Seatwave, Viagogo, GetMeIn!)

Compare your site to your competitors as well as the sites you use and admire to gain test ideas.

COMMUNICATING CHANGE!

"TELL ME AND I'LL FORGET.!SHOW ME AND I MIGHT REMEMBER.!

INVOLVE ME AND I WILL UNDERSTAND."!

COMMUNICATING CHANGE!•  Organise a working group!•  Request testing ideas!•  Share results!•  Inspire ideas for change!•  Create awareness!•  Demonstrate value!•  Outline testing process – have data, not

ideas!•  Make brainstorming part of that process!

RESULTS!

BUYER TRUST MODULE!

CTA BUTTONS!

VALUE PROPOSITION MESSAGING!

LESSONS LEARNT!

CREATING A TESTING CULTURE!

Top lessons learnt:!1.  Experiment early and often!2.  Question assumptions!3.  Remember that every visitor is an opportunity!4.  It's ok to be wrong!5.  Integrate into business objectives!

ANY QUESTIONS?!