Vab Dwivedi - Real World E-Commerce and Customer Experience Optimization Practices from Dell.com

Driving Customer Experience

Through Testing, Research &

Analytics

Vab Dwivedi

Dell | Director, Testing & Research

@vabdwivedi

linkedin.com/in/vabdwivedi

2 of Y

What’s our approach?

3 of Y

Our tenets

We Are Customer Obsessed1

Act as Subject 2

Deliver 3

Guide 4

Serve 5

Be Agile6

4 of Y

Our tenets


Act as Subject 2

Deliver 3

Guide 4

Serve 5

Be Agile6

5 of Y

Our tenets


Act as Subject Matter Experts2

Deliver 3

Guide 4

Serve 5

Be Agile6

6 of Y

Our tenets


Act as Subject 2

Deliver Advanced Analytics In A Consumable Manner3

Guide 4

Serve 5

Be Agile6

7 of Y

Our tenets


Act as Subject 2

Deliver 3

Guide Decision Making Through Insights And Experimentation4

Serve 5

Be Agile6

8 of Y

Our tenets


Act as Subject 2

Deliver 3

Guide 4

Serve As A Channel of Knowledge5

Be Agile6

9 of Y

Our tenets


Act as Subject 2

Deliver 3

Guide 4

Serve 5

Be Agile6

10 of Y

Our tenets


Act as Subject 2

Deliver 3

Guide 4

Serve 5

Be Agile6

11 of Y

Holding ourselves accountable

Influencing Agenda/Roadmap1

Delivering Actionable Insights2

Delighting 3

Managing Stakeholders4

Executing With Operational Excellence5

Driving An Innovative Vision6

12 of Y




Delighting 3




13 of Y




Delighting 3




14 of Y




Delighting Our Customers3




15 of Y




Delighting 3




16 of Y




Delighting 3




17 of Y




Delighting 3




18 of Y




Delighting 3




Setting our team up for success

20 of Y

21 of Y

22 of Y

23 of Y

24 of Y

That was ‘how’…let’s look at some projects…

Dell.com Home Page

26 of Y

27 of Y

28 of Y

29 of Y

30 of Y

Iterative testing over 18 months…

31 of Y

Iterative testing over 18 months…

32 of Y

We measured EVERYTHING!

33 of Y


34 of Y


35 of Y


36 of Y


37 of Y


38 of Y

…And then some more…

39 of Y


40 of Y


41 of Y


42 of Y


43 of Y

…Look into device type performance…

44 of Y


45 of Y


46 of Y


Drive Big Change:A new Dell.com

48 of Y

49 of Y

50 of Y

Not just a new visual identity…

51 of Y


The new Dell.com was new in every way….platform,

architecture, back end services

52 of Y




Before going live, test it to ensure a win for customers

and the business

53 of Y





and the business

Expecting a quick win….

54 of Y





and the business

Expecting a quick win….

… -33% revenue/visit

55 of Y

OK, no big deal…we know what to do

56 of Y


Let’s introduce UI enhancements

57 of Y



24 of them…through various parts in the site

58 of Y




Recipe C will be our super recipe that will bridge the

gap when we re-run the test

59 of Y




Recipe C will be our super recipe that will bridge the

gap when we re-run the test

… -45% revenue/visit

60 of Y

Maybe its not the UI…

61 of Y


It is a new platform…

62 of Y


It is a new platform…

We need re-run and this time look at the metrics that

can point to any issues with the platform

63 of Y

Performance metrics

64 of Y

Performance metrics

65 of Y

Performance metrics

66 of Y

Performance metrics

67 of Y

Performance metrics

68 of Y

Performance metrics

69 of Y

Performance metrics

70 of Y

Well that explains it…

71 of Y


Now we can work with engineering and provide

guidance on where the issues are

72 of Y




Emergency sprints scheduled to address live issues

73 of Y





Re-test now….

74 of Y





Re-test now….

… +3% revenue/visit!!!

Find something small that can make an impact: Continue shopping

76 of Y

Should ‘Continue Shopping’ be smarter?

77 of Y


78 of Y


79 of Y


80 of Y

Did it work?

81 of Y

Did it work?

82 of Y

Are you sure?

83 of Y

Are you sure?

84 of Y

Are you sure?

85 of Y

Are you sure?

86 of Y

Now iterate, globalize and future proof

87 of Y


88 of Y


89 of Y


90 of Y


91 of Y


92 of Y


93 of Y


Rapid fire: Everything doesn’t have to be a ‘project’…

95 of Y

Cycling through order codes

vs

96 of Y

Cycling through order codes

vs+9.1% Conversion

+11.9% Revenue/Visit

+91.6% Button Engagement

-1.49% Exit rates

+.6% Compare usage

97 of Y

Adding payment methods to cart

vs

98 of Y

Adding payment methods to cart

vs+4.3% Conversion


+2.2% Payment Pg > Confirm Page

99 of Y

Results sorting method

vs

100 of Y

Results sorting method

vs

+6.7% Conversion

+15% Revenue/Visit

+51% Conversion for highest price

band

-64% Engagement with sort option

101 of Y

Category page filter layout

vs

102 of Y

Category page filter layout

vs-7.5% Conversion

-6.4% Revenue/Visit

-40% Engagement

-17% Conversion for

engagers

-8% Next page

103 of Y

Sequencing options for systemsBased on business rules

and availabilityHardware, Software,

Services, Misc.

vs

104 of Y

Sequencing options for systemsBased on business rules

and availabilityHardware, Software,

Services, Misc.

vs+13.8% Conversion


+12.7% Margin/Visit

105 of Y

Clickable tabs and ‘Add to Cart’ CTA

vs

106 of Y

Clickable tabs and ‘Add to Cart’ CTA

vs+11% Conversion

+52% Engagement w/tabs & CTAs

+13% Movement through funnel

-2% Exit rate

107 of Y

Sign in page titles

vs

108 of Y

Sign in page titles

vs

+3% Conversion

-4% Guest checkout

+3.7% Movement through funnel

+1% Authentication

109 of Y

Navigation title change

vs

110 of Y

Navigation title change

-3.5% Conversion

-5.8% Revenue/Visit

-3% Engagement

No Change Movement to

browse

vs

If you only remember three things from today…

112 of Y

Our daily mantra

113 of Y

Our daily mantra

You want iterative testing & a LOT of data1

Winners and Losers are good2

Keep it simple for your customer3

114 of Y

Driving Customer Experience

Through Testing, Research &

Analytics

Vab Dwivedi

Dell | Director, Testing & Research

@vabdwivedi

linkedin.com/in/vabdwivedi

Vab Dwivedi - Real World E-Commerce and Customer Experience Optimization Practices from Dell.com

Technology

Transcript of Vab Dwivedi - Real World E-Commerce and Customer Experience Optimization Practices from Dell.com