3 Optimisation Decks : WAW Copenhagen - 27 Feb 2013
-
Upload
craig-sullivan -
Category
Technology
-
view
7.416 -
download
0
description
Transcript of 3 Optimisation Decks : WAW Copenhagen - 27 Feb 2013
#WAWCPHMeet @OptimiseOrDieReason for being : I Optimise StuffPast : John Lewis, Lovefilm, Belron, PwCNow : Director of Optimisation, Rush
Slideshare
: @OptimiseOrDie
: linkd.in/pvrg14
: slidesha.re/nlCDm6
"If you go to the men's washrooms at the Schiphol airport in Amsterdam, you may notice there's a fly in the urinals. It’s screen printed .
So what do you think most men do? That's right, they aim at the fly when they urinate.
They don't even think about it, and they don't need to read a user's manual; it's just an instinctive reaction that means 85% less spillage!
The interesting feature of these urinals is that they're deliberately designed to take advantage of this inherent human male tendency.“
This is my job.
Is there a way to fix this then?
24 Jan 2012
Three Presentations
#1 Conversion MethodologiesHow does CRO fit in with Lean UX, Agile, User Centred, Waterfall?
#3 Analytics Genius TipsTop tips from #Measurecamp + my 2013 roundup of cool stuff
#2 The Optimisers ToolkitThe best tools recommended by CRO practitioners
#1 Conversion Methodologies
What’s the problem?• #1 User Experience and Conversion
Optimisation are not a checkbox or a step in the process – it needs to be integrated in everything you do.
• #2 This work isn’t a one off exercise either – it’s an ongoing continuous improvement process – like Kaizen
• #3 Usability testing isn’t enough – other UX factors like the visceral and behavioural emotional responses we have to products need tuning too.
• #4 It’s not just about the user!• #5 It’s usually Self Centred Design
driven by Ego, Opinion, Assumption
Also, the dial won’t turn anymore
With thanks to @morys
PPC SEO
Why is this happening?• PPC changes• Advertising models flattening – i.e. mobile Google costs• Comprehensive SEO changes• Competition increasing• Fleetness of foot – Asos in Australia• Entry costs are lower now• New entrants compete without the cruft• Startups are using better ‘build and optimise’ methodologies
than nearly all corporates• The old way of doing things is going to die• The new way of doing things is your only survival ticket
So what do people do?• They throw tools at the problem• They try usability testing and research• They generate more data to look at• They make changes without measuring or testing• They hope to randomly create the optimal system• They get an expensive agency to help them• They push their team harder, like galley slaves• They experiment with riskier advertising models• They wonder why they’re burning rubber• They try more random things• Then they call a CRO person and say
“We’ve tried everything. It isn’t working. Help!”
Skinner’s Pigeon Experiment• Participants invited into a room with objects• Told to score 100 points within 30 minutes• Participants moved objects around, made
noise, jumped around, tried anything to make a counter increase the points score.
• They got horribly confused• They then created convincing lies for
themselves, to explain what they thought was working. They became superstitious and made rituals.
• The points allocation was made randomly by a goldfish, swimming back and forward in a tank.
• Know any marketing departments like this?• I’ve seen this a lot – and it paralyses companies• We need a better way. A methodology?
Lean UX
Positive– Lightweight and very fast methods– Realtime or rapid improvements– Documentation light, value high– Low on wastage and frippery– Fast time to market, then optimise– Allows you to pivot into new areas
Negative– Often needs user test feedback to
steer the development, as data not enough
– Bosses distrust stuff where the outcome isn’t known
“The application of UX design methods into product development, tailored to fit Build-Measure-Learn cycles.”
Agile UX / UCD / Collaborative Design
Positive– User centric– Goals met substantially– Rapid time to market (especially when
using Agile iterations)
Negative– Without quant data, user goals can
drive the show – missing the business sweet spot
– Some people find it hard to integrate with siloed teams
– Doesn’t’ work with waterfall IMHO
Wireframe
Prototype
TestAnalyse
Concept
Research
“An integration of User Experience Design and Agile* Software Development Methodologies”
*Sometimes
CRO
Lean Conversion Optimisation
Positive– A blend of several techniques– Multiple sources of Qual and Quant data aids triangulation– CRO analytics focus drives unearned value inside all
products
Negative– Needs a one team approach with a strong PM who is a
Polymath (Commercial, Analytics, UX, Technical)– Only works if your teams can take the pace – you might be
surprised though!
“A blend of User Experience Design, Agile PM, Rapid Lean UX Build-Measure-Learn cycles, triangulated data sources, triage and prioritisation.”
Lean CROInspection
Immersion
Identify
Triage & Triangulate
Outcome Streams
Measure
Learn
Instrument
Triage and Triangulation
• Starts with the analytics data• Then UX and user journey walkthrough from SERPS -> key paths• Then back to analytics data for a whole range of reports:• Segmented reporting, Traffic sources, Device viewport and
browser, Platform (tablet, mobile, desktop) and many more• We use other tools or insight sources to help form hypotheses• We triangulate with other data where possible• We estimate the potential uplift of fixing/improving something
as well as the difficulty (time/resource/complexity/risk)• A simple quadrant shows the value clusters• We then WORK the highest and easiest scores by…• Turning every opportunity spotted into an OUTCOME
“This is where the smarts of CRO are – in identifying the easiest stuff to test or fix that will drive the largest uplift.”
The Bucket Methodology“Helps you to stream actions from the insights and prioritisation work. Forces an action for every issue, a counter for every opportunity being lost.”
Test If there is an obvious opportunity to shift behaviour, expose insight or increase conversion – this bucket is where you place stuff for testing. If
you have traffic and leakage, this is the bucket for that issue.
InstrumentIf an issue is placed in this bucket, it means we need to beef up the
analytics reporting. This can involve fixing, adding or improving tag or event handling on the analytics configuration. We instrument both
structurally and for insight in the pain points we’ve found.
Hypothesise This is where we’ve found a page, widget or process that’s just not working well but we don’t see a clear single solution. Since we need to really shift the behaviour at this crux point, we’ll brainstorm hypotheses. Driven by
evidence and data, we’ll create test plans to find the answers to the questions and change the conversion or KPI figure in the desired direction.
Just Do It JFDI (Just Do It) – is a bucket for issues where a fix is easy to identify or the change is a no-brainer. Items marked with this flag can either be deployed in a batch or as part of a controlled test. Stuff in here requires low effort or are micro-opportunities to increase conversion and should be fixed.
Investigate You need to do some testing with particular devices or need more information to triangulate a problem you spotted. If an item is in this
bucket, you need to ask questions or do further digging.
How is it working out?• Methodologies are not Real Life ™• It’s mainly about the mindset of the team and
managers, not the tools or methodologies they play with
• Not all my clients have all the working parts• You should not be a methodology slave• Feel free to make your own or flexibly adapt• Use some, any techniques instead of ‘guessing’• Blending lean and agile with conversion optimisation
outcomes is my critical learning of the last 5 years• Doing rapid cycles of this outcome driven work for
Belron:• World Conversion Rate Increase:
2009 +5%, 2010 +10%, 2011 +15%, 2012 +25%
• If you’d like to develop a good one for your company, talk to me first!
• Don’t over complicate it.
Questions ?
#2 2013 Tips roundup!!!!!!!
Tips Roundup 2013
#1 Performance tune-ups#2 Browser money#3 Keyboard shortcuts#4 Ranking data#5 Content engagement#6 Enhanced In page#7 Duplicate transactions#8 Event tracking#9 Google API + more
#1 : Performance Tune-ups• With thanks to @Danbarker
• Add “_gaq.push(['_setSiteSpeedSampleRate', 100]);”• Amps up sampling for small & medium websites• Use the distribution report - % of pages < 3 seconds• DOM timings vital – let me explain• Avg. Document Content Loaded Time (sec) • This data is very accurate and helps conversion – pretty vital for
landing pages, where I find lots of stuff• Make yourself a pageview*content loaded time report• Work your way down from the top
• Mobile speed doesn’t count Safari – please be careful!• Read more at:
http://p.barker.dj/sitespeedtips
#2 : Browser money• Create a desktop only segment [Exclude mobile inc. tablet ]• Create a mobile only segment [Include Mobile (including tablet) & Exclude Tablet
containing yes]. Apply 3 segments desktop, tablet, mobile ONLY.• Now start segmenting like this:
Browser Segment Conv rateSafari Mobile Traffic 0.79%
Internet Explorer Desktop only 1.34%
Chrome Desktop only 1.01%
Safari Tablet Traffic 1.00%
Safari Desktop only 1.28%
Firefox Desktop only 1.20%
Android Browser Mobile Traffic 0.31%
Safari (in-app) Mobile Traffic 0.69%
Chrome Mobile Traffic 0.62%
Safari (in-app) Tablet Traffic 0.89%
Chrome Tablet Traffic 0.84%
Opera Desktop only 0.10%
Android Browser Tablet Traffic 0.71%
Mozilla Compatible Agent Mobile Traffic 0.51%
Quote the opportunity!
IE8 = 1.41% revenue
IE8 converts at 20% of IE9 and IE10
Some nasty bugs cause the problem
Worth fixing?
That 1.41 is worth nearly 6% more revenue
#2 : Browser money• You should see something like this:
#3 : GA Keyboard Shortcuts• Thanks to Farid Alhadi and @fastbloke
d t Set date range to TODAY
d y Set date range to YESTERDAY
d w Set date range to LAST WEEK
d m Set date range to LAST MONTH
d c Toggle date comparison mode (to the previous period of whatever you are looking at.Example, if you’re looking at 6 days, this will compare it to the 6 days before it)
d x Toggle date comparison mode (to the previous year of the period you are looking at)
? Open keyboard shortcut help
h Search help center
a open account panel
shift + a Go to account list
s / Search reports
shift + d Go to the default dashboard of the current profile
#4 : Ranking data• Someone searches on Google for ‘Term’• Clicks on a link to your site• What was the keyword rank for that term?• Reverse engineer the actual rankings from users machines• More accurate than some SEO tools (IMHO)• Two articles to read:
http://bit.ly/Vaisnohttp://bit.ly/13lmYF2
#5 : Content Engagement• Get a better bounce rate metric• See more detailed engagement metrics• Measure scrolling and reading activity• Came from @fastbloke but originally Justin Cutroni• Measure your scrolling and exit points from long form
content
• Very nice technique – read more at:
http://bit.ly/13lmYF2http://goo.gl/1AZZb
#6 : Enhanced Page Analyticsvar _gaq = _gaq || [];var pluginUrl ='//www.google-analytics.com/plugins/ga/inpage_linkid.js';_gaq.push(['_require', 'inpage_linkid', pluginUrl]);_gaq.push(['_setAccount', 'UA-XXXXXX-Y']);_gaq.push(['_trackPageview']);
• Correct link attribution for in-page Analytics• Very nice
#7 : Duplicate transactions in GA• Thanks to Matt Clarke and @timlb• Stop skewing the data with duplicates/reloads• Custom report to check if you’re affected
http://techpad.co.uk/content.php?sid=247• I’ve seen this a few places so worth checking,
particularly if figures don’t tally!• Read more at : http://bit.ly/13lmYF2
#8 : Event tracking• Thanks to #Measurecamp – check the stream• A beginners guide : http://bit.ly/13RFoJs• Some great ideas here : http://bit.ly/UCcptx• Don’t go for ‘Event Blizzard’• Focus on specific areas where insight is needed• Choose your naming structure carefully: http://
bit.ly/WJ4R4c• Read this complete guide : http://bit.ly/VmFSJ4
#9 : Google API and more• Use the Google API to get super custom reports• You can fetch different data types (on the fly as
well as pre-calculated)• Automate a lot of Excel work – more later…• @timlb recorded all the tips for #Measurecamp• We then had an unconference• He recorded a special session for us• Listen to great tips and more:
www.youtube.com/watch?v=JWXg1_4quwUwww.youtube.com/watch?v=z41Cdg9xbk0
• Roundup : www.measurecamp.org/aftermath/
• Which of these is most appealing?
#10 : Microdata
24 Jan 2012
#10 : Microdata – SERPS UX• Reviews – huge increases in CTR and Conversion• People• Products• Businesses and Organisations• Recipes• Events• Music• Local• Video
Helps to:• Dominate the page• Push other stuff down• Makes it more persuasive• The conversion journey starts here!
#11 : Measure viewport size• Thanks to @Beantin and others!• Measure the viewport size, not the resolution• Why?• Toolbars, chrome and setup varies• UK average is 2.2 toolbars installed• Code example here : http://bit.ly/4xaNYK• A common conversion issue• Your desk vs. Users = different• Turn off the wifi, reduce the viewport• Or keep the inside of toilet rolls
Questions?
#3 The Optimisers Toolkit
The Optimisers Toolkit
#1 Session Replay#2 Browser & Email testing#3 VOC & Feedback tools#4 Guerrilla Usability#5 Productivity tools#6 Split testing#7 Performance#8 Crowdsourcing#9 Analytics Love
#1 : Session Replay• 3 kinds of tool :
Client side • Normally Javascript based• Pros : Rich mouse and click data,
errors, forms analytics, UI interactions.• Cons : Dynamic content issue, Performance hit
Server side• Black Box -> Proxy, Sniffer, Port copying device• Pros : Gets all dynamic content, fast, legally tight• Cons : No client side interactions, Ajax, HTML5 etc.
Hybrid• Clientside and Sniffing with central data store
#1 : Session Replay• Vital for optimisers & fills in a ‘missing link’ for insight• Rich source of data on visitor experiences• Segment by browser, visitor type, behaviour, errors• Forms Analytics (when instrumented) are awesome• Can be used to optimise in real time!
Session replay tools• Clicktale (Client) www.clicktale.com• Mouseflow (Client) www.mouseflow.com• Ghostrec (Client) www.ghostrec.com• Tealeaf (Hybrid) www.tealeaf.com• Usabilla (Client) www.usabilla.com• UserReplay (Server) www.userreplay.com• SessionCam (Client) www.sessioncam.com
#2 : Feedback / VOC tools• Anything that allows immediate realtime onpage feedback• Comments on elements, pages and overall site & service• Can be used for behavioural triggered feedback• Tip! : Take the Call Centre for beers
• Kampylewww.kampyle.com
• Qualaroo www.qualaroo.com
• 4Q4q.iperceptions.com
• Usabillawww.usabilla.comAny more?
#3 – Testing tools
Email testing www.litmus.comwww.returnpath.comwww.lyris.com
Browser testing www.browsercam.comwww.crossbrowsertesting.comwww.cloudtesting.comwww.multibrowserviewer.comwww.saucelabs.com
Mobile devices www.perfectomobile.comwww.deviceanywhere.comwww.mobilexweb.com/emulatorswww.opendevicelab.com
#2a : Survey ToolsI’ve used• Surveymonkeywww.surveymonkey.com (1/5)• Zoomerang www.zoomerang.com (3/5)• SurveyGizmo www.surveygizmo.com (5/5)
• Read anything by and follow @cjforms• Any others?
#4 : Guerrilla Usability Testing• All you need is a device, time and people!• Use one of these tools for session recording:
Mediacam AV (cheap)www.netu2.co.uk
Camtasiawww.techsmith.com
Silverback (Mac)www.silverbackapp.com
Screenflow (Mac)www.telestream.net
UX Recorder (iOS)www.uxrecorder.com
#5 : Productivity tools
#5 Join.me
24 Jan 2012
#5 Pivotal Tracker
24 Jan 2012
• Seriously wasting your time doing manual Excel?• Fed up doing stuff that takes hours?• Use the Google API to roll your own reports straight into Big G• Lots of good articles but ask for advice from:
@danbarker@timlb#measurecamp
• Google Analytics + API + Google docs integration = A BETTER LIFE!• Hack your way to having more productive weeks• Learn how to do this and to have fun with GA custom reports• Ask me about the importance of training
#5 Google Docs
24 Jan 2012
#5 Conceptshare
24 Jan 2012
#5 Basecamp
24 Jan 2012
#6 – Split testing tools – Cheap!• Google Content Experiments
bit.ly/Ljg7Ds
• Multi Armed Bandit Explanationbit.ly/Xa80O8
• Optimizelywww.optimizely.com
• Visual Website Optimizerwww.visualwebsiteoptimizer.com
#7 Performance
• Google Site Speed• Webpagetest.org• Mobitest.akamai.org
Site Size Requests
www.coop.se 1200k 63m.ikea.com/se/sv/ 684k 14
High St Retailer 307k 43
Department Store 100k 18
Newspaper 195k 35
Supermarket 125k 14
Auto Sales 151k 47
Autoglass 25k 10
Remote tests : iPhone
Slides : slidesha.re/PDpTPD
Show your boss this split test:
Show the e-com director this one:
Som, feedbackRemote UX tools (P=Panel, S=Site recruited, B=Both)Usertesting (B) www.usertesting.comUserlytics (B) www.userlytics.comUserzoom (S) www.userzoom.comIntuition HQ (S) www.intuitionhq.comMechanical turk (S) www.mechanicalturk.comLoop11 (S) www.loop11.comOpen Hallway (S) www.openhallway.comWhat Users Do (P) www.whatusersdo.comFeedback army (P) www.feedbackarmy.comUser feel (P) www.userfeel.comEthnio (For Recruiting) www.ethnio.com
Feedback on Prototypes / MockupsPidoco www.pidoco.comVerify from Zurb www.verifyapp.comFive second test www.fivesecondtest.comConceptshare www.conceptshare.comUsabilla www.usabilla.com
#7 – UX and Crowd tools
#8 : Web Analytics Love• Properly instrumented analytics• Investment of 5-10% of developer time• Add more than you need• Events insights• Segmentation• Call tracking love!
#8 : Tap 2 Call trackingStep 1 : Add a unique phone number on ALL channels
(or insert your own dynamic number)
Step 2 : For phones, add “Tap to Call” or “Click to Call”
• Add event or PV trigger for any phone interaction• Incredibly useful keyword level call data• Very reliable data, easy & cheap to do• What did they do before calling?• Which page did they call you from?• What PPC or SEO keyword did they use?• What are you over or underbidding for?• Will help you shave 10, 20%+ off PPC• Which online marketing really sucks?
What about desktop?Step 1 : Add ‘Click to reveal’• Can be a link, button or a collapsed section• Add to your analytics software• This is a great budget option!
Step 2 : Invest in call analytics• Unique visitor tracking for desktop • Gives you that detailed marketing data• Easy to implement• Integrates with your web analytics• Let me explain…
safe
lit
safe
light
rep
air
phon
e nu
mbe
rca
r gl
ass
repa
irsa
felit
e w
inds
hiel
dw
inds
heild
rep
lace
men
tau
to g
lass
auto
win
dow
rep
lace
men
tau
to g
lass
rep
lace
men
tsa
fe li
ght
auto
gla
ss r
epai
rsa
felig
ht w
inds
hiel
dsa
felit
safe
lite
auto
gla
ssau
to w
indo
w r
epai
rau
to g
lass
spe
cial
ists
safe
lite
repa
irch
eap
win
dshi
eld
repl
acem
ent
safe
light
auto
glas
s re
plac
emen
tau
tolit
e w
inds
hiel
d re
pair
win
dshi
eld
repl
acem
ent p
rice
safe
lite
loca
tions
repl
acin
g w
inds
hiel
dsa
felit
e au
to g
lass
rep
air
win
dshi
eld
repa
irm
obile
win
dshi
eld
repa
irel
ite a
uto
glas
sre
plac
e w
inds
hiel
dsa
felit
e au
to g
lass
mn
safe
light
win
dow
rep
air
new
win
dshi
eld
cost
win
dshi
eld
crac
k re
pair
safe
ligh
t aut
o gl
ass
car
win
dow
rep
air
cost
auto
gla
ss r
epai
r ka
nsas
city
safe
ty li
teau
to g
lass
rep
air
hous
ton
txth
e gu
ys in
the
little
red
truc
ksa
felit
e pr
ices
0.0
5.0
10.0
15.0
20.0
25.0
Phone to Booking Ratio
Phone to Booking Ratio
What about desktop?
0800 222 1234
0800 222 1245
0800 222 1263
0800 222 1299
0800 222 1234
Telephony CloudWeb Analytics
Onbound Call analytics:
Wait timeAbandonCall lengthOperator outcomeSales valueRatings
A
B
C
Split testingPriority routingVIP teamsOn hold musicMenuMessagesCall recordingOffline promotions!Profit line!
Call centre
So what does phone tracking get you?• You can do it for free on your online channels• If you’ve got any phone sales or contact operation, this will
change the game for you• For the first time, analytics for PHONE for web to claim• Optimise your PPC spend• Track and Test stuff on phones, using web technology• The two best tests?• Talk to Freespee today!• Other countries, see the next slide
Who?Company Website Coverage
Mongoose Metrics* www.mongoosemetrics.com UK, USA, Canada
Ifbyphone* www.ifbyphone.com USA
TheCallR* www.thecallr.com USA, Canada, UK, IT, FR, BE, ES, NL
Call tracking metrics www.calltrackingmetrics.com USA
Hosted Numbers www.hostednumbers.com USA
Callcap www.callcap.com USA
Freespee* www.freespee.com UK, SE, FI, NO, DK, LT, PL, IE, CZ, SI, AT, NL, DE
Adinsight* www.adinsight.co.uk UK
Infinity tracking* www.infinity-tracking.com UK
Optilead* www.optilead.co.uk UK
Switchboard free www.switchboardfree.co.uk UK
Freshegg www.freshegg.co.uk UKAvanser www.avanser.com.au AUS
Jet Interactive* www.jetinteractive.com.au AUS* I read up on these or talked to them. These are my picks.
#9 : Web Analytics Love
• UX of web analytics tools and reports• People, Process, Human problems• Make the UI force decisions!• Playability and exploration• Skunkworks project time (5-10%)• Give it love, time, money and iteration• Spend to automate, get more time
• Optimisation can be used for evil!• Either by accident (just stupid)• Or by design, to trick people
wiki.darkpatterns.org/wiki/Home
Evil CRO
So how do I become an Optimiser?• It’s a state of mind, not a course!• You need to read lots, test loads and watch customers even more• Ask someone to mentor you• Go to conferences, events, find people online• Don’t say “I think” – say “I’ll find out” or “I’ll test that”• Volunteer for shit jobs and do them well• Be the glue between people – volunteer for cross team work• Fix bad things so your team loves you• Always connect others with outcomes• Hack your job to a better life
• Make yourself more valuable, not less• Challenge fucking everything• Be annoying and never give up!
Don’t give up!• Start with the customer at the heart of everything• It’s liberating!• Use low budget solutions, cheap or free tools to get business cases
made or gain insight• Money talks so even micro testing will illustrate ROI• Invest in people who care about customers, as well as conversion• Invest in analytics talent and instrumentation • If you can make a 10% shift in a key metric (CTR, conversion), just
from smart thinking, then what would a team of 5 get you?• If you want to hire people, ask me for advice!
“We always had a plan for
pulling out”
“We always had a plan for
pulling out”
Slideshare
: @OptimiseOrDie
: linkd.in/pvrg14
: slidesha.re/nlCDm6
More reading. Slides and resources on slideshare.net