Building a Company Wide Culture of Experimentation - Conversion Conference Las Vegas 2016
-
Upload
sean-ellis -
Category
Marketing
-
view
5.328 -
download
0
Transcript of Building a Company Wide Culture of Experimentation - Conversion Conference Las Vegas 2016
![Page 1: Building a Company Wide Culture of Experimentation - Conversion Conference Las Vegas 2016](https://reader034.fdocuments.in/reader034/viewer/2022052418/586fdddf1a28ab18428b69d3/html5/thumbnails/1.jpg)
Building a Company-Wide Culture of Experimentation
May 18, 2016Founder & CEO GrowthHackers
![Page 2: Building a Company Wide Culture of Experimentation - Conversion Conference Las Vegas 2016](https://reader034.fdocuments.in/reader034/viewer/2022052418/586fdddf1a28ab18428b69d3/html5/thumbnails/2.jpg)
Sept 2008: Dropbox Blank Slate
• Excellent product coming out of Beta
• Team of 7 engineers
• Skeptical of marketing
• Goal: Create culture of growth experimentation
#ConvCon
![Page 3: Building a Company Wide Culture of Experimentation - Conversion Conference Las Vegas 2016](https://reader034.fdocuments.in/reader034/viewer/2022052418/586fdddf1a28ab18428b69d3/html5/thumbnails/3.jpg)
Seeds of Testing Culture
• Built tracking and testing infrastructure
• Shared insights (qual and quant) with team
• Shared testing wins with team
• Encouraged broader participation
#ConvCon
![Page 4: Building a Company Wide Culture of Experimentation - Conversion Conference Las Vegas 2016](https://reader034.fdocuments.in/reader034/viewer/2022052418/586fdddf1a28ab18428b69d3/html5/thumbnails/4.jpg)
Today Dropbox has ½ Billion Users
• No marketer from April 2009 – January 2010
• Product/engineers continued to experiment
#ConvCon
![Page 5: Building a Company Wide Culture of Experimentation - Conversion Conference Las Vegas 2016](https://reader034.fdocuments.in/reader034/viewer/2022052418/586fdddf1a28ab18428b69d3/html5/thumbnails/5.jpg)
Growth is Company Wide Effort
Product
External Channels
Promise
1st Experience
Ongoing Experience
Acquisition
Activation
Revenue
Retention
Referral
Marketing PR
SupportSuccess
biz dev
CRO or Growth Team
#ConvCon
![Page 6: Building a Company Wide Culture of Experimentation - Conversion Conference Las Vegas 2016](https://reader034.fdocuments.in/reader034/viewer/2022052418/586fdddf1a28ab18428b69d3/html5/thumbnails/6.jpg)
AKA Growth Hacking
Rapid experimentation across the full funnel to learn the most effective ways
to scale sustainable customer adoption.
#ConvCon
![Page 7: Building a Company Wide Culture of Experimentation - Conversion Conference Las Vegas 2016](https://reader034.fdocuments.in/reader034/viewer/2022052418/586fdddf1a28ab18428b69d3/html5/thumbnails/7.jpg)
Testing Drives Growth
• No certainty about what works
• Rapid testing across all levers
• More testing = More learning
AcquisitionReferralActivationRevenueRetentionResurrection
#ConvCon
![Page 8: Building a Company Wide Culture of Experimentation - Conversion Conference Las Vegas 2016](https://reader034.fdocuments.in/reader034/viewer/2022052418/586fdddf1a28ab18428b69d3/html5/thumbnails/8.jpg)
Twitter Example
#ConvCon
![Page 9: Building a Company Wide Culture of Experimentation - Conversion Conference Las Vegas 2016](https://reader034.fdocuments.in/reader034/viewer/2022052418/586fdddf1a28ab18428b69d3/html5/thumbnails/9.jpg)
Change Harder Established Company
• Day onsite at Microsoft (May 2016)
• Goal: Transform to culture of growth experimentation
• Founded 1975, 118K employees
• Results: TBD
#ConvCon
![Page 10: Building a Company Wide Culture of Experimentation - Conversion Conference Las Vegas 2016](https://reader034.fdocuments.in/reader034/viewer/2022052418/586fdddf1a28ab18428b69d3/html5/thumbnails/10.jpg)
Steps to Experimentation Culture
#ConvCon
![Page 11: Building a Company Wide Culture of Experimentation - Conversion Conference Las Vegas 2016](https://reader034.fdocuments.in/reader034/viewer/2022052418/586fdddf1a28ab18428b69d3/html5/thumbnails/11.jpg)
Process First
#ConvCon
![Page 12: Building a Company Wide Culture of Experimentation - Conversion Conference Las Vegas 2016](https://reader034.fdocuments.in/reader034/viewer/2022052418/586fdddf1a28ab18428b69d3/html5/thumbnails/12.jpg)
Effective Process
#ConvCon
![Page 13: Building a Company Wide Culture of Experimentation - Conversion Conference Las Vegas 2016](https://reader034.fdocuments.in/reader034/viewer/2022052418/586fdddf1a28ab18428b69d3/html5/thumbnails/13.jpg)
Core Team to Manage Process
• Start with product manager
• Add dedicated people to core as needed to hit testing tempo goals
• Core often includes designers, growth engineers, analysts…
#ConvCon
![Page 14: Building a Company Wide Culture of Experimentation - Conversion Conference Las Vegas 2016](https://reader034.fdocuments.in/reader034/viewer/2022052418/586fdddf1a28ab18428b69d3/html5/thumbnails/14.jpg)
Effective Process
#ConvCon
![Page 15: Building a Company Wide Culture of Experimentation - Conversion Conference Las Vegas 2016](https://reader034.fdocuments.in/reader034/viewer/2022052418/586fdddf1a28ab18428b69d3/html5/thumbnails/15.jpg)
Goal: Build Large Backlog of Ideas
• Build large backlog of ideas
#ConvCon
![Page 16: Building a Company Wide Culture of Experimentation - Conversion Conference Las Vegas 2016](https://reader034.fdocuments.in/reader034/viewer/2022052418/586fdddf1a28ab18428b69d3/html5/thumbnails/16.jpg)
Prioritize Backlog
#ConvCon
![Page 17: Building a Company Wide Culture of Experimentation - Conversion Conference Las Vegas 2016](https://reader034.fdocuments.in/reader034/viewer/2022052418/586fdddf1a28ab18428b69d3/html5/thumbnails/17.jpg)
Select Tests in Weekly Meeting
• Use system for prioritization (we use ICE)• Choose tests to launch (no brainstorming)
IMPACT CONFIDENCE EASE (1 – 10) (1 – 10)
(1 – 10)
#ConvCon
![Page 18: Building a Company Wide Culture of Experimentation - Conversion Conference Las Vegas 2016](https://reader034.fdocuments.in/reader034/viewer/2022052418/586fdddf1a28ab18428b69d3/html5/thumbnails/18.jpg)
Launch Tests
#ConvCon
![Page 19: Building a Company Wide Culture of Experimentation - Conversion Conference Las Vegas 2016](https://reader034.fdocuments.in/reader034/viewer/2022052418/586fdddf1a28ab18428b69d3/html5/thumbnails/19.jpg)
Launch Tests
• Assign a project manager to each test• Use tools and allocate skills needed to implement
#ConvCon
![Page 20: Building a Company Wide Culture of Experimentation - Conversion Conference Las Vegas 2016](https://reader034.fdocuments.in/reader034/viewer/2022052418/586fdddf1a28ab18428b69d3/html5/thumbnails/20.jpg)
Analyze Results
#ConvCon
![Page 21: Building a Company Wide Culture of Experimentation - Conversion Conference Las Vegas 2016](https://reader034.fdocuments.in/reader034/viewer/2022052418/586fdddf1a28ab18428b69d3/html5/thumbnails/21.jpg)
Analyze Results
• Ensure tracking up front for hypothesis
• Run sufficient time
• Triage analysis by hypothesis first
#ConvCon
![Page 22: Building a Company Wide Culture of Experimentation - Conversion Conference Las Vegas 2016](https://reader034.fdocuments.in/reader034/viewer/2022052418/586fdddf1a28ab18428b69d3/html5/thumbnails/22.jpg)
Example Weekly Meeting Agenda
15 min: KPI review & update focus area
10 min: Review last week’s testing sprint
15 min: Key lessons learned from analyzed tests
15 min: Select tests for this week’s sprint
5 min: Check growth of idea backlog
#ConvCon
![Page 23: Building a Company Wide Culture of Experimentation - Conversion Conference Las Vegas 2016](https://reader034.fdocuments.in/reader034/viewer/2022052418/586fdddf1a28ab18428b69d3/html5/thumbnails/23.jpg)
Requires Transparent Learning
#ConvCon
![Page 24: Building a Company Wide Culture of Experimentation - Conversion Conference Las Vegas 2016](https://reader034.fdocuments.in/reader034/viewer/2022052418/586fdddf1a28ab18428b69d3/html5/thumbnails/24.jpg)
Transparent Learning
• Every test leads to learning• Proactively share with idea originator• Share wins with opt in email list• Organize learning for everyone’s easy access
#ConvCon
![Page 25: Building a Company Wide Culture of Experimentation - Conversion Conference Las Vegas 2016](https://reader034.fdocuments.in/reader034/viewer/2022052418/586fdddf1a28ab18428b69d3/html5/thumbnails/25.jpg)
Drive Broad Participation
#ConvCon
![Page 26: Building a Company Wide Culture of Experimentation - Conversion Conference Las Vegas 2016](https://reader034.fdocuments.in/reader034/viewer/2022052418/586fdddf1a28ab18428b69d3/html5/thumbnails/26.jpg)
Company Kickoff
• CEO: “everyone seek opportunities for improvement and growth”
• Share recent wins and insights
• Idea input training
• Pizza and beer brainstorm
#ConvCon
![Page 27: Building a Company Wide Culture of Experimentation - Conversion Conference Las Vegas 2016](https://reader034.fdocuments.in/reader034/viewer/2022052418/586fdddf1a28ab18428b69d3/html5/thumbnails/27.jpg)
Collect Ideas Widely
External Experts
Internal Experts
Ideas Tests Results
Core Growth Team Drives Process
#ConvCon
![Page 28: Building a Company Wide Culture of Experimentation - Conversion Conference Las Vegas 2016](https://reader034.fdocuments.in/reader034/viewer/2022052418/586fdddf1a28ab18428b69d3/html5/thumbnails/28.jpg)
Standard Experiment Doc for Ideas
• Include research, hypothesis, target lever
#ConvCon
![Page 29: Building a Company Wide Culture of Experimentation - Conversion Conference Las Vegas 2016](https://reader034.fdocuments.in/reader034/viewer/2022052418/586fdddf1a28ab18428b69d3/html5/thumbnails/29.jpg)
Provide Feedback on Ideas
• Acknowledge all ideas
• Praise favorite ideas
• Notify if selected
• Report back results
#ConvCon
![Page 30: Building a Company Wide Culture of Experimentation - Conversion Conference Las Vegas 2016](https://reader034.fdocuments.in/reader034/viewer/2022052418/586fdddf1a28ab18428b69d3/html5/thumbnails/30.jpg)
Keep it Fun
#ConvCon
![Page 31: Building a Company Wide Culture of Experimentation - Conversion Conference Las Vegas 2016](https://reader034.fdocuments.in/reader034/viewer/2022052418/586fdddf1a28ab18428b69d3/html5/thumbnails/31.jpg)
Idea from Engineer - oEmbed
• Data: Embeds = longer sessions
• Hypothesis: Same action = more embeds
• Result: 41% increase in embedded media
Slides
Videos
#ConvCon
![Page 32: Building a Company Wide Culture of Experimentation - Conversion Conference Las Vegas 2016](https://reader034.fdocuments.in/reader034/viewer/2022052418/586fdddf1a28ab18428b69d3/html5/thumbnails/32.jpg)
Idea from Analyst: To-do-list
• Data: low % of newly registered users commenting in 1st week
• Hypothesis: Increase % of first week commenters
• Test: Use a To-do-list to guide new users (start with easy activities)
#ConvCon
![Page 33: Building a Company Wide Culture of Experimentation - Conversion Conference Las Vegas 2016](https://reader034.fdocuments.in/reader034/viewer/2022052418/586fdddf1a28ab18428b69d3/html5/thumbnails/33.jpg)
Result: To-do-list
• Result: over 700% lift in 1st week commenting
#ConvCon
![Page 34: Building a Company Wide Culture of Experimentation - Conversion Conference Las Vegas 2016](https://reader034.fdocuments.in/reader034/viewer/2022052418/586fdddf1a28ab18428b69d3/html5/thumbnails/34.jpg)
Key Takeaways
• Share learning & celebrate wins
• Empower full team to generate ideas
• Ensure core team has process to manage inputs
#ConvCon
![Page 35: Building a Company Wide Culture of Experimentation - Conversion Conference Las Vegas 2016](https://reader034.fdocuments.in/reader034/viewer/2022052418/586fdddf1a28ab18428b69d3/html5/thumbnails/35.jpg)
Join GH Projects Private Betaat Projects.GrowthHackers.com
Where Teams Drive Results with High Tempo Testing:
1) Team Ideation 2) Prioritize Testing 3) Capture Learning
#ConvCon