Data Driven Growth

1

John Egan Engineering Manager - Engagement Team @ Pinterest

2

• Retail shopping rewards app • Led Growth Engineering team • Grew from 1MM to 8MM users in 3 years

• Acquired for $200MM

• Visual discovery & bookmarking app

• Eng manager for Engagement • Over 100 million MAUs • Engagement experiments have added millions of WAUs

Pre Product/Market Fit

Focus on Product First

4

Initial Traction

Do Things That Dont Scale

6

Growth Stage

Prioritize Based On ROI

8

9

New Users

MAUs

Dormant Users

10

New Users

MAUs

Dormant Users

11

New Users

MAUs

Dormant Users

12

New Users

Dormant Users

Core

Casual

Marginal

13

New Users

MAUs

Dormant Users

MAU Growth Accounting

14

Acquisition

15

Funnels

16

17

2013

18

2014

19

2015

Activation

20

1d7s• Percentage of new signups that use the app 1 or more times in the week

following signup • Quicker to run experiments with • At Pinterest this is highly correlated with user’s long-term retention

21

22

23

Engagement

24

One Key Engagement MetricMedium: Total Time Spent Reading (TTR) Twitter: MAUs with 7+ visits a month (7d28s) Uber: Weekly Trips Facebook: WAUs with 6+ visits a week (6L7s) Pinterest: Weekly Active Repiners or Clickers (WARCs)

25

Finding Your Key Engagement Metric1) What are the actions a user has to take to get value from your product?

2) What is the frequency someone need to use your product

26

User States

Core: Active multiple times a week Casual: Active once or twice a week Marginal: Active a couple times a month New: Joined in the past 28 days Dormant: Not active for 28 days Resurrected: Was dormant, but became active again in the past 28 days

27

Badging Holdout Experiment

2:50 PM 100%

28

Measuring Effect on Engagement

29

Retention & Resurrection

30

Marketing

Pros: • Coverage. Can reach the entire user base

Cons: • Not personalized, lower open rates • User’s have much lower tolerance

compared to emails

31

Transactional

Pros: • More personal than marketing

notifications

Cons: • Users need activity to generate

notifications • Need mechanisms to rate limit

32

Recommendations

Pros: • Coverage. Can reach most users • Personalized to the user

Cons: • Expensive and time consuming to build

out recommendation engine • Quality may not be good if user has low

amount of activity

33

2:50 PM 100%2:50 PM 100%

Measuring Engagement for Emails/Push

• Positive measures of quality - Must lift key engagement metric - Minimum required CTR rates

• Negative measures of quality - App deletions - Spam reports - Unsubscribes

34

Experiment Segmentation

35

All Users

36

Marginal Users (~1 month)

37

Core Users (multiple times a week)

38

Wrap Up• Pre Product/Market Fit: Product comes first

• Initial Traction: Do things that don’t scale

• Growth Stage: Prioritize projects based on return on investment

• Use funnels for analyzing acquisition flows

• 1d7s to analyze on activation

• Use user states to understand how engaged users are

• Make sure emails/notifications for true engagement

• Use segmentation for experiments & metrics in general

39

John EganGrowth Blog: jwegan.com

Email: me@jwegan.com

Twitter: @jwegan_com

Pinterest: pinterest.com/jwegan

40