Lead Scoring: How a Stanford Engineer and MIT MBA Approaches It - Ilya Mirman - Onshape - Growth...
-
Upload
mike-volpe -
Category
Business
-
view
680 -
download
4
Transcript of Lead Scoring: How a Stanford Engineer and MIT MBA Approaches It - Ilya Mirman - Onshape - Growth...
![Page 1: Lead Scoring: How a Stanford Engineer and MIT MBA Approaches It - Ilya Mirman - Onshape - Growth Camp June 2016](https://reader036.fdocuments.in/reader036/viewer/2022081512/588243991a28ab37158b460f/html5/thumbnails/1.jpg)
Lead Scoring:How a Stanford Engineer
and MIT MBA Approaches itIlya Mirman
VP of Marketing, Onshape
![Page 2: Lead Scoring: How a Stanford Engineer and MIT MBA Approaches It - Ilya Mirman - Onshape - Growth Camp June 2016](https://reader036.fdocuments.in/reader036/viewer/2022081512/588243991a28ab37158b460f/html5/thumbnails/2.jpg)
(Lead Scoring Tips & Tricks)Ilya Mirman
VP of Marketing, Onshape
![Page 3: Lead Scoring: How a Stanford Engineer and MIT MBA Approaches It - Ilya Mirman - Onshape - Growth Camp June 2016](https://reader036.fdocuments.in/reader036/viewer/2022081512/588243991a28ab37158b460f/html5/thumbnails/3.jpg)
1. Why score leads?2. Multi-factor scoring3. Implementation4. Assessing effectiveness
Agenda
![Page 4: Lead Scoring: How a Stanford Engineer and MIT MBA Approaches It - Ilya Mirman - Onshape - Growth Camp June 2016](https://reader036.fdocuments.in/reader036/viewer/2022081512/588243991a28ab37158b460f/html5/thumbnails/4.jpg)
Why score leads?> Some leads are much
better than others> Sales (probably) cannot
(and should not) work every lead
> Marketing should develop and identify the best leads for Sales to follow up on
![Page 5: Lead Scoring: How a Stanford Engineer and MIT MBA Approaches It - Ilya Mirman - Onshape - Growth Camp June 2016](https://reader036.fdocuments.in/reader036/viewer/2022081512/588243991a28ab37158b460f/html5/thumbnails/5.jpg)
Single factor scoring> Type of lead source
(or another factor)> Prioritize leads> Identify cut-off
(Hopefully, we’re all doing at least this)
![Page 6: Lead Scoring: How a Stanford Engineer and MIT MBA Approaches It - Ilya Mirman - Onshape - Growth Camp June 2016](https://reader036.fdocuments.in/reader036/viewer/2022081512/588243991a28ab37158b460f/html5/thumbnails/6.jpg)
Benefits of Using Multiple Factors
![Page 7: Lead Scoring: How a Stanford Engineer and MIT MBA Approaches It - Ilya Mirman - Onshape - Growth Camp June 2016](https://reader036.fdocuments.in/reader036/viewer/2022081512/588243991a28ab37158b460f/html5/thumbnails/7.jpg)
> Lots of “signals” regarding lead quality:●Lead source●Activity on site●SaaS app’s usage metrics●Demographics (user, company)●Community participation●Etc.
> Taken together, a better way to assess lead quality
Benefits of multiple factors
![Page 8: Lead Scoring: How a Stanford Engineer and MIT MBA Approaches It - Ilya Mirman - Onshape - Growth Camp June 2016](https://reader036.fdocuments.in/reader036/viewer/2022081512/588243991a28ab37158b460f/html5/thumbnails/8.jpg)
Example: Activity on site
![Page 9: Lead Scoring: How a Stanford Engineer and MIT MBA Approaches It - Ilya Mirman - Onshape - Growth Camp June 2016](https://reader036.fdocuments.in/reader036/viewer/2022081512/588243991a28ab37158b460f/html5/thumbnails/9.jpg)
Example: SaaS app usage
![Page 10: Lead Scoring: How a Stanford Engineer and MIT MBA Approaches It - Ilya Mirman - Onshape - Growth Camp June 2016](https://reader036.fdocuments.in/reader036/viewer/2022081512/588243991a28ab37158b460f/html5/thumbnails/10.jpg)
Example: Demographics
![Page 11: Lead Scoring: How a Stanford Engineer and MIT MBA Approaches It - Ilya Mirman - Onshape - Growth Camp June 2016](https://reader036.fdocuments.in/reader036/viewer/2022081512/588243991a28ab37158b460f/html5/thumbnails/11.jpg)
Example: Community Participation
![Page 12: Lead Scoring: How a Stanford Engineer and MIT MBA Approaches It - Ilya Mirman - Onshape - Growth Camp June 2016](https://reader036.fdocuments.in/reader036/viewer/2022081512/588243991a28ab37158b460f/html5/thumbnails/12.jpg)
Implementation
![Page 13: Lead Scoring: How a Stanford Engineer and MIT MBA Approaches It - Ilya Mirman - Onshape - Growth Camp June 2016](https://reader036.fdocuments.in/reader036/viewer/2022081512/588243991a28ab37158b460f/html5/thumbnails/13.jpg)
> Outsource to a 3rd party “black box” grading service●Some good options out there (but I wouldn’t start here)
> Use marketing automation software to define scoring criteria●Requires trial/error for tuning
> Develop a mathematical expression that can be easily implemented in CRM tools●The focus of this talk
Implementation options: Pros/Cons
![Page 14: Lead Scoring: How a Stanford Engineer and MIT MBA Approaches It - Ilya Mirman - Onshape - Growth Camp June 2016](https://reader036.fdocuments.in/reader036/viewer/2022081512/588243991a28ab37158b460f/html5/thumbnails/14.jpg)
> Could be as simple as a single factor●Though ideally, find two or more
> Need ~100-1,000 leads and 10-50 won deals● If too early to have enough won deals, could use “MQL” or some other
level of qualification (e.g., “Opportunity”)> Start looking for factors EARLY
●Test multiple hypotheses●Make sure your systems support collecting the data (potentially, record
value at time of purchase)
How early? How sophisticated?
![Page 15: Lead Scoring: How a Stanford Engineer and MIT MBA Approaches It - Ilya Mirman - Onshape - Growth Camp June 2016](https://reader036.fdocuments.in/reader036/viewer/2022081512/588243991a28ab37158b460f/html5/thumbnails/15.jpg)
> Multiple options (a blend of science and art)
> I like linear regression● Linear combination of factors● Some simplifying assumptions (e.g., factors
are independent)● May not need ALL factors (in fact, might not
be able to use all)
> Goal: figure out each factor’s weight to maximizes prediction of whether lead will BUY
What Type of Model?
![Page 16: Lead Scoring: How a Stanford Engineer and MIT MBA Approaches It - Ilya Mirman - Onshape - Growth Camp June 2016](https://reader036.fdocuments.in/reader036/viewer/2022081512/588243991a28ab37158b460f/html5/thumbnails/16.jpg)
Goal: figure out each factor’s weight
![Page 17: Lead Scoring: How a Stanford Engineer and MIT MBA Approaches It - Ilya Mirman - Onshape - Growth Camp June 2016](https://reader036.fdocuments.in/reader036/viewer/2022081512/588243991a28ab37158b460f/html5/thumbnails/17.jpg)
Putting it to practice - with Excel
![Page 18: Lead Scoring: How a Stanford Engineer and MIT MBA Approaches It - Ilya Mirman - Onshape - Growth Camp June 2016](https://reader036.fdocuments.in/reader036/viewer/2022081512/588243991a28ab37158b460f/html5/thumbnails/18.jpg)
Source Data
![Page 19: Lead Scoring: How a Stanford Engineer and MIT MBA Approaches It - Ilya Mirman - Onshape - Growth Camp June 2016](https://reader036.fdocuments.in/reader036/viewer/2022081512/588243991a28ab37158b460f/html5/thumbnails/19.jpg)
Step 1: Find factors that correlate w/purchase
![Page 20: Lead Scoring: How a Stanford Engineer and MIT MBA Approaches It - Ilya Mirman - Onshape - Growth Camp June 2016](https://reader036.fdocuments.in/reader036/viewer/2022081512/588243991a28ab37158b460f/html5/thumbnails/20.jpg)
Step 1: Find factors that correlate w/purchase
![Page 21: Lead Scoring: How a Stanford Engineer and MIT MBA Approaches It - Ilya Mirman - Onshape - Growth Camp June 2016](https://reader036.fdocuments.in/reader036/viewer/2022081512/588243991a28ab37158b460f/html5/thumbnails/21.jpg)
Step 2: Linear Regression to Establish
![Page 22: Lead Scoring: How a Stanford Engineer and MIT MBA Approaches It - Ilya Mirman - Onshape - Growth Camp June 2016](https://reader036.fdocuments.in/reader036/viewer/2022081512/588243991a28ab37158b460f/html5/thumbnails/22.jpg)
![Page 23: Lead Scoring: How a Stanford Engineer and MIT MBA Approaches It - Ilya Mirman - Onshape - Growth Camp June 2016](https://reader036.fdocuments.in/reader036/viewer/2022081512/588243991a28ab37158b460f/html5/thumbnails/23.jpg)
Regression ResultStatistical measure of how close the data are to the fitted regression line
t-stat: Measure of the variable’s relevance.
Examples:● Above 2 (or less
than -2) 95% ⇒confidence that it’s relevant
● Above 3.5 (or less than -3.5) 99.96% ⇒confidence that it’s relevant
May need to iterate:● Are all variables
significant?● Does the sign of
the coefficient make sense?
![Page 24: Lead Scoring: How a Stanford Engineer and MIT MBA Approaches It - Ilya Mirman - Onshape - Growth Camp June 2016](https://reader036.fdocuments.in/reader036/viewer/2022081512/588243991a28ab37158b460f/html5/thumbnails/24.jpg)
Predicted vs. ActualWhat the regression model predicts
Difference between “Predicted” and “Actual”
Actual(add together “Predicted” and “Residuals”)
![Page 25: Lead Scoring: How a Stanford Engineer and MIT MBA Approaches It - Ilya Mirman - Onshape - Growth Camp June 2016](https://reader036.fdocuments.in/reader036/viewer/2022081512/588243991a28ab37158b460f/html5/thumbnails/25.jpg)
Predicted vs. ActualLet’s create a column for “Cumulative Deals”
![Page 26: Lead Scoring: How a Stanford Engineer and MIT MBA Approaches It - Ilya Mirman - Onshape - Growth Camp June 2016](https://reader036.fdocuments.in/reader036/viewer/2022081512/588243991a28ab37158b460f/html5/thumbnails/26.jpg)
Now, let’s SORT by lead grade...> This data set:
●Over 30,000 leads●Less than 600 deals (2%)
> Note the high hit rate of deals for the high score leads (over 50%)
![Page 27: Lead Scoring: How a Stanford Engineer and MIT MBA Approaches It - Ilya Mirman - Onshape - Growth Camp June 2016](https://reader036.fdocuments.in/reader036/viewer/2022081512/588243991a28ab37158b460f/html5/thumbnails/27.jpg)
Results> Top 10% of leads
drive 70% of deals
> Top 20% of leads drive 85% of deals
![Page 28: Lead Scoring: How a Stanford Engineer and MIT MBA Approaches It - Ilya Mirman - Onshape - Growth Camp June 2016](https://reader036.fdocuments.in/reader036/viewer/2022081512/588243991a28ab37158b460f/html5/thumbnails/28.jpg)
Evaluating & Improving Your Models
![Page 29: Lead Scoring: How a Stanford Engineer and MIT MBA Approaches It - Ilya Mirman - Onshape - Growth Camp June 2016](https://reader036.fdocuments.in/reader036/viewer/2022081512/588243991a28ab37158b460f/html5/thumbnails/29.jpg)
Comparing Multiple Models
![Page 30: Lead Scoring: How a Stanford Engineer and MIT MBA Approaches It - Ilya Mirman - Onshape - Growth Camp June 2016](https://reader036.fdocuments.in/reader036/viewer/2022081512/588243991a28ab37158b460f/html5/thumbnails/30.jpg)
Pop Quiz: Which Model is Better?
![Page 31: Lead Scoring: How a Stanford Engineer and MIT MBA Approaches It - Ilya Mirman - Onshape - Growth Camp June 2016](https://reader036.fdocuments.in/reader036/viewer/2022081512/588243991a28ab37158b460f/html5/thumbnails/31.jpg)
1. Identify parameters of interest2. Collect data on LEADS and DEALS3. Identify factors that correlate w/purchase4. Create Excel spreadsheet5. Run linear regression6. Iterate on which parameters to include in model7. Tune model to add/remove factors8. Evaluate quality of model
(Rinse & repeat, at the right time)
Summary
![Page 32: Lead Scoring: How a Stanford Engineer and MIT MBA Approaches It - Ilya Mirman - Onshape - Growth Camp June 2016](https://reader036.fdocuments.in/reader036/viewer/2022081512/588243991a28ab37158b460f/html5/thumbnails/32.jpg)
imirman [at] onshape.com
(Thanks!)
QUESTIONS?