"Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan...
-
Upload
hyper-wellbeing -
Category
Technology
-
view
26 -
download
0
Transcript of "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan...
![Page 1: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/1.jpg)
1
![Page 2: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/2.jpg)
Using Data Science to Design Effective Precision Preventative Behavioral Medicine
Ryan QuanData ScientistOmada Health
![Page 3: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/3.jpg)
Outline Omada Health
• Context! Who are we? What do we do? Data science?
Building a Data Science Culture from the Ground Up• Winning hearts & minds with data
The Data Science / Product Relationship• What we’ve learned doesn’t work, and what we think works
The Paths to Personalized (Precision) Prevention• How to target the right patient, at the right time, in the right way
• Impactful experimentation: beyond the sub-group analysis
3
![Page 4: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/4.jpg)
Outline Omada Health
• Context! Who are we? What do we do? Data science?
Building a Data Science Culture from the Ground Up• Winning hearts & minds with data
The Data Science / Product Relationship• What we’ve learned doesn’t work, and what we think works
The Paths to Personalized (Precision) Prevention• How to target the right patient, at the right time, in the right way
• Impactful experimentation: beyond the sub-group analysis
4
![Page 5: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/5.jpg)
OMADA HEALTH 5OMADA HEALTH
![Page 6: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/6.jpg)
THE PROBLEM
6
![Page 7: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/7.jpg)
7
![Page 8: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/8.jpg)
8
![Page 9: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/9.jpg)
THE SOLUTION
9
![Page 10: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/10.jpg)
Lifestyle Change
10
58% Reduction
![Page 11: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/11.jpg)
11
Large Body of Clinical Evidence for DPP
![Page 12: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/12.jpg)
THE PROBLEMTHE SOLUTION
WHAT’S THE DEAL?
12
![Page 13: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/13.jpg)
13
Behavior Change is Hard
![Page 14: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/14.jpg)
EDUCATION
![Page 15: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/15.jpg)
TRACKERS
![Page 16: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/16.jpg)
CALORIE COUNTING
![Page 17: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/17.jpg)
HEALTH COACHING
![Page 18: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/18.jpg)
SOCIAL SUPPORT
![Page 19: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/19.jpg)
One Dimension is Not Enough
19
![Page 20: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/20.jpg)
20
![Page 21: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/21.jpg)
INTRODUCINGThe Omada Program
![Page 22: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/22.jpg)
OMADA HEALTH
PARTICIPANT PEERGROUP
Participants are matched into peer groups for
encouragement and healthy competition with the entire group progressing along a
shared timeline.
EASY-TO-USETECHNOLOGY
A wireless scale, pedometer, mobile apps, and interactive
portal are used to track weight loss, activity, meals,
and program accomplishments.
ONLINE INTERACTIVE LESSONS
Participants learn how to eat healthier, increase activity
levels, and overcome challenges through fun lessons
and games accessed via an online portal or mobile app.
FULL-TIME PROFESSIONALPREVENT COACH
A dedicated coach provides participants with support,
motivation, and personalized recommendations along their
Prevent journey.
22OMADA HEALTH
OVER 19 POINTS OF ENGAGEMENT PER WEEK
Group DiscussionPrevent Coach ConversationSkill ChallengeSuccess Updates
Weigh-InsExercise LogMeal TrackingLesson Completions
![Page 23: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/23.jpg)
23
![Page 24: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/24.jpg)
OMADA HEALTH 24
OMADA
![Page 25: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/25.jpg)
25
![Page 26: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/26.jpg)
26Data Science
![Page 27: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/27.jpg)
Data Science @ Omada
27
• Use analytics, machine learning and experimentation to parameterize what works (and what doesn’t) for behavior change
• Deploy the right intervention, at the right time, for the right patient
Precision Preventative Medicine
![Page 28: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/28.jpg)
The Data (one participant out of 100k+)
28
5
0
Weight Loss (%
)
15
10
![Page 29: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/29.jpg)
5
0
Weight Loss (%
)
15
10
29
Private Messages• Private messages between
HC and participants• Over 1.2M messages• Data:
- Emotionally charged raw text (NLP)
The Data (one participant out of 100k+)
![Page 30: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/30.jpg)
5
0
Weight Loss (%
)
15
10
30
Lessons / Curriculum• Based on original DPP
curriculum• Basic nutrition/activity
information• Interactive w/ games, etc.• Data:
- Completion rates- Reading times- Comprehension
The Data (one participant out of 100k+)
![Page 31: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/31.jpg)
5
0
Weight Loss (%
)
15
10
31
Physical Activity Tracking• Personalized adaptive daily step goals• MyFitnessPal and auto-step tracking
integrations• Over 3.6M activities recorded• Data:
- Steps entered- Minutes of activity- Location (physical proximity triggers)
The Data (one participant out of 100k+)
![Page 32: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/32.jpg)
5
0
Weight Loss (%
)
15
10
32
Group Messaging / Social Support• Group communication • participant <> participant• participant <> health coach• Data:
- Emotionally charged text communication (NLP)
The Data (one participant out of 100k+)
![Page 33: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/33.jpg)
5
0
Weight Loss (%
)
15
10
33
Meal Tracking
• Daily food tracking• 6.2M meals tracked• Active feedback from coach• Data:
- Raw text (food)- Healthiness- Portion size- Time of day
The Data (one participant out of 100k+)
![Page 34: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/34.jpg)
5
0
Weight Loss (%
)
15
10
34
Weighing-in / Progress • Low-power 3G scale• Automatically uploads to
profile on progress page• Weighing in as habit
formation• 6M weigh-ins• Data:
- Weight value- Time of day
The Data (one participant out of 100k+)
![Page 35: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/35.jpg)
The Data
![Page 36: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/36.jpg)
Outline Omada Health
• Context! Who are we? What do we do? Data science?
Building a Data Science Culture from the Ground Up
• Winning hearts & minds with data
The Data Science / Product Relationship• What we’ve learned doesn’t work, and what we think works
The Paths to Personalized (Precision) Prevention• How to target the right patient, at the right time, in the right way
• Impactful experimentation: beyond the sub-group analysis
36
![Page 37: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/37.jpg)
Building a data science cultureScenario:
You’re the first/only data scientist at your company. You have amazing data with a lot of potential, but your company has no data science
team or experience with data driven product development.
What do you do? Two choices:
![Page 38: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/38.jpg)
Building a data science cultureScenario:
You’re the first/only data scientist at your company. You have amazing data with a lot of potential, but your company has no data science
team or experience with data driven product development.
What do you do? Two choices:Choice 1: Data Police
![Page 39: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/39.jpg)
Building a data science cultureScenario:
You’re the first/only data scientist at your company. You have amazing data with a lot of potential, but your company has no data science
team or experience with data driven product development.
What do you do? Two choices:Choice 2: Build a data
science culture
![Page 40: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/40.jpg)
Building a data science culture
Lots of ways to kick this off:• Brown bag lunch presentations / journal club• Deploy visualization tools• Data education/proselytization• Pro-active analytics as a service• Start a data blog…
![Page 41: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/41.jpg)
Building a data science culture“Plot of the Week (PotW)”
• Internal weekly email data blog • ‘Buzz-feedy’ stories using our data in interesting ways• Exposed company to opportunities with data• Mobilized support and excitement for data science
![Page 42: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/42.jpg)
Building a data science culture“Plot of the Week (PotW)” examples:
1. All in the Family Provides view into raw data
2. Minds over Matter Illustrates potential uses of our data
3. Just Breathe Educates subject matter with data
![Page 43: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/43.jpg)
Building a data science culture“Plot of the Week (PotW)” examples:
1. All in the Family Provides view into raw data
2. Minds over Matter Illustrates potential uses of our data
3. Just Breathe Educates subject matter with data
Our scales become ‘part of the family’ sending weight data of anyone (anything) that steps
on it
![Page 44: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/44.jpg)
PotW: All in the Family
44
Family of two, losing weight together
![Page 45: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/45.jpg)
PotW: All in the Family
45
Another family of two, losing weight together, a few trips to Sandals in there too?
![Page 46: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/46.jpg)
PotW: All in the Family
46
Family of three, growing child
![Page 47: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/47.jpg)
PotW: All in the Family
47
Family of three, neighborhood party
![Page 48: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/48.jpg)
PotW: All in the Family
48
???
![Page 49: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/49.jpg)
Outline Omada Health
• Context! Who are we? What do we do? Data science?
Building a Data Science Culture from the Ground Up• Winning hearts & minds with data
The Data Science / Product Relationship• What we’ve learned doesn’t work, and what we think works
The Paths to Personalized (Precision) Prevention• How to target the right patient, at the right time, in the right way
• Impactful experimentation: beyond the sub-group analysis
49
![Page 50: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/50.jpg)
The Data Science & Product Relationship
Product
50
Sub-optimal - data science as ‘advisor’
Data-Science
• Exploration/data mining
• Modeling• Proto-typing• Lean-experimentation
• Prioritizes engineering work
• Creates data-models• Implements experiments• Requests analysis
Recommendations/insights
Requests
![Page 51: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/51.jpg)
The Data Science & Product Relationship
Product
51
Sub-optimal - data science as ‘advisor’
Data-Science
Recommendations/insights
Requests
![Page 52: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/52.jpg)
The Data Science & Product Relationship
Data Science
52
ProductOptimal - data science as ‘driver’
• Has own backlog/engineers• Drives data-model design• Experimentation built into
Product• Data scientists can launch
production experiments
Hypothesis
Generation
Experimentation
IterationOptimization
![Page 53: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/53.jpg)
Outline Omada Health
• Context! Who are we? What do we do? Data science?
Building a Data Science Culture from the Ground Up• Winning hearts & minds with data
The Data Science / Product Relationship• What we’ve learned doesn’t work, and what we think works
The Paths to Personalized (Precision) Prevention
• How to target the right patient, at the right time, in the right way• Impactful experimentation: beyond the sub-group analysis
53
![Page 54: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/54.jpg)
Outline Omada Health
• Context! Who are we? What do we do? Data science?
Building a Data Science Culture from the Ground Up• Winning hearts & minds with data
The Data Science / Product Relationship• What we’ve learned doesn’t work, and what we think works
The Paths to Personalized (Precision) Prevention• How to target the right patient, at the right time, in the right way
• Impactful experimentation: beyond the sub-group analysis
54
![Page 55: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/55.jpg)
Impactful Experimentation
55
Pros Cons• Causality !=
Correlation• Common language of
scientific discovery
• Low N• Labor intensive, expensive• Slow• Bias in patient populations• Results often not generalizable
For over a century, the Randomized Control Trial (RCT) has been the gold standard for scientific discovery
![Page 56: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/56.jpg)
Impactful Experimentation
56
Largest meta-analysisof DPP ~5.5k participants
>60k Omada participants
vs
At Omada, we have built Randomized Control Experiments into the product to take advantage of the measurement power of experimentation, while the digital nature of our program mitigates the cons (e.g. high N, quick iteration, inexpensive, generalized population, deeper data).
![Page 57: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/57.jpg)
Real-time, Randomized Controlled Experiments (RCEs)
35 in-product experiments & growing quicklyThis enables:
• Rapid exploration of product hypotheses• Structured innovation and evolution of
Prevent• Massive personalization of sub-specifics of
the intervention • Centralized intervention delivery allows
coordinated optimization across all demographics and sectors
• Asking fundamental about what works, and what doesn’t, in behavior change
![Page 58: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/58.jpg)
Personalization through ExperimentationSubgroup Analysis
Answers questions like “Do older people respond differently to Tx than younger?”, or “Do males respond better than females?”
Broad cuts lose resolution to subtle changes
Guided by heuristics (e.g. demographics) and variables available – can lead to biases ITT
Age > 65
Uplift Modeling• Statistical model, based on experimental
data• Predicts who the intervention is likely to
be most impactful for• Don’t need heuristics (other than
choosing inputs) – let data do the personalization for you
![Page 59: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/59.jpg)
59
Subgroup Analysis: Physical Activity Engagement
• Physical activity is a large part of a healthy lifestyle• The second phase of the Omada Program focuses on
increasing patient’s physical activity:1. Providing pedometers to patients to collect physical activity (“steps”)
data2. Educational components and health-coach interaction about exercise3. Setting daily “step goals” for participants
![Page 60: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/60.jpg)
60
Subgroup Analysis: Personalization of Step Goals
Challenge: can we provide personalized step goals to increase physical activity behaviors?
Patients are challenged with daily ‘step goals’ to increase physical activity
![Page 61: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/61.jpg)
61
What does the data tell us:
Subgroup Analysis: Personalization of Step Goals
![Page 62: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/62.jpg)
62
Historical steps recorded, segmented by age/BMI
Assign each participant a personalized step goal based on: similar age/BMI historical mean + 20%
Subgroup Analysis: Personalization of Step Goals
![Page 63: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/63.jpg)
63
Experiment: 50% receive ‘personalized steps’, 50% receive static (7500/day)
Personalized
Static (7500)
Step
s Tra
cked
/ W
eek
Program Week Program Week
Subgroup Analysis: Personalization of Step Goals
![Page 64: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/64.jpg)
64
Experiment: 50% receive ‘personalized steps’, 50% receive static (7500/day)
Adaptive goals may be more impactful for younger [18-30] males
PersonalizedStatic (7500)
Step
s Tra
cked
/ W
eek
Program Week Program Week
Subgroup Analysis: Personalization of Step Goals
![Page 65: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/65.jpg)
65
PersonalizedStatic (7500)
Step
s Tra
cked
/ W
eek
Program Week Program Week
No apparent effect in middle-aged adults
Experiment: 50% receive ‘personalized steps’, 50% receive static (7500/day)Subgroup Analysis: Personalization of Step Goals
![Page 66: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/66.jpg)
Personalization through ExperimentationSubgroup Analysis
Answers questions like “Do older people respond differently to Tx than younger?”, or “Do males respond better than females?”
Broad cuts lose resolution to subtle changes
Guided by heuristics (e.g. demographics) and variables available – can lead to biases ITT
Age > 65
Uplift Modeling• Statistical model, based on experimental
data• Predicts who the intervention is likely to
be most impactful for• Don’t need heuristics (other than
choosing inputs) – let data do the personalization for you
![Page 67: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/67.jpg)
Personalization through ExperimentationSubgroup Analysis
Answers questions like “Do older people respond differently to Tx than younger?”, or “Do males respond better than females?”
Broad cuts lose resolution to subtle changes
Guided by heuristics (e.g. demographics) and variables available – limiting and can lead to biases ITT
Age > 65
Uplift Modeling• Statistical model, based on experimental
data• Predicts who the intervention is likely to
be most impactful for• Don’t need heuristics (other than
choosing inputs) – let data do personalization for you
![Page 68: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/68.jpg)
Experiment: Health coach feedback on participant food tracking
Context:• Building awareness of nutrition through food tracking is a
core part of the Prevent program• Ran experiment to understand the effects of coach feedback
on participant food tracking• Participants in the experiment received feedback about their
food choices from their coaches, those in the control arm did not.
Experiment: Providing impactful food feedback
![Page 69: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/69.jpg)
Experimental design
RxParticipant X, has tracked protein shake for 5 days in a row and has shown recent weigh gain – please provide feedback
Experiment: Providing impactful food feedback
1. Candidate Selection• Participants who track a meal & predicted to gain weight
2. Randomization• 50% are eligible to receive feedback
3. Prescription Generation• Coaches are notified of tracking behavior, with suggested
feedback4. Intervention Delivery
• Coach reaches out to participant with feedback on the flagged meal
5. Monitor outcomes
![Page 70: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/70.jpg)
RxParticipant X, has tracked protein shake for 5 days in a row and has shown recent weigh gain – please provide feedback
Experiment: Providing impactful food feedback
1. Candidate Selection• Participants who track a meal & predicted to gain weight
2. Randomization• 50% are eligible to receive feedback
3. Prescription Generation• Coaches are notified of tracking behavior, with suggested
feedback4. Intervention Delivery
• Coach reaches out to participant with feedback on the flagged meal
5. Monitor outcomes
Experimental design
![Page 71: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/71.jpg)
RxParticipant X, has tracked protein shake for 5 days in a row and has shown recent weight gain – please provide feedback
Experiment: Providing impactful food feedback
1. Candidate Selection• Participants who track a meal & predicted to gain weight
2. Randomization• 50% are eligible to receive feedback (other 50% in ‘control’)
3. Prescription Generation• Coaches are notified of tracking behavior, with suggested
feedback4. Intervention Delivery
• Coach reaches out to participant with feedback on the flagged meal
5. Monitor outcomes
Experimental design
![Page 72: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/72.jpg)
1. Candidate Selection• Participants who track a meal & predicted to gain weight
2. Randomization• 50% are eligible to receive feedback
3. Prescription Generation• Coaches are notified of tracking behavior, with suggested
feedback4. Intervention Delivery
• Coach reaches out to participant with feedback on the flagged meal
5. Monitor outcomes
Experiment: Providing impactful food feedback
RxParticipant X, has tracked protein shake for 5 days in a row and has shown recent weight gain – please provide feedback
Experimental design
![Page 73: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/73.jpg)
RxParticipant X, has tracked protein shake for 5 days in a row and has shown recent weight gain – please provide feedback
Experiment: Providing impactful food feedback
1. Candidate Selection• Participants who track a meal & predicted to gain weight
2. Randomization• 50% are eligible to receive feedback
3. Prescription Generation• Coaches are notified of tracking behavior, with suggested
feedback4. Intervention Delivery
• Coach reaches out to participant with feedback on the flagged meal
5. Monitor outcomes
Experimental design
![Page 74: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/74.jpg)
5. Monitor outcomes 10-15% increase in days with meals tracked
Experiment: Providing impactful food feedback
![Page 75: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/75.jpg)
8-12% increase in meals with healthiness tracked5. Monitor outcomes
Experiment: Providing impactful food feedback
![Page 76: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/76.jpg)
+8% in relative week-16 weight loss
5. Monitor outcomes
Experiment: Providing impactful food feedback
![Page 77: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/77.jpg)
Using experimental data for personalizationUplift modeling, built on experimental data, can tell you probability of incremental response for an individual, given a hypothetical intervention
When delivering interventions:
• The Sure Things: response regardless of intervention• The Lost Causes: no response regardless of
intervention• The Sleeping Dogs: less likely to respond with
intervention• The Persuadables: only respond to because of
intervention
Uplift modeling targets The Persuadables and leaves the Sleeping Dogs alone.
![Page 78: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/78.jpg)
Uplift modeling Example: Food Feedback
• Random forest uplift model trained on likelihood of participant continuing to track meals after receiving food feedback
• Model accounts for participant demographics, program behavior and response to previous feedback
• Preliminary results: targeting ~80% of eligible population with uplift model increases response ~2% (from 5% to 7%)
![Page 79: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/79.jpg)
Outline
79
Omada Health• Context! Who are we? What do we do? Data science?
Building a Data Science Culture from the Ground Up• Winning hearts & minds with data
The Data Science / Product Relationship• What we’ve learned doesn’t work, and what we think works
The Paths to Personalized (Precision) Prevention• How to target the right patient, at the right time, in the right way
• Impactful experimentation: beyond the sub-group analysis
![Page 80: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/80.jpg)
WHAT’S NEXT?
80
![Page 81: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/81.jpg)
THANKS!@omadahealth
@sciencethedata@ryancquan
![Page 82: "Using Data Science to Design Effective Precision Preventative Behavioral Medicine" - Ryan Quan (Data Scientist, Omada Health)](https://reader035.fdocuments.in/reader035/viewer/2022070514/587f864b1a28ab28518b4cd9/html5/thumbnails/82.jpg)
82