Social Media Monitoring to Identify Risk
-
Upload
epidemico -
Category
Data & Analytics
-
view
302 -
download
0
Transcript of Social Media Monitoring to Identify Risk
![Page 1: Social Media Monitoring to Identify Risk](https://reader031.fdocuments.in/reader031/viewer/2022022001/58f0622d1a28abef248b45d7/html5/thumbnails/1.jpg)
May 25, 2016
Nabarun Dasgupta, Chief Science Officer
Epidemico, Inc.
felming • 3rd Pharmacovigilance & Risk Management Strategies Philadelphia
Social Media Monitoring
to Identify Risk
![Page 2: Social Media Monitoring to Identify Risk](https://reader031.fdocuments.in/reader031/viewer/2022022001/58f0622d1a28abef248b45d7/html5/thumbnails/2.jpg)
Disclosures
![Page 3: Social Media Monitoring to Identify Risk](https://reader031.fdocuments.in/reader031/viewer/2022022001/58f0622d1a28abef248b45d7/html5/thumbnails/3.jpg)
Epidemico is a wholly-owned subsidiary of Booz Allen Hamilton
Epidemico is commercializing aspects of this research.
![Page 4: Social Media Monitoring to Identify Risk](https://reader031.fdocuments.in/reader031/viewer/2022022001/58f0622d1a28abef248b45d7/html5/thumbnails/4.jpg)
FDA has been funding research technology for 3.5 years.
Office of the Chief Scientist (OCS)
Center for Tobacco Products (CTP)
Office of International Programs (OIP)
![Page 5: Social Media Monitoring to Identify Risk](https://reader031.fdocuments.in/reader031/viewer/2022022001/58f0622d1a28abef248b45d7/html5/thumbnails/5.jpg)
GSK has been a development partner to
extend technology and obtain GxP software
validation over the last 3 years.
![Page 6: Social Media Monitoring to Identify Risk](https://reader031.fdocuments.in/reader031/viewer/2022022001/58f0622d1a28abef248b45d7/html5/thumbnails/6.jpg)
A public-private partnership entering Year 3.
The WEB-RADR project is supported by the Innovative Medicines
Initiative Joint Undertaking (IMI JU) under grant agreement n°
115632, resources of which are composed of financial contributions
from the European Union's Seventh Framework Programme
(FP7/2007-2013) and EFPIA companies’ in kind contribution.
![Page 7: Social Media Monitoring to Identify Risk](https://reader031.fdocuments.in/reader031/viewer/2022022001/58f0622d1a28abef248b45d7/html5/thumbnails/7.jpg)
Introduction
![Page 8: Social Media Monitoring to Identify Risk](https://reader031.fdocuments.in/reader031/viewer/2022022001/58f0622d1a28abef248b45d7/html5/thumbnails/8.jpg)
![Page 9: Social Media Monitoring to Identify Risk](https://reader031.fdocuments.in/reader031/viewer/2022022001/58f0622d1a28abef248b45d7/html5/thumbnails/9.jpg)
Infographic elementsSlide sub title
The Drug I’m TakingThe Drug I’m Responsible for Monitoring
![Page 10: Social Media Monitoring to Identify Risk](https://reader031.fdocuments.in/reader031/viewer/2022022001/58f0622d1a28abef248b45d7/html5/thumbnails/10.jpg)
Proto-AEPosts with resemblance to Adverse Events
![Page 11: Social Media Monitoring to Identify Risk](https://reader031.fdocuments.in/reader031/viewer/2022022001/58f0622d1a28abef248b45d7/html5/thumbnails/11.jpg)
AstheniaMedDRA: 10003549
DizzinessMedDRA: 10013573
![Page 12: Social Media Monitoring to Identify Risk](https://reader031.fdocuments.in/reader031/viewer/2022022001/58f0622d1a28abef248b45d7/html5/thumbnails/12.jpg)
Data Processing
![Page 13: Social Media Monitoring to Identify Risk](https://reader031.fdocuments.in/reader031/viewer/2022022001/58f0622d1a28abef248b45d7/html5/thumbnails/13.jpg)
![Page 14: Social Media Monitoring to Identify Risk](https://reader031.fdocuments.in/reader031/viewer/2022022001/58f0622d1a28abef248b45d7/html5/thumbnails/14.jpg)
Collect unstructured data from
social media APIs, third-party
authorized resellers, and
automated scraping.
Acquire
![Page 15: Social Media Monitoring to Identify Risk](https://reader031.fdocuments.in/reader031/viewer/2022022001/58f0622d1a28abef248b45d7/html5/thumbnails/15.jpg)
detectsentiment
languagetranslation
statistics consolidatemultiples
detectadverseevents
detectbenefits
de-identifygeo-tag
Data are passed through a series
of apps, emerging as meaningful
bits of information.
Process
![Page 16: Social Media Monitoring to Identify Risk](https://reader031.fdocuments.in/reader031/viewer/2022022001/58f0622d1a28abef248b45d7/html5/thumbnails/16.jpg)
detectsentiment
languagetranslation
statistics consolidatemultiples
detectadverseevents
detectbenefits
de-identifygeo-tag API mobile RSS tables reportsCSV
,visuals
Relevant data are passed to another
series of apps in preparation for
human interpretation and analysis.
Export
![Page 17: Social Media Monitoring to Identify Risk](https://reader031.fdocuments.in/reader031/viewer/2022022001/58f0622d1a28abef248b45d7/html5/thumbnails/17.jpg)
API mobile RSS tables reportsCSV
,visuals
Relevant data are passed to another
series of apps in preparation for
human interpretation and analysis.
Export
![Page 18: Social Media Monitoring to Identify Risk](https://reader031.fdocuments.in/reader031/viewer/2022022001/58f0622d1a28abef248b45d7/html5/thumbnails/18.jpg)
Internet Vernacular
Translation
![Page 19: Social Media Monitoring to Identify Risk](https://reader031.fdocuments.in/reader031/viewer/2022022001/58f0622d1a28abef248b45d7/html5/thumbnails/19.jpg)
lost their eyesight
seeing weird color
seeing weird colour
doublevision
couldn’t see
visión doble
blind
googley eyed
blurry vision
changes in visioncross eyed
seeing weird
vision change
blindness
cross visionvisual snow
googly eyed
seeing double
making me eat like a mouse
lost appetite
#notevenhungry
appetite is nonexistent
apetite surpressed
didn’t get hungry
dont want to eat
killed my apetite
miss feeling hungry
killed my appetite
can’t eat
sin hambre
lost my appetite
no appetitey
lack of apetite
stomach small
lost teh appetite
never hungrynever want to eat
cant eat
couldntcrosseyed
blurry
anorexic
apetite surpressed
lack of apetite
killed my apetite lost teh appetite
making me eat like a mouse
never want to eat
Implied
#notevenhungry
no appetitey
Invented words and hashtags
googley eyed
googly eyed
seeing weird colour
seeing weird color
Varied Spelling
sin hambre
visión doble
Other Languages
blurry vision
Emoticons
Typos
![Page 20: Social Media Monitoring to Identify Risk](https://reader031.fdocuments.in/reader031/viewer/2022022001/58f0622d1a28abef248b45d7/html5/thumbnails/20.jpg)
making me eat like a mouse
anorexic
lost appetite
#notevenhungry
appetite is nonexistent
apetite surpressed
didn’t get hungry
dont want to eat
miss feeling hungry
killed my appetite
can’t eat
sin hambre
lost my appetite
no appetitey
lack of apetite
stomach small
lost teh appetite
never hungrynever want to eat
cant eat
lost their eyesight
seeing weird color
seeing weird colour
doublevision
couldn’t see
visión doble
blind
googley eyed
changes in visioncross eyed
seeing weird
vision change
blindness
cross visionvisual snow
googly eyed
seeing double
couldntcrosseyed
blurry
killed my apetite
blurry vision
Decreased appetite
MedDRA 10061428
Loss of appetite
SNOMED 79890006
Visual impairment
MedDRA 10047571
Visual impairment
SNOMED 397540003
![Page 21: Social Media Monitoring to Identify Risk](https://reader031.fdocuments.in/reader031/viewer/2022022001/58f0622d1a28abef248b45d7/html5/thumbnails/21.jpg)
![Page 22: Social Media Monitoring to Identify Risk](https://reader031.fdocuments.in/reader031/viewer/2022022001/58f0622d1a28abef248b45d7/html5/thumbnails/22.jpg)
Machine Learning
vs.
Natural Language Processing
![Page 23: Social Media Monitoring to Identify Risk](https://reader031.fdocuments.in/reader031/viewer/2022022001/58f0622d1a28abef248b45d7/html5/thumbnails/23.jpg)
Machine LearningFind reports of importance.
Natural Language ProcessingExtract the most relevant elements.
![Page 24: Social Media Monitoring to Identify Risk](https://reader031.fdocuments.in/reader031/viewer/2022022001/58f0622d1a28abef248b45d7/html5/thumbnails/24.jpg)
You are all MedDRA Certified Coders, right?
How to build a machine
learning classifier
![Page 25: Social Media Monitoring to Identify Risk](https://reader031.fdocuments.in/reader031/viewer/2022022001/58f0622d1a28abef248b45d7/html5/thumbnails/25.jpg)
Proto-AE Trash
Medicine Prozak® (Fluoxetine) for the treatment of bulimia
nervosa . Buy now Prozac-20mg "Fluoxetine" ...
Benefit
81%27%74%4%2%75%92%
Instant Feedback:
% Confidence that this is a Proto-AE
You just trained the classifier.
Congratulations!
![Page 26: Social Media Monitoring to Identify Risk](https://reader031.fdocuments.in/reader031/viewer/2022022001/58f0622d1a28abef248b45d7/html5/thumbnails/26.jpg)
System Performance
![Page 27: Social Media Monitoring to Identify Risk](https://reader031.fdocuments.in/reader031/viewer/2022022001/58f0622d1a28abef248b45d7/html5/thumbnails/27.jpg)
82,000,000+posts collected
Public posts from Facebook, Twitter, and patient forums back to 2012.
![Page 28: Social Media Monitoring to Identify Risk](https://reader031.fdocuments.in/reader031/viewer/2022022001/58f0622d1a28abef248b45d7/html5/thumbnails/28.jpg)
2,500+regulated products
Drugs, medical devices, vaccines, biologics,
tobacco products, supply chain
![Page 29: Social Media Monitoring to Identify Risk](https://reader031.fdocuments.in/reader031/viewer/2022022001/58f0622d1a28abef248b45d7/html5/thumbnails/29.jpg)
400,000+manually curated
Certified coders train machine learningclassifiers and build dictionary.
![Page 30: Social Media Monitoring to Identify Risk](https://reader031.fdocuments.in/reader031/viewer/2022022001/58f0622d1a28abef248b45d7/html5/thumbnails/30.jpg)
10,000+Vernacular English phrases
Updated daily by human curators.
Mandarin, French, Spanish, and Dutch next.
![Page 31: Social Media Monitoring to Identify Risk](https://reader031.fdocuments.in/reader031/viewer/2022022001/58f0622d1a28abef248b45d7/html5/thumbnails/31.jpg)
Positive Predictive Value or Precision
7 OF 10 posts contain adverse event information (0.68).
Can increase to 100% with manual curation (may vary by product).
If the algorithm says it is a Proto-AE, then 7 out of 10 times is actually is.
![Page 32: Social Media Monitoring to Identify Risk](https://reader031.fdocuments.in/reader031/viewer/2022022001/58f0622d1a28abef248b45d7/html5/thumbnails/32.jpg)
Sensitivity or Recall
Automated tools identify 9 OF 10 adverse events
across all products, all time, all data sources (0.88).
The algorithm correctly identifies 9 out of 10 Proto-AEs from the pool of everything.
![Page 33: Social Media Monitoring to Identify Risk](https://reader031.fdocuments.in/reader031/viewer/2022022001/58f0622d1a28abef248b45d7/html5/thumbnails/33.jpg)
Albuterol/Salbutamol
Jan 1 to May 24, 2016
![Page 34: Social Media Monitoring to Identify Risk](https://reader031.fdocuments.in/reader031/viewer/2022022001/58f0622d1a28abef248b45d7/html5/thumbnails/34.jpg)
In their own words. Indicatorscore
![Page 35: Social Media Monitoring to Identify Risk](https://reader031.fdocuments.in/reader031/viewer/2022022001/58f0622d1a28abef248b45d7/html5/thumbnails/35.jpg)
5.25% Twitter posts are Proto-AEs.
There were 10,185 mentions from Jan 1 to May 24, 2016.
Of these, 535 posts had adverse event information.
![Page 36: Social Media Monitoring to Identify Risk](https://reader031.fdocuments.in/reader031/viewer/2022022001/58f0622d1a28abef248b45d7/html5/thumbnails/36.jpg)
System organ class (n=535)
Nervous system
General Medical Psychiatric Injury &
Poisoning
Infections
Respiratory
![Page 37: Social Media Monitoring to Identify Risk](https://reader031.fdocuments.in/reader031/viewer/2022022001/58f0622d1a28abef248b45d7/html5/thumbnails/37.jpg)
Tremor
Altered state of
consciousness
Drug
ineffective
Insomnia
Bronchitis
Feeling
jittery
Incorrect
dose
Malaise
Hyper-
sensitivity
Non-
specific
rxn
Cough
Restlessness
Pain
Psycho-
motor
hyper-
activity
Dizziness OverdoseHeart
rate
increased
Lung
disorderMemory
impairmentSomnolence Fatigue Dependence
Top preferred terms for Proto-AEs (n=535)
![Page 38: Social Media Monitoring to Identify Risk](https://reader031.fdocuments.in/reader031/viewer/2022022001/58f0622d1a28abef248b45d7/html5/thumbnails/38.jpg)
Disproportionately reported Proto-AEs match clinical
trials - with additional AEs not seen in RCTs.
Social media data since 2013; RCT data from NIH DailyMed
RCT PrevelanceNervousness 20%
Tremor 20%
Headache 7%
Tachycardia 5%
Palpitations 5%
Muscle cramps 3%
Sleeplessness 2%
Weakness 2%
Dizziness 2%
Nausea 2%
Top Social Media Proto-AEs by PRRN PRR
Arrhythmia 2 123
Tachycardia 14 45
Tremor 1084 37
Feeling jittery 132 35
Palpitations 56 32
Mouth ulceration 5 31
Candida infection 2 14.7
Heart rate increased 60 11.0
Epistaxis 17 8.8
Pallor 3 8.8
![Page 39: Social Media Monitoring to Identify Risk](https://reader031.fdocuments.in/reader031/viewer/2022022001/58f0622d1a28abef248b45d7/html5/thumbnails/39.jpg)
What are good use cases
for social data in safety
and risk management?
![Page 40: Social Media Monitoring to Identify Risk](https://reader031.fdocuments.in/reader031/viewer/2022022001/58f0622d1a28abef248b45d7/html5/thumbnails/40.jpg)
Social media can fill knowledge gaps.
In addition to adverse events, the pharmaceutical companies using our social media data have identified
several use cases well-suited for social listening. Social media have already led to product defect recalls,
revision of markings for dosing on a syringe, AE labeling revision requests, and formal safety study
commitments.
http://www.gsk-clinicalstudyregister.com/study/202115
qualitycomplaints
clinicaltrials
benefits alcoholinteractions
drugabuse
patientquestions
counterfeit
![Page 41: Social Media Monitoring to Identify Risk](https://reader031.fdocuments.in/reader031/viewer/2022022001/58f0622d1a28abef248b45d7/html5/thumbnails/41.jpg)
What about bias?
![Page 42: Social Media Monitoring to Identify Risk](https://reader031.fdocuments.in/reader031/viewer/2022022001/58f0622d1a28abef248b45d7/html5/thumbnails/42.jpg)
All data have bias.Can we interpret the output within context and derive insight?
![Page 43: Social Media Monitoring to Identify Risk](https://reader031.fdocuments.in/reader031/viewer/2022022001/58f0622d1a28abef248b45d7/html5/thumbnails/43.jpg)
When do we actually listen to patients?
Each source of safety information provides different perspectives on patient experience.
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Clinical Trials Sponsor AE Practice Data Social Media
PRO Seriousness Completeness
Social media can elucidate what
is important to patients.
1. Patient-Reported Outcomes
Traditional PV focus on most serious of events
may be less central in social media.
2. Seriousness
Social media posts may by less complete,
but may deliver novel insight in aggregate.
3. Completeness
Three Dimensions of Safety Information
Hypothetical data for illustrative purposes only.
![Page 44: Social Media Monitoring to Identify Risk](https://reader031.fdocuments.in/reader031/viewer/2022022001/58f0622d1a28abef248b45d7/html5/thumbnails/44.jpg)
What do regulations say
about reporting of
adverse events in social
media?
![Page 45: Social Media Monitoring to Identify Risk](https://reader031.fdocuments.in/reader031/viewer/2022022001/58f0622d1a28abef248b45d7/html5/thumbnails/45.jpg)
Our observations.
It is clear that if the AE is on a company-owned social marketing platform, it has to be reported. However, anonymized data from general social media sites may be handled as observational data. The EMA GVP VI module has additional expectations. The IMI WEB-RADR project will provide research guidance for regulatory agencies in 2017. In the US, the focus is on multi-source causality, which can include social media.
But, in practice, the legal/compliance departments within each pharmaceutical company have their unique interpretations.
![Page 46: Social Media Monitoring to Identify Risk](https://reader031.fdocuments.in/reader031/viewer/2022022001/58f0622d1a28abef248b45d7/html5/thumbnails/46.jpg)
This work is licensed under the Creative Commons
Attribution-ShareAlike 4.0 International License.
To view a copy of this license, visit
http://creativecommons.org/licenses/by-sa/4.0/
Slides available at @epidemico