Research Data Analytics at Thomas Jefferson University Jack London, PhD Thomas Jefferson University...
-
Upload
millicent-stafford -
Category
Documents
-
view
227 -
download
1
Transcript of Research Data Analytics at Thomas Jefferson University Jack London, PhD Thomas Jefferson University...
Research Data Analytics at Thomas Jefferson University
Jack London, PhDThomas Jefferson University
Sidney Kimmel Cancer CenterPhiladelphia PA USA
2015 i2b2 European Academic User Group meeting
October 6, 2015
2
Disclaimer
In addition to my faculty position at Thomas Jefferson University in Philadelphia, I am a consultant for TriNetX Corporation.
3
Thomas Jefferson University and the Sidney Kimmel Cancer Center (SKCC), Philadelphia
Located between New York City and Washington DC
Jefferson Medical College (JMC) was founded in 1824.
JMC is the second largest private medical school in the U.S.
The NCI-designatedSKCC has ~ 400 physicians and scientists
dedicated to discovery and development of novel approaches for cancer
treatment.
4
SKCC’s IT infrastructureGE Centricity inpatient EMR
Allscripts outpatient (ambulatory care) EHR
EPIC inpatient and outpatient
Cerner A/P lab system EPIC Beaker
OpenSpecimen research biobank management
TIES clinical text extraction
i2b2 research data mart
TriNetX data analytics network
5
Current Jefferson Data Resource Landscape
TJUH CLINICAL DATA WAREHOUSE
DEMOGRAPHICS (gender, race, age, vital status, ethnicity)
DIAGNOSES (ICD9)
PROCEDURES (ICD9)
CLINICAL LABS (LOINC)
MEDICATIONS
TJUH CLINICAL DATA WAREHOUSE
DEMOGRAPHICS (gender, race, age, vital status, ethnicity)
DIAGNOSES (ICD9)
PROCEDURES (ICD9)
CLINICAL LABS (LOINC)
MEDICATIONS
i2b2 RESEARCH DATA MARTi2b2 RESEARCH DATA MART
IMPAC METRIQcancer registry site, stage, histology,
treatment, survival (ICD-O-3)
IMPAC METRIQcancer registry site, stage, histology,
treatment, survival (ICD-O-3)
CERNER A/P“omic” data
CERNER A/P“omic” data
FORTE ONCOREclinical trial data
FORTE ONCOREclinical trial data
OPEN SPECIMENbiospecimen annotation (SNOMED)
OPEN SPECIMENbiospecimen annotation (SNOMED)
6
Jefferson’s i2b2 Research Data Mart
• Built on “informatics for integrating biology and the bedside” (i2b2) version 1.7.02
• RDM data are de-identified. Re-identification possible via an honest broker, who has access to a re-identification application.
• Currently > 45 million observations on > 450,000 patients. Data refreshed weekly.
7
Patient data obtained from TJUH EMRDEMOGRAPHICS
Age
Ethnicity
Gender
Race
Vital Status (alive/dead)
DIAGNOSES
Disease systems --> diseases (organized by ICD9 coding)
CLINICAL LAB RESULTS
Chemistry
Coagulation
Hematology
MEDICATIONS
Anti-neoplastic
INPATIENT PROCEDURES
Diagnostic and Treatment procedures (organized by ICD9 coding)
8
Patient mutation data obtained from Pathology Molecular Diagnostic Testing (both outsourced and in-house)
ALK rearrangement
BRAF c.1782T>G p.D594E
BRAF c.1801A>G p.K601E
BRAF c.1799T>A p.V600E
EGFR Deletion in exon 19
EGFR Insertion in exon 20
EGFR c.2236G>A p.E746K
EGFR c.2236_2250del15
p.E746_A750delELREA
EGFR c.2156G>C p.G719A
EGFR c.2155G>T p.G719C
EGFR c.2155G>A p.G719S
EGFR c.2573T>G p.L858R
EGFR c.2582T>A p.L861Q
EGFR c.2303G>T p.S768I
JAK2 c.1849G>T p.V617F
JAK3 c.2164G>A p.V722I
KRAS c.35G>C p.G12AKRAS c.34G>T p.G12CKRAS c.35G>A p.G12DKRAS c.34G>C p.G12RKRAS c.34G>A p.G12SKRAS c.35G>T p.G12VKRAS c.38G>A p.G13D
NRAS c.183A>T p.Q61HNRAS c.181C>A p.Q61KNRAS c.182A>T p.Q61LNRAS c.182A>G p.Q61R
PIK3CA c.1633G>A p.E545KPIK3CA c.3140A>T p.H1047LPIK3CA c.3140A>G p.H1047R
PTEN c.754G>T p.D252YPTEN c.59G>A p.G20E
RET rearrangement
ROS1 rearrangement
SMAD4 c.1157G>A p.G386D
TP53 c.843C>A p.D281ETP53 c.811G>T p.E271*TP53 c.857A>C p.E286ATP53 c.400T>C p.F134LTP53 c.734G>A p.G245DTP53 c.388C>G p.L130VTP53 c.524G>A p.R175HTP53 c.817C>T p.R273CTP53 c.818G>A p.R273HTP53 c.318C>G p.S106RTP53 c.659A>G p.Y220CTP53 c.707A>G p.Y236C
9
Molecular Diagnostics ontology
10
Specimen annotation from campus biobanks
Anatomic origin (SNOMED)
Class (tissue, fluid)
Type (frozen, FFPE)
Pathology (normal, malignant, diseased)
Slide images
Eight biobanks, including the TJUH paraffin block archive of ~400,000 cases since 1990.
11
Specimen annotation management
TJUH clinical paraffin block archive
Pathology Department research tissue bank
Brain tumor bank(J. Evans, PI)
Pancreatic tumor bank(C. Yeo, PI)
Breast tumor bank(J. Palazzo, PI)
Thyroid tumor bank(E. Pribitkin, PI)
Brain tumor bank(D. Andrews, PI)
Liver tumor bank(V. Navarro, PI)
JJJjj
jj
Jefferson integrated Research Specimen
management (OpenSpecimen)
> 230,000 patients > 650,000 specimens
> 100,000 patients
via i2b2 RDM
Cancer patients having comprehensive annotation from the Tumor Registry and banked
specimens
12
Biospecimen ontology
13
Pathology images are available via i2b2 query tool
14
Patient data from Jefferson Tumor Registry
Primary Cancer Diagnosis
Age at diagnosis/date of diagnosis
Survival (months) from diagnosis
Tumor histology and behavior
Stage (AJCC/TNM, clinical and pathological)
Grade
Recurrence
local, distant
Treatment
chemotherapy, radiation, surgery, transplant, palliative
Disease-specific factors
ex: (prostate --> Gleason score)
Over 100,000 cases since 1990.
15
Tumor Registry ontology
16
Typical SKCC Investigator Queries
Example #1:
Form cohort of “triple negative” (estrogen receptor, progesterone receptor, and her2 negative), African American patients, having matched normal and malignant frozen tissue specimens.
Example #2:
Form cohort of patients with a primary diagnosis of papillary thyroid cancer, and expressing a V600E BRAF mutation.
17
Additional data on selected cohort can be retieved
18
Example data summaries from the i2b2 RDMCLINICAL DIAGNOSES OF TJUH PATIENTS WITH THYROID SPECIMENS
19
Jefferson – TriNetX project
In the fall of 2014, the SKCC informatics group entered into a collaboration with a Cambridge, Massachusetts based start-up company, TriNetX, Inc.
TriNetX facilitates collaboration between pharmaceutical companies and academic healthcare providers through the creation of a global, federated data network that connects academic and industry clinical researchers in real-time to the patient populations they are attempting to study.
The TriNetX applications accesses a site’s i2b2 database, and displays aggregate query results in an advanced, flexible manner.
20
TriNetX application offers an alternative query tool with enhanced data visualization
Google-like query interface
Graphic result display
21
TriNetX application offers an alternative query toolwith enhanced data visualization
Interactive display capability
22
Cohort definition via i2b2 can be used to predict accrual for proposed clinical trials
23
Problem confronting clinical trials research: studies that fail to accrue
An Institute of Medicine report1 on cancer cooperative group trials found that 40% were never completed because of failure to achieve minimum accrual goals:
“The ultimate inefficiency is a clinical trial that is never completed because of insufficient patient accrual, and this happens far too often.”
These non-accruing trials are often kept open for many months before closure, consuming personnel resources in their setup and operation at a significant cost to institutions, without providing any return in definitive research findings.
Furthermore, while many of these trials register zero patients, others accrue some patients, resulting in thousands of patients nationwide who are recruited to unproductive research studies.2
1. Nass SJ, Moses HL, Mendelsohn J, editors. Committee on Cancer Clinical Trials and the NCI Cooperative Group Program Board on Health Care Services; A National Cancer Clinical Trials System for the 21st Century: Reinvigorating the NCI Cooperative Group Program. Washington DC: National Academies Press, 2010.
2. Cheng, S., M. Dietrich, S. Finnigan, A. Sandler, J. Crites, L. Ferranti, A. Wu, and D. Dilts. A sense of urgency: Evaluating the link between clinical trial development time and theaccrual performance of CTEP-sponsored studies. 2009 ASCO Annual Meeting Proceedings. J of Clinical Oncology, 2009.
24
Study design
The overall objective of this study was to evaluate whether accrual for proposed cancer clinical trials could be predicted by performing cohort queries that are based on the trial’s eligibility criteria on recent patient data in Jefferson’s i2b2 research data mart (RDM), created from de-identified integrated hospital clinical, tumor registry, and specimen data.
To determine the ability of the i2b2 RDM to predict accrual for prospective trials, we retrospectively used the RDM to obtain patient populations for two years prior to recent trials and compared these cohort sizes to the actual accrual observed after the trial was opened. We considered 90 interventional cancer trials opened at KCC in the years 2008, 2009, and 2010, since these have been open for at least two years and their accrual performance could be evaluated.
25
Study methodology
o We constructed RDM cohort queries corresponding to the trial eligibility criteria for the two years prior to each trial’s opening (e.g., we considered TJUH patient populations from 2007 and 2008 for trials opened in 2009).
o We computed an annual cohort size by averaging the 2-year totals.
o We then compared our RDM annual cohort size for the 2 years preceding a trial’s opening to the annual target goal for that trial and the trial’s actual accrual performance.
• Since we initially assumed that 50% of eligible participants would enroll in a study, the RDM cohort would have to be at least twice the accrual goal for a prediction of “successful” trial accrual.
• We defined a trial’s actual accrual performance as “successful” if it accrued at least 80% of its target enrollment.
26
Results
To assess the predictive precision of our proposed project, a contingency table was produced for the 90 trials analyzed.
•A trial was denoted as potentially successful in meeting its annual target accrual (“PREDICTED SUCCESS” row) if the retrospective i2b2 cohort analysis indicated sufficient patients for the trial.
•A trial was denoted as actually successful in meeting its annual target accrual if the trial satisfactorily approached the protocol’s stated target annual accrual (“ACTUAL SUCCESS” column).
Contingency table comparing i2b2 accrual predictions with actual accrual success, assuming only 50% of potential participants identified by i2b2 are enrolled.
Our methodology has 0.969 (= 31/32 trials) accuracy (95% C.I. (0.908, 1)) for predicting successful accrual (i.e. specificity) and 0.397(= 23/58 trials) accuracy (95% C.I. (0.271, 0.522)) for predicting failed accrual (i.e. sensitivity). The positive predictive value, or precision rate, is 0.958 (= 23/24 trials) (95% C.I. (0.878, 1)).
27
Results
Our results show that the methodology, while having an excellent positive predictive value (95.8%, predicted failure for 23 of the 24 trials that actually failed ), is not good at predicting failed accrual (39.7%, 23/58 trials).
In other words:
if the methodology predicts "failed accrual," then we should trust this prediction and should not proceed to open the trial with its current eligibility criteria;
however, a prediction of accrual success using this method is no guarantee that target goals will be met.
28
How can this methodology be useful?
A benefit of analyzing potential trial accrual during the protocol design phase is that it offers an opportunity to “tweak” eligibility rules when insufficient patient cohorts are found.
A change in participation criteria that does not impact significantly on the scientific objectives of the trial may provide a sufficiently large potential patient pool.
Not opening the 23 trials that were correctly predicted to fail to accrue over the 3 years studied would have prevented the waste of about $200,000 in trial startup costs alone, and the participation of 57 patients in studies which did not contribute to advancing science or clinical care.
Selected areas of research using RDM: Hallgeir Rui, MD, PhD: Molecular Cancer Epidemiology, cancer pharmacogenetics, individualised cancer risk assessment and prognostication.
Raphael E. Bonita, MD: Jefferson Heart Institute, correlation of troponin levels and heart failure in transplant patients.
Hushan Yang, PhD: Molecular Cancer Epidemiology.
Jordan Winter, MD: Surgery, whipple procedure survival study.
Scott Waldman, MD, PhD: Pharmacology and experimental therapeutics.
Ron Myers, PhD: Gene environmental risk assessmant.
Stephen Peiper, MD: Biomarker discovery using Next Generation Sequencing.