Cochrane Present TechCochrane Future Tech

Ida Sim, MD, PhDUniversity of California San FranciscoOpen mHealth

September 17, 2013

• In 2003, estimate of 10-45,000 reviews needed to cover existing evidence as of 2003– projected Cochrane to hit 10,000 reviews between 2010 and

2015

Mallet and Clarke, EBM 2003(8):100-1

Total Protocols

Total Reviews

Total Updated Reviews1000th Cochrane review

5665 total reviews today

How can we better leverage technology and knowledge to both help us prepare systematic reviews more efficiently but also deliver the outputs better to our

end-users?

Setting the Stage: PICOT

P opulationI nterventionC omparisonO utcomeT ime frame

Intervention

P I Synthesized evidence, the "Cochrane Way"

C O T

(Target) Population

P Health care systems, clinicians

I Synthesized evidence, the Cochrane Way

C O T

Outcome

P Health care systems, clinicians

I Synthesized evidence, the Cochrane Way

C O Population-level health outcomes & costs

T

Comparison

P Health care systems, clinicians

I Synthesized evidence, the Cochrane Way

C Eminence-based medicine

O Population-level health outcomes & costs

T

Time frame

P Health care systems, clinicians

I Synthesized evidence from the Cochrane Way

C Eminence-based medicine

O Population-level improvement in health, costs

T Too many years

T

Setting the Present Stage

Cochrane Present

P health systems, clinicians

I Cochrane Way C eminence-based med

O pop-level health & cost

T too many

Cochrane Future

P I C O T

Cochrane Way: Workflow

Technology for Pain Points

Setting the Future Stage

Cochrane Present

P health systems, clinicians

I Cochrane Way C eminence-based med

O pop-level health & cost

T too many years

Cochrane Future

P I Cochrane Way C O T

Additional Outcomes

Cochrane Present

P health systems, clinicians

I Cochrane Way C eminence-based med

O pop-level health & cost

T too many years

Cochrane Future

P I Cochrane Way C O cost

T

Chronic Diseases Drive Cost

• 46% of morbidity and 59% mortality worldwide due to chronic diseases1

• Health systems targeting chronic care– 1/3 of deaths due to poor health behaviors– need to engage patients in self-care

• Patients expect personalized medicine– want evidence at the individual-patient level

WHO | Facts related to Chronic Diseasehttp://www.who.int/dietphysicalactivity/publications/facts/chronic/en/

Additional Outcomes and Population

Cochrane Present

P health systems, clinicians

I Cochrane Way C eminence-based med

O pop-level health & cost

T too many years

Cochrane Future

P add patients & families

I Cochrane Way C O add ind-level health

T

Time Frame

• Personal digital technologies will play large role in chronic disease and transforming health– 20,000 health apps on iTunes, 8,000 on Google

Play– technologies evolve rapidly

• US Institute of Medicine goal of a continuous Learning Health System

Riley et al. Clinical and Translational Medicine 2013, 2:10

New Time Frame

Cochrane Present

P health systems, clinicians

I Cochrane Way C eminence-based med

O pop-level health & cost

T too many years

Cochrane Future

P add patients & families

I Cochrane Way C O add ind-level health

T continuous

New Comparison Intervention

Cochrane Present

P health systems, clinicians

I Cochrane Way C eminence-based med

O pop-level health & cost

T too many years

Cochrane Future

P add patients & families

I Cochrane Way C Big Data

O add ind-level health

T continuous

Three Illustrative Projects

• N-of-1 studies for chronic pain• Kaiser: Diabetes and depression

management• Health eHeart virtual cohort

PREEMPT Project

• Chronic Pain is highly prevalent (>100 million Americans) and difficult to treat

• Few studies on comparative effectiveness of analgesics, yielding only average population-level estimates

R01-NR013938, PI R. Kravitz

50 people

100 people

oxycodone

Pain frequency, intensity

50 people

hydrocodone

population

Pain frequency, intensity

none of us are average

hydrocodone betteroxycodone better

which works better for you?

N-of-1 study design: within-subject crossover design

Kravitz, et al. Contemp Clin Trials 2009; 30:436-445

BPI

individual

pain intensity

Youpain intensity

oxycodone

hydrocodone

hydrocodone

oxycodone

oxycodone

hydrocodone

there you are!

hydrocodone betteroxycodone better

n = 1

(n = 1).N

Zucker DR et al. J Clin Epidemiol. 2010;63(12):1312-23.

(n = 1).NΣ

flip direction of research inference

population

Differences from Traditional Research

• Occurs in the course of clinical care • Patient participation, addresses

patient questions• Patient choice in outcomes measured• Individual treatment effect• Aggregate to population-level effect

S. California Kaiser Complete CareImproving self-management of diabetes in patients with depression

Continuous Evaluation

• eHealth interventions will play a substantial role in chronic care management and in shaping health care systems

• Evaluations need to occur while they are being designed, developed, and deployed.

Catwell and Sheikhl, PLoS Med 2009; 6(8):e1000126

Usability Studies

Preliminary effectiveness

User Requirements

Pilot Field Testing

Rigorous effectiveness

Differences from Traditional Research

• Evidence is needed on intermediate non-clinical outcomes (e.g., effective design features) as well as end clinical effectiveness

• Many of these evaluations will not be published in academic journals

• Enrolling 1,000,000 patients, capturing– self report data: food intake, mood– sensor data: weight, BP, activity– social data: Facebook– EHR data: including text mining for CV events– biospecimens and CV tests for San Francisco area

patients

• Trades precise data on fewer patients for messier data on 1000x more patients– will be supplemented with targeted data

collection for specific studies

Future Cochrane Way?

Cochrane Present

P health systems, clinicians

I Cochrane Way C eminence-based med

O pop-level health & cost

T too many years

Cochrane Future

P add patients & families

I Cochrane Way? C Big Data

O add ind-level health

T continuous

"A good hockey player plays where the puck is. A great hockey player plays

where the puck is going to be." Wayne Gretsky, Edmonton Oilers

Where the Puck is Today

Studies in PDF Forest plots in PDF

Cochrane Way

Draws the most sound inference from the totality of the available evidence

Where the Puck will be

Evidence = study protocol + results

Synthesized evidence

continuous, multi-modal, personal data

more personalized questions

large, less controlled studies

individual-level evidence

range of acceptable rigor

continuous release schedule

Cochrane Way

Drawing the most sound inference from the totality of the available evidence

Build to Basic Principles

• Data sciences• Operational• Methodological

Data Sciences

• Beyond PDF reports of study results– embrace data publishing

• publish protocols as computable models

• publish all results as open computable data

– text mining to extract meaning from prose

• No data silos: linked open data

• Describe the data: metadata and ontologies for– study questions

– study design (Ontology of Clinical Research)

– systematic reviews (Cochrane Ontology)

Data Sciences Goal

• Capture open data in clearly described computable form that can be re-purposed for multiple needs today, and unknown needs tomorrow

Operational

• Pursue hybrid semi-automated approaches– seek out and work with the best

designers of human-computer interaction

• Support distributed, collaborative knowledge work, tapping into crowds

Methods Needed

• Heterogeneity of treatment effect• Continuous evaluation methods• Large-scale assessment of biases

and confounders• Better decision support for assessing

biases and confounders• ...many more

The "Human Evidence Project"

+ + MethodsData

Metadata+

Most sound individual and population-level summary evidence for continuous learning

Conclusion

• How to get from Cochrane Present to Cochrane Future?– new methods built on open data and

ontologies– design hybrid human-computer systems

• In new Big Data world, methodologically sound but pragmatic, sustainable evidence synthesis will be critical

• Need Cochrane now more than ever