The CIELO Project: Towards a Research Analytics Commons

Philip R.O. Payne, PhD, FACMI

Professor and Chair, College of Medicine, Department of Biomedical InformaticsProfessor, College of Public Health, Division of Health Services Management and Policy

Associate Director for Data Sciences, Center for Clinical and Translational ScienceExecutive-in-residence, Office of Technology Commercialization and Knowledge Transfer

2

Overview

1) Background and Motivation

2) Conceptualization of CIELO

3) Functional and Technical Architecture

4) Future Directions

5) Today’s Objectives

6) Discussion

3

Overview

1) Background and Motivation

2) Conceptualization of CIELO

3) Functional and Technical Architecture

4) Future Directions

5) Today’s Objectives

6) Discussion

Critical Dimensions of a Learning Healthcare System: Systems Thinking Applied to Patient Centered Research

4

Environment and Culture

• Instrumenting the clinical environment

• Generating hypotheses

• Creating a culture of science and innovation

Precision Medicine

• Rapid evidence generation cycle(s)

• ‘omics’• Analytics/decision

support

Data Science

• System-level analyses• Data science• Visualization• Reproducible

analytics

Integrated and High Performing

Healthcare Research and Delivery Systems

Learning from every

patient encounter

Leveraging the best

science to improve care

Identifying and solving

complex problems

Rapid Translation

Our Focus!

5

The Need for Reproducible Research

Creating a high performing healthcare research and delivery system requires both economies of scale and increased efficiencies Timeliness Resource utilization Data “liquidity”

Central to this argument is a need to exchange research findings and evidence between and among stakeholders in a consumable manner Design Data Analysis

Doing so allows for reproducible research with cumulative benefits

6

Why is it Hard to Reproduce Research?

Data sharing alone is insufficient to this task How was data pre-processed? What analytical workflows were utilized? What additional parameters influenced data analysis? How were results “packaged” for dissemination?

Many socio-technical barriers to addressing these questions, including: Intellectual property and data-level concerns Availability of technology platforms/tools Documentation Metadata Standards Many, many other issues…

7

A Community Dialogue

8

BD2K and the Vision for a Research Commons

Phil Bourne’s Vision (Associate Director for Data Science, NIH) “To foster an ecosystem that enables biomedical

research to be conducted as a digital enterprise that enhances health, lengthens life and reduces illness and disability”

Creation of a commons providing for: Cloud infrastructure for data and computing Search Security Reproducibility standards App store

Source: Phil Bourne, “Ask Not What the NIH Can Do For You; Ask What You Can Do For The NIH”

9

Source: Phil Bourne, “Ask Not What the NIH Can Do For You; Ask What You Can Do For The NIH”

10

Overview

1) Background and Motivation

2) Conceptualization of CIELO

3) Functional and Technical Architecture

4) Future Directions

5) Today’s Objectives

6) Discussion

11

Translating a Problem into a Solution: The Problem Definition Process

Establish the Need for a Solution

Justify the Need

Contextualize the Problem

Write the Problem

Statement

Adapted from: Spradlin, “Are You Solving The Right Problem?”, HBR, September 2012

12

CIELO: Enabling Collaborative Data Analytics in Patient-Centered Research

Project Goals:1) Provide members of the research community with

access to an open-source/-standards “app store” for data analysis and software sharing

2) Reduce time and cost of research while enhancing the reproducibility and transparency of data analysis.

3) Evolve and meet emerging community needs

Blue-Sky: not grounded in the realities of the present: visionary <blue–sky thinking> (Merriam Webster Dictionary)

13

Process To-Date for CIELO

Ideation

MVP Development

Stakeholder Review and

Requirements Gathering

MVP Re-Engineering

Process and Outcomes Measure

Team formation and

proposal development

CIEHLO Prototype

Review andFeedback fromStakeholders

IterativeUser-Centered

Design

You Are Here!

Contextualizes

14

CIELHO Conceptual Model

15

Overview

1) Background and Motivation

2) Conceptualization of CIELO

3) Functional and Technical Architecture

4) Future Directions

5) Today’s Objectives

6) Discussion

16

Building on Existing Tools and Approaches

Sharing of Technical Artifacts

Social NetworkingMetadata

Github/Gitlab

Activity FeedsDiscussion Forums

FolksonomySemantic Search

Partitioning of access Bundling code and data Data model harmonization Cross-linkage (URIs/APIs)

Project-level feeds Linkage to metadata

Current ontologies Linkage to social functions

17

CIELHO Workflow Model

18

Community-Defined Requirements

Integration with analogous platforms and tools Ex. Sage Bionetworks Synapse

Incorporation of data security/confidentiality controls Particularly in the context of analyses involving PHI or similarly privileged data

sets

Convergence towards common data model for submission and reuse of data sets

Ex. OMOP

Multi-tiered sharing model Open access Limited access Private (for defined collaborators)

Semantic search and discovery of code and data

Connectivity to linked open data sets

Social networking at a project and individual level

Community-Defined Requirements: Focus for Public Beta Integration with analogous platforms and tools

Ex. Sage Bionetworks Synapse

Incorporation of data security/confidentiality controls Particularly in the context of analyses involving PHI or similarly privileged data

sets

Convergence towards common data model for submission and reuse of data sets

Ex. OMOP

Multi-tiered sharing model Open access Limited access Private (for defined collaborators)

Semantic search and discovery of code and data

Connectivity to linked open data sets

Social networking at a project and individual level

19

20

How Will We Evaluate the CIEHLO?

21

Overview

1) Background and Motivation

2) Conceptualization of CIELO

3) Functional and Technical Architecture

4) Future Directions

5) Today’s Objectives

6) Discussion

22

Future Directions: Shared Execution Environment (VAULT)

23

Overview

1) Background and Motivation

2) Conceptualization of CIELO

3) Functional and Technical Architecture

4) Future Directions

5) Today’s Objectives

6) Discussion

24

Meeting Objectives (1)

Provide a cross-section of stakeholders with a review of current technical functionality and design decisions surrounding the CIELO platform;

Identify needs for future functional/technical extensions to the platform, with a particular emphasis on:1) Shared analytic tool execution environments and

mechanisms2) Data model promotion/harmonization3) Crowd-sourced feedback and rating systems4) Minimum standards for “bundle” population (code

and example data)

25

Meeting Objectives (2)

Identify socio-cultural barriers and opportunities as they relate to creating and sustaining a CIELO user community;

Identify opportunity to promote and fund the ongoing development and adoption of CIELO as it can be positioned as a solution to enhancing research credibility and reproducibility.

26

Meeting Deliverables (1)

Stakeholder verification/validation of current CIELO functionality;

An enumeration and prioritization of future functional needs/requirements;

An enumeration of socio-cultural barriers and opportunities as they related to the creation and sustainability of an adopter/adapter community;

27

Meeting Deliverables (2)

An enumeration of communication, advocacy, and funding targets intended to position CIELO as a solution to enhancing research credibility and reproducibility;

An enumeration of targeted end-users and their communities;

A whitepaper and project plan that formalizes all of the preceding deliverables and provides a “roadmap” for future CIELO development and dissemination efforts.

28

Overview

1) Background and Motivation

2) Conceptualization of CIELO

3) Functional and Technical Architecture

4) Future Directions

5) Today’s Objectives

6) Discussion

29

“Information liberation + new incentives = rocket fuel for innovation” – Aneesh Chopra (The Advisory Board Company)

Philip R.O. Payne, PhD, FACMIphilip.payne@osumc.edu

"Without feedback from precise measurement, invention is doomed to be rare and erratic. With it, invention becomes commonplace” – Bill Gates (2013 Gates Foundation Annual Letter)

“Data is beyond simply quantifying, it is seeing measurement as the intervention” – Carol McCall (GNS Healthcare)