Evaluation in an Era ofDigital Technology Innovation

Reflections for Philanthropy

Eric C. Schneider, MD, MSc, FACPSenior Vice President for Research and Policy

@ericschneidermd

Outline

• Challenges in evaluating digital health technology • A human-centered measurement approach• Concluding thoughts

Evaluating digital technology:Are these devices?

• Clinical trials testing • Five-phase model• Reliance on randomized, controlled trials• Highly-selected patient samples• Clinical outcomes/surrogate biological markers• 12-18 years from lab to marketing

Digital health technologies: Many value propositions

• Increase patient engagement• Close communication gaps between patient and providers• Improve providers’ ability to tailor services to population

segments• Enable convenient and cost-effective care delivery• Improve decision making

4

M. Hostetter, S. Klein, and D. McCarthy, Taking Digital Health to the Next Level: Promoting Technologies That Empower Consumers and Drive Health System Transformation, The Commonwealth Fund, October 2014.

Innovation on Three Different Time HorizonsUnknown

Known

Partially known

Existing Business Model: Process

InnovationExecute

New/Disruptive Business Model

Search

New Opportunities via Business Model

InnovationExecute/Search

Source: Steve Blank, Lean Innovation Management – Making Corporate Innovation Work

Digital technology innovation is unpredictable

Dependencies: VIVIFY Health and CHRISTUS St. Michael Health

• Goal: Reduce 30-day readmissions and reduce Care Transition Nurse case load

• Implement a remote patient monitoring solution• Transmits patient data using personal health devices

(tablets, weight scales, blood pressure cuffs, etc.)• If patient data indicates a health concern, device alerts

Care Transition Nurse via text/email

Measurement Myopia:VIVIFY Health Evaluation Results• Pilot study: patients with chronic congestive heart failure

who consented to participate in care transition program• Results

• 44 patients• Decrease in cost of care from $12,937/per patient to $1,231/per

patient• ROI of $2.44, including cost of technology

• A winner?

Evaluating Smarter:VIVIFY Health’s Qualitative Insights•Connectivity issues

• Patients outside of cellular service areas

•Equipment issues• Tablets overheated

•Bugs!• Packaging promoted egg-laying

Ambar Kulshreshtha JCK, Abhinav Goyal, Elkan F. Halpern, and Alice J Watson. Use of Remote Monitoring to Improve Outcomes in Patients with Heart Failure: A Pilot Trial. International Journal of Telemedicine and Applications. 2010;2010(870959):7

A New Approach: Human-Centered Designwww.sagehealthadvisor.com

EMOTIONAL NEEDS

PERSONALNEEDS

FUNCTIONAL NEEDS

MEDICALNEEDS

Measuring the impact of digital health technologies

Technology•Feasibility•Functionality•Unexpected

Bugs

Workflow•Take up

•Use Patterns

•Stickiness

•Safety

•Unintended Consequences

•Costs

Clinical•Adherence to

Treatment•Use of Health

Services•Quality of

Care•Patient

Engagement•Patient

Experience•Provider

Experience

Person/Life

•Social Relationships

•Quality of Life

•Caregiver experience

•Self-Efficacy

•Functioning

•Health Status

•Emotional Status

Conclusion: Evaluation Approach

• Start with a “person/life” holistic view

• Assess technology compatibility with human-centered design principles

• Beware of workflow and adjacent technologies

• Expect the unexpected!

References1. Jones SS, Heaton PS, Rudin RS, Schneider EC. Unraveling the IT productivity

paradox--lessons for health care. N Engl J Med. 2012;366(24):2243-2245.2. HIMSS. Case Study: Decreasing Costs and Improving Outcomes Through

Community-Based Care Transitions and Care Coordination Technology Healthcare Information and Management Systems;March 2014.

3. Broderick A. Partners HealthCare: Connecting Heart Failure Patients to Providers Through Remote Monitoring. Online: The Commonwealth Fund;2013.

4. Ambar Kulshreshtha JCK, Abhinav Goyal, Elkan F. Halpern, and Alice J Watson. Use of Remote Monitoring to Improve Outcomes in Patients with Heart Failure: A Pilot Trial. International Journal of Telemedicine and Applications. 2010;2010(870959):7.

5. Rudin et al. The value of health information technology: filling the knowledge gap. Am J Managed Care. 2014; eSP1-eSP8.

12