Show-Me ECHO: Project Evaluation

Mirna Becevic, PhD, MHAAssistant Research ProfessorDepartment of DermatologyMissouri Telehealth Network

Show-Me ECHO• Extension for Community Health Care Outcomes (ECHO) is a guided practice

model utilizing disruptive innovation technologies

• Guided practice: Practicing new learning under direct supervision (hear/see/do)1

• Disruptive innovation technologies: using complicated products and services (telehealth) in a new, simpler way (telementoring)2

1Dr. Madeline Hunter’s model of mastery learning http://www.onetohio.org/library/Documents/Dr%20Madeline%20Hunter%20Article1.pdf

2Hwang, Jason, and Clayton M. Christensen. "Disruptive innovation in health care delivery: a framework for business-model innovation." Health Affairs27.5 (2008): 1329-1335

Show-Me ECHO

Show-Me ECHO

Demonstrations

CME didactics

Case-based learning

Show-Me ECHOObjective:

To educate rural health clinicians in a specific disease state in order to fill considerable gaps in expertise, and expand access to specialty care for rural and underserved Missourians3

3Katzman, J. G., Comerci, G., Boyle, J. F., Duhigg, D., Shelley, B., Olivas, C., Daitz, B., Carroll, C., Som, D., Monette, R., Kalishman, S. and Arora, S. (2014), Innovative Telementoring for Pain Management: Project ECHO Pain. J. Contin. Educ. Health Prof., 34: 68–75. doi: 10.1002/chp.21210

Show-Me ECHO

Evaluation methods:

1. Evaluation of aggregate attendee data

2. Assessment of participating provider practice change (Medicaid claims data)

3. Participating provider self-efficacy (pre-post)

4. Anonymous CME evaluations

1. ENDO ECHO evaluation example

• 132 board certified endocrinologists

• 62 of those in St. Louis area

2. ENDO ECHO evaluation example

Map of board certified endocrinologists

Prevalence of diabetes in Missouri, 20114

4Missouri Diabetes Report 2015, MO HealthNet Division and DHSS

2. ENDO ECHO evaluation example

Blue: endocrinologists

Orange: ENDO ECHO participants

1. Evaluation of aggregate attendee data (ENDO ECHO)

Measure NumberTotal sessions 25Total attendees 244Individual attendees 89Case presentations 69 (67 new, 2 follow-up)CME credits awarded 16.5 hours (MDs and Dos)

1. Evaluation of specific topics (ENDO ECHO)

Thyroid block

(3 topics)

• Case presentations

• Presenter

Diabetes block 1

(5 topics)

• Case presentations

• Presenter

Diabetes block 2

(3 topics)

• Case presentation

• Presenter

Fall curriculum

2. Medicaid claims data

Objective: To educate rural health clinicians in a specific disease state in order to fill considerable gaps in expertise, and expand access to specialty care for rural Missourians3

Outcomes:

• Provider behavior change (prescription patterns), pre-post;

• Change in patient specific type (diagnoses), pre-post

• Change in patient behavior (ED visits, hospitalizations, outpatient visits), pre-post

2. Medicaid claims data

• Baseline for all ECHOs, 2014.

2. Medicaid claims data – ENDO ECHO example

PCP1 PCP2 PCP3 PCP4 Control PCP

Number of sessions attended

X XX Y YY 0

Number of cases presented

X XX Y YY 0

Type of case presentation (DX)

X XX Y YY 0

Number of sessions withoutcases presented

X XX Y YY 0

Didactic topic A attended

X XX Y YY 0

Didactic topic B attended

X XX Y YY 0

Number of patients with DX A

X XX Y YY Z

Number of Drug A prescriptions

X XX Y YY Z

3. PCP self-efficacy• ECHO intervention (education) may influence provider

decision making, which may affect their capacity to see certain types of patients.

• Self-efficacy is based on the social cognitive theory: unless people believe that they can achieve something, they will lack motivation to do it, and doubt that they can make difference by taking or changing their actions3.

• Primary care physician self-efficacy is associated with self-reported preventative care delivery in adult and pediatric primary care setting4

3Bandura, A. (2010). Self-Efficacy. In Corsini Encyclopedia of Psychology. 1-3

4Ozer, E., Adams, S., Gardner, L., et al (2004) Provider self-efficacy and the screening of adolescents for risky health behaviors. Journal of Adolescent Health. 35(2), 101-107

3. PCP self-efficacy

• Pre-Post survey

3. PCP self-efficacy – ENDO ECHO example• Observed decrease in self efficacy in some cases

• High pre-ECHO self efficacy (confidence in screening of patients with diabetes, glucose self-monitoring, etc), much lower post-ECHO self efficacy

Knowing what you don’t know is more important than knowing what you think you know.

4. CME surveys

• 5-point Likert scale

• Follow standard CME wording, and adjusted to ECHO specific terminology

Asthma – preliminary results

Asthma prevalence in Missouri ("Asthma in Missouri," [Online]. Available: https://www.cdc.gov/asthma/stateprofiles/asthma_in_mo.pdf. [Accessed 12 January 2017].) and map of counties analyzed

Asthma preliminary results

Average daily cost of asthma visit

Number of visits saved

Cost-effectiveness

$850 (in-patient) 1,600 $1,360,000

$500 (ED) 1,173 $586,500

TOTAL 2,773 $1,946,500

Estimated average cost by MO HealthNet

Asthma preliminary results

Thank you!