Implementation of an Electronic Information System to Enhance Practice at an Opioid Treatment...
description
Transcript of Implementation of an Electronic Information System to Enhance Practice at an Opioid Treatment...
Implementation of an Electronic Information System to Enhance Practice at an Opioid Treatment Program (R01 DA022030)
Steven Kritz, MD; Carlota John-Hull, MD; Ben Louie, BA; Melissa Lin, MS; Roberto Zavala, MD; Charles Madray, RPA-C, MBA; Lawrence S. Brown, Jr., MD, MPH, FASAM
Division of Medical Services, Research and Information Technology, Addiction Research and Treatment Corp, Brooklyn, NY 11201
NIDA RFA-DA-06-001 (R01): Enhancing Practice Improvement in Community-Based Care for Prevention and Treatment of Drug Abuse
DISCLOSURES
• There are no financial interests or other disclosures to report for any of the authors involved in this project
• Grant support provided by NIDA: R01 DA022030
BACKGROUND Electronic information systems rarely
utilized or evaluated in substance abuse treatment settings
ARTC serves a racially, ethnically and economically disenfranchised population
ARTC serves a population that experiences significant disparities in access and quality of healthcare
PURPOSE• Evaluate the implementation of an
electronic information system using the following domains: – Quality – Risks– Satisfaction– Productivity– Finance
STUDY DESIGN Prospective, comparative study Pre-post implementation evaluation 3-year timeline
AIMS & HYPOTHESESSpecific AIM 1: Quality
Hypothesis - Improved capture or
timeliness of: Medical Assessments Multidiscipline
Assessments HCV Viral Load
Specific AIM 2: RisksHypothesis - Rates will
decrease for: Patient Complaints Patient Incidents Medication Errors
Specific AIM 3: SatisfactionHypothesis - Overall satisfaction will increase for:
Patients Clinicians Managers
AIMS & HYPOTHESES
Specific AIM 4: ProductivityHypothesis: Visits will increase for:
Counseling Primary Care HIV Case
Management
Specific AIM 5: Financial Performance
Hypotheses: Revenue per capita
staff will increase Cost per visit will
decrease
Aim Measure Data Source(Pre)
Data Source(Post)
Quality
% Medical assessments
performed on-time
Paper chart Electronic record
% Multidiscipline assessments
performed on-time
Paper chart Electronic record
% HCV Viral Loadobtained
Paper chart Electronic record
Risks# of Complaint /
Incidents / Medication Error
Reports
Reports to CQI Manager
Reports to CQI Manager
AIMS, MEASURES & DATA SOURCES
Aim Measure Data Source (Pre)
Data Source (Post)
Patient Satisfaction Survey SurveySatisfaction
Productivity
Clinician/Manager Satisfaction
Survey Survey
# Counseling visits Clinician logs Electronic logs# Primary care visits Clinician logs Electronic logs
# HIV case management visits
Clinician logs Electronic logs
FinancialRevenue per capita Finance/
HR DeptsFinance/
HR DeptsPerformance
Cost per visit Finance Dept Finance Dept
AIMS, MEASURES & DATA SOURCES
PRE-POST DATA ANALYSIS
Quality Chi-square exact test
Risks Cochran-ArmitageSatisfaction Chi-square testProductivity t-testFinance t-test
QUALITY: ANNUAL MEDICAL & MULTIDISCIPLINE ASSESSMENTS
Hypothesis: Implementation of the electronic health record will result in a higher percent of patients having annual medical assessments performed within 30 days of admission anniversary; and annual multidiscipline assessments performed on or before the admission anniversary
QUALITY: ANNUAL MEDICAL & MULTIDISCIPLINE ASSESSMENTS
Pre-implementation Eligibility: Admission date: 7/1/06 to 6/30/07Length of stay >365 days# of Eligible Patients: 420Post-implementation Eligibility: Admission date: 11/1/08 to 10/31/09Length of stay >365 days# of Eligible Patients: 423
QUALITY: STUDY POPULATION
PRE POST
Admission Date: 7/1/06 – 6/30/07 11/1/08 – 10/31/09
N 420 423 Mean Age: 45+/- 8.8 yrs 47+/-9.9
yrs (19-66) (20-78) Percent Female: 31% 30% Race/Ethnicity:
% Hispanic 52% 47% % African American 36% 41%
Measure Study Period# (%)
On Time# (%) Late + Not
Completed
P-value
AnnualMedical
Assessments*
Pre-implementati
on
350 (83%)
70 (17%)
<0.001Post- implementati
on
411 (97%)
12(3%)
Annual Multidisciplin
e Assessments
**
Pre-implementati
on
294(70%)
126 (30%)
<0.0001Post-
implementation
407 (96%)
16(4%)
QUALITY: ASSESSMENT RESULTS
* + 30 days of 1-year anniversary ** < 365 days after admission
QUALITY: 30-DAY & 90-DAY MULTIDISCIPLINE ASSESSMENTS
Hypothesis: Implementation of the electronic health record will result in a higher percent of patients having 30-day and 90-day multidiscipline assessments performed on or before the due date
QUALITY: 30-DAY MULTIDISCIPLINE ASSESSMENTS Pre-implementation Eligibility: Admission date: 7/1/06 to 6/30/07Length of stay >30 days # of Eligible Patients: 613Post-implementation Eligibility: Admission date: 10/1/09 to 9/30/10Length of stay >30 days # of Eligible Patients: 704
QUALITY: 90-DAY MULTIDISCIPLINE ASSESSMENTS Pre-implementation Eligibility: Admission date: 7/1/06 to 6/30/07Length of stay >90 days# of Eligible Patients: 576Post-implementation Eligibility: Admission date: 10/1/09 to 9/30/10Length of stay >90 days# of Eligible Patients: 608
Measure Study Period# (%)
On Time# (%) Late + Not
Completed
P-value
30-DayMultidisciplin
e Assessments
*
Pre-implementati
on
441 (72%)
172 (28%)
<0.001Post- implementati
on
614 (87%)
90 (13%)
90-Day
Multidiscipline
Assessments**
Pre-implementati
on
242 (42%)
334 (58%)
<0.001 Post-
implementation
423 (70%)
185 (30%)
QUALITY: ASSESSMENT RESULTS
* < 30 days after admission ** < 90 days after admission
QUALITY: HEPATITIS C VIRAL LOAD
Hypothesis: Implementation of the electronic health record will result in a higher percent of Hepatitis C antibody positive patients tested for hepatitis C viral load
Pre-implementation Eligibility: Admission date: 7/1/06 to 6/30/07 Length of stay: >60 days # of Eligible Patients: 670
Post-Implementation Eligibility: Admission date: 10/1/09 to 9/30/10 Length of stay: >60 days # of Eligible Patients: 653
Study PeriodHCV
Antibody Positive
(%)/Negative
(%)
Appropriately Referred +
Refused
HCV VL Done / HCV VL Done + Not Done
(%)
P-value
Pre-implementati
on
342 (51%) /328 (49%)
160 + 4 = 164
151/178 (85%)
NS
Post-implementati
on
296 (45%) /357 (55%)
212 + 5 = 217
64/79(81%)
QUALITY: HCV RESULTS
* HCV VL Not Done: 27 (pre); 15 (post)
RISK: MEDICATION ERRORS, PATIENT COMPLAINTS & PATIENT
INCIDENTS
Hypothesis: Implementation of the electronic information system will result in a lower number of Medication Errors, Patient Complaints & Patient Incidents
RISK: MEDICATION ERRORS, PATIENT COMPLAINTS & PATIENT
INCIDENTS Pre-implementation Period: 7/1/06 to 6/30/07 # of Medication Errors: 8 # of Patient Complaints: 15 # of Patient Incidents: 64
This domain was not included in the post-implementation data collection: - N relatively small - Processes did not change post-
implementation
SATISFACTION: PATIENT SURVEYS (6 QUESTIONS)Hypothesis: Implementation of the electronic health record will result in increased patient satisfaction
# of Surveys Administered (Pre & Post): 1,000 - Apportioned by Clinic Census
- Process: First come/first served with 2-ride MetroCard
Patient Survey:How long have you been with ARTC?
< 90 days 90 days-1 yr > 1 yr0
102030405060708090
100
8%
22%
70%
8%17%
75%
%
SATISFACTION: PATIENT SURVEY RESULTS
• Length of time with ARTC not different pre-post
• 35% completed pre & post-implementation surveys
• Mean Score & Standard Deviation: Q1-6 - Pre: Mean Score: 3.78* Std Deviation: 0.750 - Post: Mean Score: 3.74* Std Deviation: 0.775
*P=NS for each question & overall
Patient Survey: Q6: How satisfied are you with the overall quality of care you receive?
Not sat
isfied
Sligh
tly Sa
tisfie
d
Somew
hat S
atisfie
d
Satis
fied
Very
satisfi
ed0
20406080
100
3% 8% 15%
48%26%
3% 9% 16%
49%
23%%
PRE: Mean: 3.86 Std Deviation: 0.991 Range: 1-5POST: Mean: 3.80 Std Deviation: 0.999 Range: 1-5
SATISFACTION: STAFF SURVEYS (17 QUESTIONS)
Hypothesis: Implementation of the electronic health record will result in increased staff satisfaction
Eligible employees: Clinicians and Managers Pre Post
# of eligible employees: 148 155
# of surveys returned: 99 (67%) 92 (59%)
SATISFACTION: STAFF SURVEY RESULTS
• 54% completed pre & post-implementation surveys
• Mean Score & Standard Deviation: Q1-17 - Pre: Mean Score: 3.11* Std Deviation: 0.819 - Post: Mean Score: 3.32* Std Deviation: 0.728
*P=NS overall; P<0.03 for Q1, 7, 8, 11, 13Pre Mean Score > Post Mean Score for Q2 & 16 only (P=NS)
SATISFACTION: STAFF SURVEYS
Questions with significant post-survey satisfaction:
Q1: How satisfied are you with the ability to access needed reports or obtain information for needed reports? (P=0.008)Q7: How satisfied are you with the ability of the system to track your productivity and/or your staff? (P=0.005)Q8: How satisfied are you with the organization of the patient records and/or reports? (P=0.03)Q11: How satisfied are you that the patient record and/or management report format helps to prevent you from overlooking information? (P=0.03)Q13: How satisfied are you that you can communicate patient and/or administrative information to and from administrative staff? (P=0.03)
Staff Survey:Q6: How satisfied are you with the system overall?
Not sat
isfied
Sligh
tly Sa
tisfie
d
Somew
hat S
atisfie
d
Satis
fied
Very
satisfi
ed0
20406080
100
5% 14%
48%28%
5%5% 7%
46% 40%
2%%
PRE: Mean: 3.14 Std Deviation: 0.904 Range: 1-5POST: Mean: 3.26 Std Deviation: 0.841 Range: 1-5
PRODUCTIVITYHypothesis: Implementation of the electronic health record will result in increased visits per clinician for addiction counseling, primary medical care, and HIV-related case managementEligible staff: • Human Services Counselors• Medical Staff• Case Managers
Pre Evaluation Period: 7/1/06 to 6/30/07Post Evaluation Period: 11/1/09 to 10/31/10
PRODUCTIVITY Human Services Staff Addiction related counseling services (Pre):
64,345 Addiction related counseling services (Post):
52,652 Medical Services Staff Primary medical care services (Pre): 5,221 Primary medical care services (Post): 4,028 Case Managers HIV counseling services (Pre): 2,680 HIV counseling services (Post): 3,058
HUMAN SERVICES
CLINIC PRE POST CHANGE T-VALUE P-VALUESIGN
TEST P VALUE
11 10791 8652 -2139 -7.41 0.0003 0.01613 9984 8440 -1544 14 12298 11012 -128621 8682 5926 -275622 6707 5668 -103923 8401 6722 -167924 7482 6232 -1250
PRODUCTIVITY: RESULTS
MEDICAL SERVICES
CLINIC PRE POST CHANGE T-VALUE P-VALUESIGN
TEST P VALUE
11 833 748 -85 -234 0.057 0.1113 921 443 -478 14 507 369 -13821 809 547 -26222 820 548 -27223 599 737 13824 732 636 -96
PRODUCTIVITY: RESULTS
CASE MANAGER SERVICES
CLINIC PRE POST CHANGE T-VALUE P-VALUESIGN
TEST P VALUE
11 429 447 18 0.40 0.72 1
13 188 981 793
14 852 533 -319
21 844 690 -154
22 367 407 40
PRODUCTIVITY: RESULTS
FINANCIAL PERFORMANCE
Measures PRE (2007) POST (2010)Revenue per capita staff: $66,900 $67,280
Cost per patient visit: $28.09 $29.68
Hypotheses: Implementation of the electronic health record will result in: Increased revenue per capita staff Decreased cost per patient visit
STUDY FINDINGSQuality: Highly statistically significant
improvement in timely completion of Annual Medical and Annual, 30-Day & 90-Day Multidiscipline assessments
No statistically significant change in obtaining HCV Viral Load for patients positive for HCV Antibody
STUDY FINDINGSRisks: The prevalence of risk management
events was too low to detect a statistically meaningful change
Satisfaction: Patient Satisfaction unchanged pre & post
implementation of the electronic system Staff Satisfaction trended upward post
implementation of the electronic system
STUDY FINDINGSProductivity: Productivity significantly declined for
Human Services staff There was a non-significant productivity
decline for Medical Services staff There was a non-significant productivity
increase for Case Manager staff
STUDY FINDINGSProductivity Confounders: Staff capability to utilize the electronic
system varied considerably Increased turnover of staff due to
inability to adapt to new system Electronic system upgrades required
frequent retraining of staff Preparation for APGs and other billing
issues required major training
STUDY FINDINGSFinancial Performance:
Revenue per capita staff increased by 0.6%
Cost per patient visit increased by
5.7%
IN CONCLUSION …Despite results that were somewhat less robust than expected in some of the domains examined; had we
not implemented the electronic information system, the recent changes in documentation and
reimbursement for services would have paralyzed our agency
ACKNOWLEDGEMENTS PATIENTS AND
STAFF OF THE ADDICTION RESEARCH AND TREATMENT CORPORATION, A COMMUNITY-BASED SUBSTANCE ABUSE SERVICE AGENCY
STUDY TEAM Principal Investigator: Lawrence S. Brown, Jr., MD, MPH, FASAM Executive Senior Vice President Sub-investigators: Carlota John-Hull, MD, Director of Medical Services Melissa Lin, MS, Director of Evaluation and Research Steven Kritz, MD, Research Project Manager Roberto Zavala, MD, Research Assistant Ben Louie, BA, Implementation Project Manager Consultants: Crystal Fuller, PhD, Mailman School of Public Health, Columbia University John Kimberly, PhD, Wharton School of Business, University of Pennsylvania