Implementation Insights into CDS for Imaging Order …...#SIIM18 @SIIM_Tweets Implementation...

#SIIM18 @SIIM_Tweets

Implementation Insights into CDS for Imaging Order Entry

Cree M. Gaskin, M.D.Professor and Vice Chair, Operations and Informatics

Chief, Musculoskeletal Imaging and InterventionAssociate Chief Medical Information Officer

University of Virginia Health System

Presenter

Presentation Notes

Session Chair



Christine BaranImaging Informatics Operations Manager

Enterprise Imaging InformaticsMaineHealth Information Services

Bob Coleman

Presenter

Presentation Notes

My co-session chair Bob Coleman understandably had to withdraw due to the development of his son’s wedding plans. Bob extends his apologies, though in good faith, he prepared content and recruited his colleague Christine Baran. Overview of CMS regulations and deadlines



Adam Prater, M.D., M.P.H.Assistant Professor of Radiology and Imaging Services

Director of Stroke Imaging, Grady Memorial HospitalDirector of Imaging Informatics, Grady Memorial Hospital

Emory Healthcare


1. Provide an overview of the CMS AUC regulations.2. Share insights into what to consider when selecting

and implementing a CDS system. 3. Provide advice on how to manage CDS to improve

patient care without upsetting ordering providers.

Objectives


Next up

Christine BaranImaging Informatics Operations Manager

Enterprise Imaging InformaticsMaineHealth Information Services


Disclosures

• I have no disclosures relevant to the content of this presentation.


Background

•1993: ACR Task Force on AC• to define national guidelines for the appropriate use

of imaging tests• Using a combination of scientific evidence, and when

data insufficient, expert consensus

Presenter

Presentation Notes

K.K. Wallace, M.D. (former chair ACR Board of Chancellors) introduced ACR AC to the U.S. Senate House Ways and Means Committee; intent to eliminate inappropriate utilization


ACR AC

•1, 2, or 3: usually not appropriate

•4, 5, or 6: may be appropriate

•7, 8, or 9: usually appropriate

Major limitation: Information not connected to the point-of-care

Presenter

Presentation Notes

1, 2, or 3 are in the “usually not appropriate” category, where the harms of doing the procedure outweigh the benefits; The middle category, designated “may be appropriate”, is represented by 4, 5, or 6 on the scale. The middle category is where the risks and benefits are equivocal or unclear, the dispersion of the individual ratings from the group median rating is too large (i.e., disagreement), the evidence is contradictory or unclear, or there are special circumstances or subpopulations which could influence the risks or benefits embedded in the variant. 7, 8, or 9 are in the “usually appropriate” category, where the benefits of doing a procedure outweigh the harms or risks.


Structured indications map to CDS

© 2015 Epic Systems Corporation. Used with permission.


Mapping


ROE-DS Impact on Outpatient CT Utilization

Sistrom CL, Dang PA, Weilburg JB, Dreyer KJ, Rosenthal DI, and Thrall JH. Effect of computerized order entry with integrated decision support on the growth of outpatient procedure volumes: seven-year time series analysis. Radiology 251 (1):147-155, 2009.

Annual CompoundGrowth Rate

12%

Annual CompoundGrowth Rate

1%

ROE Penetration

CT Scan Utilization


ROE-DS Impact on Outpatient CT Utilization


Commercial CDS: early experience

• Easier said than done!• RadPort by Nuance• UVA: A first customer, put in test system (2010)

• Tech support = sketchy• User interface = cumbersome • Local adaptability = minimal • DS Rules = one site (MGH), unclear path for updates

• UVA “pulled plug” before PRD go-live (March 2011)

Presenter

Presentation Notes

Late 2010: RadPort by Nuance (adaptation of MGH’s homegrown CDS appearing in early publications) Implementation in TST did not inspire confidence Cumbersome user interface – risk for user acceptance? Limited adaptability to site or insurer requests Based on one site’s rules with unclear path for updates Pulled plug on project prior to March 2011 go-live and sat on it


Williams A, et al. Adopting a Commercial Clinical Decision Support for Imaging Product: Our Experience. JACR 2014; 11(2): 202–204.

Acromegaly!

Commercial CDS: early experience

• U Colorado (April 2011-2012): RadPort

• Result: cumbersome + frustrating lack of desired indications, but also too many useful ones (“background noise”)

• #1 indication for head CT?

• U Colorado pulled the plug; Vendor withdrew product from market

Presenter

Presentation Notes

Limited database of acceptable indications which could not be altered Providers lost free text capability for “reason for exam” and had to choose from alphabetical drop down list Net result: cumbersome and time-consuming; frustrating lack of desired indications, but too many useful ones creating “background noise” Unable to modify content Changed from drop down menus to checkboxes, but frustration reigned Providers chose what was convenient (acromegaly became leading indication for head CT)


UVA successful in 2014

• Still believed in the concept + potential benefits of CDS• Judged ACR Select to be “good enough” in 2014

• Vendor understood/addressed previous problems• ACR partnership: standardization, knowledge

updates• Improved integration with our existing EHR• User interface more acceptable to providers• Flexible configuration/adaptable to site’s needs

• Has run continuously since


Implementation: Engagement

• Leadership: someone needs to initiate conversation• Buy-in from key institutional leaders, executive committees

• Benefits: appropriateness, costs, radiation• Plus: supports providers + facilitates billing (ICD-10 code linking)• Emphasize CMS requirement • Show them: minimal hindrance• Encourage test-drive (portal)

• Team: Leader (imaging?) and stakeholders• UVA: Vice Chair/ACMIO, RIS analyst, vendor, few health IT staff


Implementation: Hosting?

• Vendor host via cloud vs. UVA host on site

• UVA concerns as a first adopter: • Patient data security?• Speed and risk of service interruptions? • Locally hosted – more demands on our staff, overall hindrances to trouble

shooting, support and analytics

• Cloud hosting: standard, safe, fast enough, easier to update, more support from vendor, access to analytics package


Implementation: Manage Change

• Ordering provider acceptance is key• However, ordering providers may:

• See differences in workflow• Have more workflow steps• Feel their autonomy is being challenged• Worry about being judged• Not recognize benefits or understand why

Implement gently!



• Problem: Providers can be frustrated by structured indications and BPA’s, both of which are new.

• One solution: Start with “silent mode.” Fewer changes. And, benefit of baseline data as collect scores in background.



• Problem: Providers may not understand why they are entering structured indications or who to contact with concerns.

• Solution: Communication. • Information system training• Email broadcasts • And at point-of-care . . .




• Provider education at point-of-care• Added reference link at order entry



• Problem: Providers don’t want something new.

• Solution: Timed addition of structured indications with an EHR upgrade.

Not an option for all, but perception was part of EHR upgrade, rather than an add-on that might be elective.


Implementation: Parameters

• Problem: Providers are frustrated by BPA’s.

• Solution: Lessen their impact• Don’t fire BPA for green scores.• If ultrasound doesn’t require authorization, don’t turn it on• Don’t block “red” orders• Don’t require an acknowledgment reason for “red” orders unless you

are going to do something with it



• Problem: Older faculty might be less accepting of BPA’s than trainees

• Solution: We started BPA’s with ED and inpatient orders – mostly trainees. Faculty orders -- outpatient.



Implementation: Indications

• Problem: Structured indications required, but challenging

• Drop-down menu or checkboxes?• Short lists or exhaustive lists?

• Solutions: several . . .

Presenter

Presentation Notes

Required to map indication to electronic feedback Drop down menus? Or, check boxes? Exhaustive vs. short list? Remember U Colorado?


Structured indications

• Provide search functions• Lists alone are not enough






• Groups? Common, less common, peds, trauma, oncology?• Filtering: Age, gender, patient setting (ED vs. outpatient),

provider specialty



Just skip the boxes and insert text


Novel indication or additional history?



• Problem: Structured indications slightly increase work

• Solution: Make it easy and give them something back



Alternate exams

• Facilitate changing orders without starting order over• Retains reason and order priority so not re-entered


CMS allowable? Your out of pocket? Global? Technical? Professional? Vary by insurer?


Give them something back: cost


Development request and waited for relative cost similar to restaurant guide


Give them something back: cost



Give them something back: dose


Give them something back: ICD10

• For billing, outpatient orders are linked to ICD10 code• Providers can be frustrated if they select an indication,

and have to select a similar ICD10 code description• Structured indications can be mapped to ICD10 codes

in cooperation between billing, compliance and vendor• CDS can nominate an ICD10 code to facilitate

diagnosis code linking


Mapping: Critical M.D. involvement

- Map your exams to various ACR, NCCN, ACC, SPR, SNMMI exam names.

- When a provider selects your specific exam name, the decision support score will reflect the score attached to that standard exam for the selected indication, patient age, and gender.


Mapping: Alternative procedures

When the logic behind the scenes says that a _______ (ACR exam name) is a more appropriate exam, which of your exam names do you want to show instead of the ACR Exam name?


Data: Is CDS working?

• UVA – early adopter of commercial CDS• Literature and experience lacking regarding impact of

commercial products. Early product lacked tools to assess.• We had questions . . .

• How is overall appropriateness affected by CDS?• Does this vary by department or provider?• Are certain exams problematic – returning only very high/low scores?• Other issues? Studies not scored? Only getting free text replies?

Huber TC, Krishnaraj A, Monaghan D, Gaskin CM. Developing an Interactive Data Visualization Tool to Assess the Impact of Decision Support on Clinical Operations. JDI 2018 “online first” DOI 10.1007/s10278-018-0065-z


Data: Is CDS working? • UVA used a data visualization tool to create

a dynamic dashboard with adjustable inputs (date, dept, provider type, pt age, modalities, scores, patient class

• to detect trends and monitor for problems

Huber TC, Krishnaraj A, Monaghan D, Gaskin CM. Developing an Interactive Data Visualization Tool to Assess the Impact of Decision Support on Clinical Operations. JDI 2018 “online first” DOI 10.1007/s10278-018-0065-z

Presenter

Presentation Notes

Fig. 1 Dashboard layout in Tableau. The pane at the top right displays summary data describing the total number of exams, appropriateness score, and a timeline displaying study volume by score over time. In the middle of the dashboard, the data are displayed in terms of ordering department. The total number of studies falling into each appropriateness category is displayed as well as the percentage of the total. Finally, the bottom two panes provide information regarding specific types of exams. On the left is a breakdown of the most common exams with their appropriateness scores, so the user can search for high-yield issues that affect frequent orders. The total number of studies of each appropriateness category is displayed as well. On the right is a cloud map of studies of which more than 25% were scored as low utility (appropriateness score 1–3, per ACR appropriateness criteria). This can help identify problematic exams that are receiving excessively high frequency of low scores which could be due to technical reasons or which may warrant education in clinical areas. Fig. 2 Timeline feature of the dashboard. Exam volume by appropriateness category can be displayed. In this example, completed exams with appropriateness score in the Blow utility^ cate Fig. 3 Screenshot of single pane from CDS dashboard demonstrating visualization of ordering department-specific percentages of exams of varying degrees of appropriateness. For confidentiality in this manuscript, the department names are obscured. One department demonstrated 54% of exams of low utility, whereas others (not shown) had only 5% or less exams of low utility. This type of visualization could lead to exploration of these results and if warranted, clinical or technical interventiongory are displayed. An obvious decline in the volume of “low utility^ studies can be observed Fig. 4 Screenshot of single pane from CDS dashboard demonstrating visualization of provider type-specific percentages of exams of varying degrees of appropriateness. In this example, the constraints of which are not important to this demonstration, the dashboard indicates that physician assistants had a higher percentage of exams (18%) with low utility scores, while residents had the fewest (8%) Fig. 5 Cloud map displaying studies with more than 25% of orders scoring as low utility. Features such as this allow the user to quickly identify exam types in need of further investigation


Data: Results with BPA’s

• Inpatient/ED• MRI, CT, US, PT/NM• Adult patients• Appropriateness scores• 6 months silent mode• Vs. 24 months feedback• “no teeth” or penalty

Huber TC, Krishnaraj A, Patrie J, Gaskin CM. Impact of a Commercially AvailableClinical Decision Support Program on Provider Ordering Habits. JACR 2018 in press https://doi.org/10.1016/j.jacr.2018.03.045

Presenter

Presentation Notes

Fig 1. Appropriateness score categorization relative frequency distributions for the pre-intervention (ie, silent mode) study period and the intervention (ie, feedback mode) study period.

https://doi.org/10.1016/j.jacr.2018.03.045



• By modality• MRI, CT, US: improved• PET/NM: not improved


Presenter

Presentation Notes

Fig 2. Appropriateness score categorization relative frequency distributions for the pre-intervention (ie, silent mode) study period and intervention (ie, feedback mode) study periods by imaging modality.




• By modality• MRI, CT, US: improved• PET/NM: not improved





• By provider type• Trainees showed more


Presenter

Presentation Notes

Fig 3. Appropriateness score category relative frequency distributions for the pre-intervention (ie, silent mode) and intervention (ie, feedback mode) study periods by ordering provider type.




• Appropriate• May be appropriate• Low utility• 3 years• 6 month intervals• Regression modeling


Presenter

Presentation Notes

Fig 4. Relative frequencies (%) for the number of low utility, marginal utility, and indicated appropriateness score categorizations observed during months 1 to 6, 7 to 12, 13 to 18, 19 to 24, 25 to 30, 31 to 36. Multinomial regression model predictions for low utility, marginal utility, and indicated appropriateness score categorization relative frequencies as a function of “time interval” are indicated by the red line, the gray line, and the blue line, respectively.



Conclusions

• Commercial CDS may be implemented successfully • Technical success• Ordering provider acceptance• Documented improvements

• Radiologist needed, at least for mapping, if not leadership

• Implement gently. Communicate, consider stages, reduce BPA frequency and hard stops, give back


Future?

• Potential to bypass preauth with CDS

• Imagine outpatient MRI order green score from CDS Automatic approval Add-on into same day open slot

• Efficiency, satisfaction, business captured

• Pilots are beginning. If successful, may become common.


Thank you!Cree M. [email protected]

Presenter

Presentation Notes

Boards vs real life

Implementation Insights into CDS for Imaging Order …...#SIIM18 @SIIM_Tweets Implementation...

Documents

Transcript of Implementation Insights into CDS for Imaging Order …...#SIIM18 @SIIM_Tweets Implementation...