Speech Recognition: Pros & Cons

Dr. Manohar Shroff Radiologist in Chief, Hospital for Sick Children

Professor, Department of Medical Imaging, University of Toronto

Report Generation in Radiology

•  Initially only oral reports

•  Hand written reports

•  Dictation to a secretary who takes notes in shorthand and then goes on to type the report

•  Dictation into a tape and then trancription by a secretary

•  Dictation into an electronic phone system with local or remote transcriptionist

•  VOICE RECOGNITION

1942: Chicago, Illinois. Provident Hospital. Dr. B.W. Anthony, head of the radiology department, dictating a report on x-rays. Credit: Delano, Jack. http://www.websters-online-dictionary.net/images//photos

Once upon a time…..

Hinduja Hospital, Mumbai

Scheduling

Protocol Optimization

Image Acquisition and Processing

Quality Assurance

Interpretation

Requests

Reporting & Communication

Data Access

Clinical decision support

Auto information input & link

Auto clincial & protocol input into report

Linked QA on the fly

Linked education resources

Voice Recognition, Structured reports

Autosend critical result: sms, email, call

Clinical Excellence Closing the ‘imaging’ loop: Leveraging IT

Clinical Mission of Most Radiology Departments: Provide relevant, timely, extraordinary imaging care supported by world class education & research

Voice recognition software in radiology reporting has significantly reduced the report turnaround time, an essential part of providing quality and timely patient care

Before Voice Recognition

•  Dictation was still on tapes •  Delay in reporting and lost tapes •  7 transcriptionists (3 full time and 4

contract) •  Approximate cost of transcription per

year: 12,000 dollars per radiologist

Workflow Before Voice Dictation

All urgent, emergent and inpatient reports written with hand and faxed to ER & floors

GOALS for Voice Recognition •  Strategic Goal to reduce Report Turn

Around Time •  Improve Communications Between

Departments (Critical Results) •  Streamline reporting and improve

efficiency •  PACS driven Workflow •  Created standardized Templates ??

Operations:

Innovation:

Education:

Research

Present Future

Impact of Informatics and Information Technology & the Digital World

Labor intensive

Transformation through innovation and paradigm shift

IT leveraged

Passive Active/interactive

Apprenticeship Simulation

Clinical Transformational

Report Turn Around Time (> 97%) (in hours)

2007 2008 2010 2012

•  Improved Report Turnaround Time •  Decreased human costs •  Potential for measuring data •  Quality Improvement •  Improved patient care •  Improved communication

•  Radiologist as the transcripitionist •  Increased radiologist time per report •  Increased errors in reports •  Lost human jobs

PROS:

CONS:

VOICE RECOGNITION: Transcriptionist in a box

CON: Radiologist as the transcriptionist: Journal of Digital Imaging, 21 (4) 2008: 384-389

•  Non-academic, outpatient setting •  Reports took 50% longer to dictate •  Increased errors •  Additional costs per report ($ 6.10 vs $

2.97) •  Unhappy radiologists: increased user

fatigue, alteration of dictation style and substance

CON: Radiologist as the transcriptionist: Journal of Digital Imaging, 21 (4) 2008: 384-389

•  Transcriptionists cost $ 16 / hr; Radiologists cost $ 175 /hr

•  Average dictation time was 4.11 mins with VR compared to 2.09 mins

•  5.1 error per report for VR vs 0.12 errors for transcriptionist

•  Additional 104 minutes of radiologist’s time per day: can equal 10 more dictated MR reports

Unhappy People

Focus more on correcting dictated errors rather than looking at images Taking eyes of the report

Quality x Efficiency x Safety = Better Outcomes

Clinical Excellence

Healthier Children. A Better world

Radiology reports generated by voice recognition technology:

comparative quality before and after audit recommendations

Andre Pereira, Lisa Jong, Suhas

Kotecha, Erika Mann* Courtesy: Erika Mann, MD,FRCPC Staff pediatric radiologist & Quality

Assurance Lead at SickKids Radiology reports generated by voice recognition technology: comparative quality before and after audit recommendations:

Department of Medical Imaging Research Day 2012, University of Toronto

First audit (2008) � Hospital for Sick Children � Errors classified in:

�  Spelling �  Spacing �  Sentence structure �  Report organization �  Intent / interpretation

○  Incoherent phrase ○  Incorrect laterality ○ Lack / wrong measurement units ○ Discrepancy between impression and body of report

Radiology reports generated by voice recognition technology: comparative quality before and after audit recommendations: Department of Medical Imaging Research Day 2012, University of Toronto

First audit - results •  2008 (1.6 +/- 1.9 errors per report)

–  Error free: 37% –  Significant errors: 20% –  Insignificant errors: 43%

37

20

43

Error free

Significant

Insignificant

Recommendations after 1st audit

•  Keep built-in spelling and grammar review tool activated when dictating

•  Encourage use of macros / templates / structured report

•  Careful proof-reading of templates to avoid recurring errors

•  Careful review of second reader reports •  Special attention to final interpretation

Second audit (2010 -2011)

•  Hospital for Sick Children •  Same methodology as first audit •  Significant errors:

•  Incoherent phrase •  Incorrect laterality •  Lack / wrong measurement units •  Discrepancy between impression and body of report

Radiology reports generated by voice recognition technology: comparative quality before and after audit recommendations: Department of Medical Imaging Research Day 2012, University of Toronto

Second audit - results •  2010 (0.6 +/-0.9 errors per report )

–  Error free: 56% –  Significant errors: 2.6% –  Insignificant errors: 41.4%

56

2.6

41.4Error free

Significant

Insignificant

Radiology reports generated by voice recognition technology: comparative quality before and after audit recommendations: Department of Medical Imaging Research Day 2012, University of Toronto

VR errors must be measured….

•  Significant improvement in quality of reports after first audit recommendations.

•  More reports containing fewer mistakes. •  Shift of errors to intent / interpretation

type. •  Transition to speech recognition software

must include special attention to detection and correction of errors previously handled by transcriptionists.

Radiology reports generated by voice recognition technology: comparative quality before and after audit recommendations: Department of Medical Imaging Research Day 2012, University of Toronto

Comparative (percentages)

First Audit

Radiology reports generated by voice recognition technology: comparative quality before and after audit recommendations - Research Day 2012

34.5

20.5

45

Error free

Significant

Insignificant

58

4.2

37.8Error free

Significant

Insignificant

Second Audit human intervention

Targetted Quality Improvement:

Major Error

RA FL US CT MR TOTAL Staff # 1 2 0 1 0 0 3 Staff # 2 0 0 1 0 0 1 Staff # 3 1 0 0 0 0 1 Staff # 4 3 0 1 0 0 4 Staff # 5 1 0 0 0 0 1 Total 7 0 3 0 0 10

JDMI (AJR October 2011) •  615 consecutive breast reports

analyzed – Complex cases from tumor board rounds at

PMH/MSH (all modalities) •  Bias against screening reports where templates

are common – VR (MSH) compared to TR (PMH)

2009-2010 – Same faculty and trainees report at both

sites •  Major errors: subjectively “affected

understanding of the report”

JDMI, U of T

•  Trainee vs. faculty: –  No difference in major error rates –  Faculty with higher minor error rates

•  No difference in native vs. non-native English speakers •  Findings > Impression •  Least errors in procedure reports and mammo

8x major errors and 2x minor errors in VR relative to TR (after normalizing for report type, report length, etc.)

CAR Guidelines – Final Report a. The final report is considered to be the definitive means of communicating to the referring physician or other healthcare professional the results of an imaging examination or procedure. Additional methods of communication of results are necessary in certain situations. c. The timeliness of reporting any imaging examination varies with the nature and urgency of the clinical problem. The written final report should be made available in a clinically appropriate, timely manner. d. The final report should be proofread carefully to avoid typographical errors, accidentally deleted words, and confusing or conflicting statements, and should be authenticated by the reporting radiologist, whenever possible. h. Voice recognition systems are widely employed to facilitate timely reporting. These systems are not foolproof and methods should be in place to allow detection and correction of program generated errors.

People = Success

Liu D, Zucherman M, Tulloss W; Journal of Digital Imaging, 19 (1) 2006: 98-104

Synoptic Reports for Rectal Cancer

Einstein’s 3 rules of work

•  Out of clutter, find simplicity •  From discord, find harmony •  In the middle of difficulty,

lies opportunity

To Conclude: Voice Recognition is here to stay: It has significant advantages and major disadvantages. Change management is very important as we adopt such new technology

Thank you for your attention…