Slides (.ppt)

download Slides (.ppt)

of 53

  • date post

    26-May-2015
  • Category

    Documents

  • view

    158
  • download

    0

Embed Size (px)

Transcript of Slides (.ppt)

  • 1. 01/27/15 Automated Retrieval and Generation ofAutomated Retrieval and Generation of Brain CT Radiology ReportsBrain CT Radiology Reports Gong Tianxia SOC NUS

2. 01/27/15 Outline Background Motivation Research Work Conclusion 3. 01/27/15 Background Computer Tomography (CT) has been used to examine the abnormality of human brain due to various causes The result of each brain CT examination consists of: A set of CT scan image A report written by a radiologist 4. 01/27/15 Abnormalities Head traumas epidural hemorrhage(EDH) acute subdural hemorrhage (SDH_Acute) chronic subdural hemorrhage (SDH_Chronic) intracerebral hemorrhage (ICH) intraventricular hemorrhage (IVH) subarachnoid hemorrhage (SAH) Fractures Edemas Others Midline shift Etc. 5. 01/27/15 Background Brain CT Scans Samples Normal EDH 6. 01/27/15 Background Brain CT Scans Samples ICH SDH_Acute, SDH_Chronic, Midline Shift 7. 01/27/15 Background Report Unenhanced axial CT head was obtained. No previous study is available for comparison. There is acute subdural haemorrhage overlying the left convexity & midline falx, which measures up to a maximum of 1.4 cm in thickness. Subarachnoid haemorrhage is seen in the sulci at the left fronto-temporal lobe, bilateral Sylvian fissure & cistern and the basal cistern. Intraventricular extension of haemorrhage with blood seen in all four ventricles is noted. There is intraparenchymal haemorrhage in the bilateral frontal lobes raising the suspicion of haemorrhagic contusion. There is considerable mass effect with midline shift to the right, generalised effacement of cerebral sulci and compression of the left lateral ventricle. Prominence of the right temporal horn is suspicious for a hydrocephalus. No skull vault fracture is seen in the CT scan. 8. 01/27/15 Background Comments Acute left fronto-temporal-parietal subdural haematoma with bifrontal parenchymal haematoma and bilateral subarachnoid haemorrhage with intraventricular extension. Associated mass effect with midline shift to the right, compression of the left lateral ventricle and generalised effacement of cerebral sulci. Hydrocephalus with right ventricle dilated. 9. 01/27/15 Motivation Radiology reports contain rich information which is not used in many medical database systems The proposed system is aimed to: Provide convenient search functions for radiology reports and images Help doctors, radiologists, and medical informaticians to gather needed information for their research Give references to radiologists to compare results Facilitate education systems for researchers, junior doctors, and medical students Integrate medical records from various sources Provide platform for medical community to exchange information and knowledge 10. 01/27/15 Automated Retrieval and Generation of Brain CT Radiology Reports Content-based Retrieval of CT Scan Brain Images Two Research Directions 11. 01/27/15 Information Extraction from Radiology Reports Automatic Generation of Medical Reports Free Text Assisted Medical Image Retrieval Related Work 12. 01/27/15 Information Extraction from Radiology Reports Automatic Generation of Medical Reports Free Text Assisted Medical Image Retrieval Research Work 13. 01/27/15 MedLEE: Medical Language Extraction and Encoding System RADA: RADiology Analysis Tool Statistical Natural Language Processor for Medical Reports Related Work 14. 01/27/15 MedLEE: Medical Language Extraction and Encoding System 15. 01/27/15 RADA: Radiology Analysis Tool 16. 01/27/15 Statistical Natural Language Processor for Medical Reports 17. 01/27/15 An example of structured representation output Statistical Natural Language Processor for Medical Reports 18. 01/27/15 Negations Insufficient understanding of the text Ungrammatical writing styles Large vocabulary Assumed knowledge between writer and reader Challenges 19. 01/27/15 Information Extraction from Radiology Reports Automatic Generation of Medical Reports Free Text Assisted Medical Image Retrieval Related Work 20. 01/27/15 Most existing medical report automatic generation systems use the following template filling approaches: Structured Data Entry Mail Merge Canned Text Automatic Generation of Medical Reports 21. 01/27/15 NLG: NLG is still premature application of medical document generation There is still no system based on NLG principles in routine use generates medical reports with fluent, concise and readable text Challenges of NLG in general domain also exist in medical domain Systems that automatically generate medical report from medical images are still lacking. Challenges 22. 01/27/15 Information Extraction from Radiology Reports Automatic Generation of Medical Reports Free Text Assisted Medical Image Retrieval Related Work 23. 01/27/15 NeuRadIR: Web-Based Neuroradiological Information Retrieval System Information Retrieval on MR Brain Images and Radiology Reports Free Text Assisted Medical Image Retrieval 24. 01/27/15 NeuRadIR 25. 01/27/15 MRI Brain Image and Report Retrieval 26. 01/27/15 Complexity of the system, as the system Consists of many functional components Needs knowledge from various research areas Challenges 27. 01/27/15 Information Extraction from Brain CT Radiology Reports Automatic Generation of Brain CT Radiology Reports Radiology Reports Assisted Brain CT Images Retrieval Research Areas 28. 01/27/15 Information Extraction from Brain CT Radiology Reports Automatic Generation of Brain CT Radiology Reports Radiology Reports Assisted Brain CT Images Retrieval Research Areas 29. 01/27/15 Information Extraction from Brain CT RadiologyInformation Extraction from Brain CT Radiology ReportsReports Our major task in this research area is to extract structured medical findings from the free text brain CT radiology reports 30. 01/27/15 Input & OutputInput & Output Input example: An extra-dural haematoma overlying the right frontal lobe is seen measuring 1.2 cm in thickness. Finding haematoma type extradural location overlying brain_part lobe orientation right orientation frontal thickness 1.2 cm Output example: 31. 01/27/15 System ArchitectureSystem Architecture The system will have these components Document Chunker Parser Term Mapper Finding Extractor Report Constructor 32. 01/27/15 Document ChunkerDocument Chunker Decompose the radiology report into three sections Reasons for examination Detailed description of observations and findings Comments or conclusion We will focus on second and third sections, as they contain medical findings 33. 01/27/15 ParserParser Parse each sentence of a report and outputs a typed dependence tree Parser output example: null:seen nsubjpass:hematoma det:An amod:extra-dural partmod:overlying dobj:lobe det:the amod:right amod:frontal auxpass:is partmod:measuring dobj:cm num:1.2 prep-in:thickness Grammatical relation to parent word 34. 01/27/15 Term MapperTerm Mapper Maps words to standard forms specified in our medical knowledge source (Unified Medical Language System UMLS and other radiology thesaurus) Reduces spelling variations 35. 01/27/15 Finding ExtractorFinding Extractor Apply semantic rules that are derived from semantic features of the words to translate the typed dependency relationship to logical relationship between findings and modifiers (findings attributes) Merge the same finding from different sentences into one finding Remove the redundant finding 36. 01/27/15 Report ConstructorReport Constructor Construct structured report according to findings, modifiers, and their logical relationship extracted from the finding extractor 37. 01/27/15 Research AreasResearch Areas Information Extraction from Brain CT Radiology Reports Automatic Generation of Brain CT Radiology Reports Radiology Reports Assisted Brain CT Images Retrieval 38. 01/27/15 Automatic Generation of Brain CTAutomatic Generation of Brain CT Radiology ReportsRadiology Reports A traditional approach based on typical NLG system Content determination Discourse planning Sentence aggregation Lexicalization Referring expression generation Linguistic realization 39. 01/27/15 Content DeterminationContent Determination Creates a set of messages from the features extracted from the new brain CT Images Doctors use size, shape and location of the potential hemorrhage region to determine head trauma types The system uses similar features for content determination: area, major axis length, minor axis length, eccentricity, solidity, extent, adjacency to skull, adjacency to background 40. 01/27/15 Content DeterminationContent Determination Image Segmentation Features Extraction 41. 01/27/15 Discourse PlanningDiscourse Planning Uses Rhetorical Structure Theory (RST) to organize the text based on relationships that hold between parts of the text 42. 01/27/15 Sentence AggregationSentence Aggregation Groups messages together into sentences and paragraphs 43. 01/27/15 Sentence AggregationSentence Aggregation Groups messages together into sentences and paragraphs 44. 01/27/15 LexicalizationLexicalization Decides which specific words and phrases should be chosen to express the domain concepts and relations which appear in the messages Uses hardcoded specific word and phrases to standardize the output language radiology reporting Uses NLG system to generate radiology reports of various writing styles to cater different user groups (at later stage of our project) 45. 01/27/15 Final StepsFinal Steps Referring Expression Generation Linguistic Realization 46. 01/27/15 A Machine Learning ApproachA Machine Learning Approach Based on the concept of statistical machine translation Image and report are two representations of the same medical condition In a sense, image and text are two different