Don’t Let Notes Be Misunderstood: A Negation Detection Method for Assessing Risk of Suicide in...
-
Upload
george-gkotsis -
Category
Data & Analytics
-
view
281 -
download
1
Transcript of Don’t Let Notes Be Misunderstood: A Negation Detection Method for Assessing Risk of Suicide in...
![Page 1: Don’t Let Notes Be Misunderstood: A Negation Detection Method for Assessing Risk of Suicide in Mental Health Records](https://reader033.fdocuments.in/reader033/viewer/2022042906/589dfa761a28ab1e718b5b07/html5/thumbnails/1.jpg)
Don't Let Notes Be Misunderstood:
A Negation Detection Method for Assessing Risk of Suicide in
Mental Health RecordsGeorge Gkotsis, Sumithra Velupillai, Anika Oellrich
Harry Dean, Maria Liakata and Rina Dutta
Biomedical Research Centre Nucleus – King’s College London
![Page 2: Don’t Let Notes Be Misunderstood: A Negation Detection Method for Assessing Risk of Suicide in Mental Health Records](https://reader033.fdocuments.in/reader033/viewer/2022042906/589dfa761a28ab1e718b5b07/html5/thumbnails/2.jpg)
e-HOST-ITElectronic health records to predict HOspitalised Suicide attempts:Targeting Information Technology solutions
AimTo determine whether structured and free-text data in Electronic Health Records (EHRs) can be used to quantify changes in symptoms, behaviour patterns and health service-utilisation and predict serious suicide attempts
![Page 3: Don’t Let Notes Be Misunderstood: A Negation Detection Method for Assessing Risk of Suicide in Mental Health Records](https://reader033.fdocuments.in/reader033/viewer/2022042906/589dfa761a28ab1e718b5b07/html5/thumbnails/3.jpg)
Motivation
![Page 4: Don’t Let Notes Be Misunderstood: A Negation Detection Method for Assessing Risk of Suicide in Mental Health Records](https://reader033.fdocuments.in/reader033/viewer/2022042906/589dfa761a28ab1e718b5b07/html5/thumbnails/4.jpg)
• Health records contain many structured fields, such as:• Personal information/contact• Diagnosis• Prescription• Interventions• Scans & Measurements
Most of the structured fieldsare left blank
The mysterious case of health records (1/2)
Max Weber
Theory ofFormal Rationality
![Page 5: Don’t Let Notes Be Misunderstood: A Negation Detection Method for Assessing Risk of Suicide in Mental Health Records](https://reader033.fdocuments.in/reader033/viewer/2022042906/589dfa761a28ab1e718b5b07/html5/thumbnails/5.jpg)
The mysterious case of health records (2/2)• Free text contains a lot of information
• Traditional information access technology returns many false positives
Example1. Patient is suicidal
2. Patient is not suicidal
• Meaning can be expressed in multiple ways
Example1. He has suicidal thoughts2. He wants to end his life3. She wants to kill herself
![Page 6: Don’t Let Notes Be Misunderstood: A Negation Detection Method for Assessing Risk of Suicide in Mental Health Records](https://reader033.fdocuments.in/reader033/viewer/2022042906/589dfa761a28ab1e718b5b07/html5/thumbnails/6.jpg)
![Page 7: Don’t Let Notes Be Misunderstood: A Negation Detection Method for Assessing Risk of Suicide in Mental Health Records](https://reader033.fdocuments.in/reader033/viewer/2022042906/589dfa761a28ab1e718b5b07/html5/thumbnails/7.jpg)
CRIS database• 226,000 patients• 18.6 million documents (Event)
• Suicide-related data• 783,000 documents contain the word suicid*• 111,000 patients
Anonymous Reviewer:“Overall,I think the paper is well thought out and written, and I am
envious of their access to such a large patient dataset”
![Page 8: Don’t Let Notes Be Misunderstood: A Negation Detection Method for Assessing Risk of Suicide in Mental Health Records](https://reader033.fdocuments.in/reader033/viewer/2022042906/589dfa761a28ab1e718b5b07/html5/thumbnails/8.jpg)
Problem description
![Page 9: Don’t Let Notes Be Misunderstood: A Negation Detection Method for Assessing Risk of Suicide in Mental Health Records](https://reader033.fdocuments.in/reader033/viewer/2022042906/589dfa761a28ab1e718b5b07/html5/thumbnails/9.jpg)
Negation detection – definition (1/2)
“The determination of whether a finding or disease mentioned within narrative medical
reports is present or absent”*
Negex, Chapman et al.Journal of Biomedical Informatics, 2001
![Page 10: Don’t Let Notes Be Misunderstood: A Negation Detection Method for Assessing Risk of Suicide in Mental Health Records](https://reader033.fdocuments.in/reader033/viewer/2022042906/589dfa761a28ab1e718b5b07/html5/thumbnails/10.jpg)
Negation detection – definition (2/2)
Negation Detection
Sentence
Target keyword
Positive/Negative
![Page 11: Don’t Let Notes Be Misunderstood: A Negation Detection Method for Assessing Risk of Suicide in Mental Health Records](https://reader033.fdocuments.in/reader033/viewer/2022042906/589dfa761a28ab1e718b5b07/html5/thumbnails/11.jpg)
Towards negation detection resolution• Fundamental NLP taskReduced to identifying the scope of negation
Examples:
No issues other than her indicating that she might commit suicide
He continues to deny any suicidal thoughts and is happy to come to the XXX for medical review tomorrow
+
-
![Page 12: Don’t Let Notes Be Misunderstood: A Negation Detection Method for Assessing Risk of Suicide in Mental Health Records](https://reader033.fdocuments.in/reader033/viewer/2022042906/589dfa761a28ab1e718b5b07/html5/thumbnails/12.jpg)
State-of-the-art: Negex (1/2)• Lexical-based approach
• Collection of negation cues/expressions• Pseudo-negation expressions• Termination cues for scope• Search scope of 6 words surrounding the target keyword
• pyConTextNLP
![Page 13: Don’t Let Notes Be Misunderstood: A Negation Detection Method for Assessing Risk of Suicide in Mental Health Records](https://reader033.fdocuments.in/reader033/viewer/2022042906/589dfa761a28ab1e718b5b07/html5/thumbnails/13.jpg)
State-of-the-art: DEEPEN (2/2)• Wrapper over NegEx• Applied over the (predicted to be) negated sentences• Uses a dependency parse tree
![Page 14: Don’t Let Notes Be Misunderstood: A Negation Detection Method for Assessing Risk of Suicide in Mental Health Records](https://reader033.fdocuments.in/reader033/viewer/2022042906/589dfa761a28ab1e718b5b07/html5/thumbnails/14.jpg)
“Negation’s Not Solved”*• Optimizable, but not generalizable• Annotation guidelines are different• Spans considered can be nouns or whole phrases• Amount of overlap allowed (or not)
*Wu et al.PLOS One, 2014
![Page 15: Don’t Let Notes Be Misunderstood: A Negation Detection Method for Assessing Risk of Suicide in Mental Health Records](https://reader033.fdocuments.in/reader033/viewer/2022042906/589dfa761a28ab1e718b5b07/html5/thumbnails/15.jpg)
Proposed solution
![Page 16: Don’t Let Notes Be Misunderstood: A Negation Detection Method for Assessing Risk of Suicide in Mental Health Records](https://reader033.fdocuments.in/reader033/viewer/2022042906/589dfa761a28ab1e718b5b07/html5/thumbnails/16.jpg)
Workflow
Annotated Dataset
Preprocessing
CoreNLPSentence
Parse tree
Target Node
Negation Detection
1. Pruning
2. Identification of
dominating
subordinate
clause
3. Identification of
negation
governing the
target-node
4. Negation
resolution
Positive/Negative
TargetKeyword‘suicid*’
MHRs
‘suicid*’
Annotation
Dataset and annotation
Proposed Methodology
![Page 17: Don’t Let Notes Be Misunderstood: A Negation Detection Method for Assessing Risk of Suicide in Mental Health Records](https://reader033.fdocuments.in/reader033/viewer/2022042906/589dfa761a28ab1e718b5b07/html5/thumbnails/17.jpg)
2941
3125
Positive Negative
Dataset and annotation• Generation
• Random sampling from SLAM Events of 6k sentences containing the word “suicid*”
• Annotation• One expert annotated the complete corpus• Another expert repeated the annotations for 25% of the sentences𝛋=0.93 (IAA=97.9%)
Limitations• Linguistic focus• Patient-agnostic
![Page 18: Don’t Let Notes Be Misunderstood: A Negation Detection Method for Assessing Risk of Suicide in Mental Health Records](https://reader033.fdocuments.in/reader033/viewer/2022042906/589dfa761a28ab1e718b5b07/html5/thumbnails/18.jpg)
1. PruningLabels• Subordinate conjunctions• ,• S• SBAR• SINV
![Page 19: Don’t Let Notes Be Misunderstood: A Negation Detection Method for Assessing Risk of Suicide in Mental Health Records](https://reader033.fdocuments.in/reader033/viewer/2022042906/589dfa761a28ab1e718b5b07/html5/thumbnails/19.jpg)
2. Identification of dominating subordinate clause
SBAR
![Page 20: Don’t Let Notes Be Misunderstood: A Negation Detection Method for Assessing Risk of Suicide in Mental Health Records](https://reader033.fdocuments.in/reader033/viewer/2022042906/589dfa761a28ab1e718b5b07/html5/thumbnails/20.jpg)
3. Identification of governing nodesS
![Page 21: Don’t Let Notes Be Misunderstood: A Negation Detection Method for Assessing Risk of Suicide in Mental Health Records](https://reader033.fdocuments.in/reader033/viewer/2022042906/589dfa761a28ab1e718b5b07/html5/thumbnails/21.jpg)
Evaluation
![Page 22: Don’t Let Notes Be Misunderstood: A Negation Detection Method for Assessing Risk of Suicide in Mental Health Records](https://reader033.fdocuments.in/reader033/viewer/2022042906/589dfa761a28ab1e718b5b07/html5/thumbnails/22.jpg)
Comparison1. Proposed Model (uses 15 negation words)2. NegEx (uses 272 rules)3. pyConTextNLP-N (uses [2])4. pyConTextNLP-O (uses [1])
no, without, nil,not, n't, never, none, neith, nor, nondeny, reject, refuse, subside, retract
![Page 23: Don’t Let Notes Be Misunderstood: A Negation Detection Method for Assessing Risk of Suicide in Mental Health Records](https://reader033.fdocuments.in/reader033/viewer/2022042906/589dfa761a28ab1e718b5b07/html5/thumbnails/23.jpg)
Results (1/3)
Positive NegativePositive 2782 331Negative 159 2794
Total 2941 3125Pred
ictio
n
Class
![Page 24: Don’t Let Notes Be Misunderstood: A Negation Detection Method for Assessing Risk of Suicide in Mental Health Records](https://reader033.fdocuments.in/reader033/viewer/2022042906/589dfa761a28ab1e718b5b07/html5/thumbnails/24.jpg)
Results (2/3)
Precision Recall FM AccuracyNegEx 93.4 92.1 92.8 93
pyContextNLP-N 94.1 92.9 93.5 93.7pyContextNLP-O 80.7 86 83.2 83.2
Proposed 89.4 94.6 91.9 91.9
![Page 25: Don’t Let Notes Be Misunderstood: A Negation Detection Method for Assessing Risk of Suicide in Mental Health Records](https://reader033.fdocuments.in/reader033/viewer/2022042906/589dfa761a28ab1e718b5b07/html5/thumbnails/25.jpg)
Results (3/3)
![Page 26: Don’t Let Notes Be Misunderstood: A Negation Detection Method for Assessing Risk of Suicide in Mental Health Records](https://reader033.fdocuments.in/reader033/viewer/2022042906/589dfa761a28ab1e718b5b07/html5/thumbnails/26.jpg)
Discussion & Conclusion
![Page 27: Don’t Let Notes Be Misunderstood: A Negation Detection Method for Assessing Risk of Suicide in Mental Health Records](https://reader033.fdocuments.in/reader033/viewer/2022042906/589dfa761a28ab1e718b5b07/html5/thumbnails/27.jpg)
Distribution of sentence length
Proper punctuation and sentence chunking are crucial!
![Page 28: Don’t Let Notes Be Misunderstood: A Negation Detection Method for Assessing Risk of Suicide in Mental Health Records](https://reader033.fdocuments.in/reader033/viewer/2022042906/589dfa761a28ab1e718b5b07/html5/thumbnails/28.jpg)
Discussion• Corpus of 6k sentences from Mental Health Records• Annotation of high quality• Evaluation - focus on positive cases
• Parse trees+Require minimum number of negation keywords+Further potential
• Statement extraction (subject-predicate-object)• Temporal characteristics• Degree of suicidality
- Expensive - Error prone for long sentences
![Page 29: Don’t Let Notes Be Misunderstood: A Negation Detection Method for Assessing Risk of Suicide in Mental Health Records](https://reader033.fdocuments.in/reader033/viewer/2022042906/589dfa761a28ab1e718b5b07/html5/thumbnails/29.jpg)
Future work (1/2)Expand on expressions of suicidality
oStudy how negation detection can be used to strengthen predictive power of mental health recordsoOngoing cohort study (pupils with ASD)o Large scale study based on hospitalisation events
![Page 30: Don’t Let Notes Be Misunderstood: A Negation Detection Method for Assessing Risk of Suicide in Mental Health Records](https://reader033.fdocuments.in/reader033/viewer/2022042906/589dfa761a28ab1e718b5b07/html5/thumbnails/30.jpg)
Future work (2/2)• Evaluate our tool against other datasets/domains
• Consider syntactic dependency parser (instead of constituency-based)• spaCy• SyntaxNet
![Page 31: Don’t Let Notes Be Misunderstood: A Negation Detection Method for Assessing Risk of Suicide in Mental Health Records](https://reader033.fdocuments.in/reader033/viewer/2022042906/589dfa761a28ab1e718b5b07/html5/thumbnails/31.jpg)
https://github.com/gkotsis/negation-detection
Source code