Detecting Linguistic Characteristics of Alzheimer's...

of 23 /23
Detecting Linguistic Characteristics of Alzheimer's Dementia by Interpreting Neural Models Sweta Karlekar Computer Science, UNC Chapel Hill Graduate Student: Tong Niu Research Mentor: Dr. Mohit Bansal

Embed Size (px)

Transcript of Detecting Linguistic Characteristics of Alzheimer's...

  • Detecting Linguistic Characteristics of Alzheimer's Dementia by Interpreting Neural Models

    Sweta Karlekar

    Computer Science, UNC Chapel Hill

    Graduate Student: Tong Niu

    Research Mentor: Dr. Mohit Bansal

  • Alzheimer’s Disease (AD)

    • Most common form of Dementia

    • Caused by cortical degeneration

    • Decline in language comprehension and ability

    • Medication can slow or halt progression

    Image courtesy of National Institute on Aging, National Institutes of Health, U.S. Department of Health and Human Services

  • Evaluation TechniquesMental Status and Mood Testing

    Physical and Neurological Exams

    Extensive Medical History

    Brain Imaging

  • The Task

    Transcripts of Spoken Languages

    samples

    Binary Classification of

    AD+ or AD-

  • But first, let’s look at the methodology.

  • ML vs. DL

    Image from: https://codeutsava.in/blog/40

  • Natural Language Processing

    Artificial Intelligence LinguisticsNLP

    GOAL: Have computers understand natural language to perform useful tasks.

  • NLP + Deep Learning

    Image from: https://www.xenonstack.com/blog/data-science/overview-of-artificial-intelligence-and-role-of-natural-language-processing-in-big-data

  • Back to the task…

  • Dataset

    • Dementia Bank dataset

    • Transcripts and speech

    samples

    • Non-AD + AD Patients

    • Includes POS tags

    • Noun, verb, adjective,

    adverb, present participle, determiner, etc.

    Image from: https://www.researchgate.net/figure/The-Cookie-theft-picture_fig1_317095410

  • Previous WorksAuthor ML vs. DL Description Accuracy

    Rudzicz et al. Machine Learning Extracted over 200+ lexical features 67.0%

    Orimaye et al. Machine LearningUsed syntactic,

    lexical, and n-gram features

    86.1%

    Konig et al. Machine Learning Analyzed speech audio 87.0%

    Orimaye et al. Deep Learning Deep Neural + Language Model 87.5%

  • Neural Models

    Recurrent Neural Network (RNN)

    Convolutional Neural Network (CNN)

    Convolutional/Recurrent Neural Network (CNN-RNN)

    Images from: http://colah.github.io

    http://colah.github.io

  • ResultsAuthor Model Description Accuracy

    Rudzicz et al. Machine Learning 200+ lexical features 67.0%

    Orimaye et al. Machine Learning Syntactic, lexical, and n-gram features 86.1%

    Konig et al. Machine Learning Speech audio 87.0%

    Orimaye et al. Deep Language Model Transcripts 87.5%

    - CNN Transcripts 82.8%

    - RNN Transcripts 83.7%

    - CNN-RNN Transcripts 84.9%

    - CNN-RNN Transcripts + POS 91.1%

  • But what did the neural model look at?

  • Saliency Heat MapsIn

    put T

    ext

  • Saliency Heat MapsIn

    put T

    ext

  • Saliency Heat MapsIn

    put T

    ext

  • Activation Clustering

    Image from: https://towardsdatascience.com/applied-deep-learning-part-1-artificial-neural-networks-d7834f67a4f6

  • Activation Clustering• Short answers and bursts of speech

    • “Okay”, “yes”, “oh!”, “yes”, “fine”

    • Repeated requests for clarification

    • “Did I say facts?”, “Did I get any?”, “Did I say elephant?”

    • Starting with interjections

    • “Well I gotta see it”, “Oh I just a lot of uh…”, “So all the words that you can”

  • Activation Clustering

  • Conclusion

    • Applied 3 different neural models to AD classification

    • Achieved a new benchmark accuracy

    • Utilized two visualization techniques

  • Future Work• Multi-class classification to differentiate among stages

    • Apply to other neurological diseases:

    • Huntington’s

    • Diffuse Lewy Body

    • How early can we catch AD in language?

    • Agatha Christie and Iris Murdoch novels

  • Questions?


JOURNAL OF LA An Investigation of Deep Learning Frameworks ... · this reason, RNN is considered to be a proper model for consistent “spoofing”. CNN+RNN: To further explore the
JOURNAL OF LA An Investigation of Deep Learning Frameworks ... · this reason, RNN is considered to be a proper model for consistent “spoofing”. CNN+RNN: To further explore the
Introduction to PyTorch - College of Computing · Introduction to PyTorch. Outline Deep Learning RNN CNN Attention Transformer Pytorch Introduction Basics Examples. Introduction to
Introduction to PyTorch - College of Computing · Introduction to PyTorch. Outline Deep Learning RNN CNN Attention Transformer Pytorch Introduction Basics Examples. Introduction to
NVIDIA DRIVE UPDATE€¦ · Sensor Complexity - More higher resolution cameras New DNN Models-CNN, RCNN, RNN DNN GOPs/Frame increasing 20+ DNN Models required for most demanding ODD
NVIDIA DRIVE UPDATE€¦ · Sensor Complexity - More higher resolution cameras New DNN Models-CNN, RCNN, RNN DNN GOPs/Frame increasing 20+ DNN Models required for most demanding ODD
RNN Explore
RNN Explore
Diagnosis of COVID-19 from X-rays Using Combined CNN-RNN ...€¦ · 24/08/2020  · Diagnosis of COVID-19 from X-rays Using Combined CNN-RNN Architecture with Transfer Learning .
Diagnosis of COVID-19 from X-rays Using Combined CNN-RNN ...€¦ · 24/08/2020  · Diagnosis of COVID-19 from X-rays Using Combined CNN-RNN Architecture with Transfer Learning .
Deep Learningcs230.stanford.edu/projects_spring_2018/reports/8291012.pdfFor the 2D CNN and RNN models, the spectrogram of the time-series data is used . as input (spectrogram is a
Deep Learningcs230.stanford.edu/projects_spring_2018/reports/8291012.pdfFor the 2D CNN and RNN models, the spectrogram of the time-series data is used . as input (spectrogram is a
CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN
CNN RNN Autoencoder - UNISTisystems.unist.ac.kr/wp-content/uploads/sites/209/2017/... · 2018. 12. 28. · CNN RNN Autoencoder. Today 1. Machine*Learning 2. Deep*Learning –CNN –RNN
Unconstrained Arabic & Urdu Text Recognition using Deep ... · Unconstrained Arabic & Urdu Text Recognition using Deep CNN-RNN Hybrid Networks Thesis submitted in partial fulfillment
Unconstrained Arabic & Urdu Text Recognition using Deep ... · Unconstrained Arabic & Urdu Text Recognition using Deep CNN-RNN Hybrid Networks Thesis submitted in partial fulfillment
Broadcast News Transcripts in Academic News Databases ... · Broadcast News Transcripts in Academic News Databases . ... CBS, CNN, FOX, NBC (with CNBC and MSNBC), NPR, and PBS. ...
Broadcast News Transcripts in Academic News Databases ... · Broadcast News Transcripts in Academic News Databases . ... CBS, CNN, FOX, NBC (with CNBC and MSNBC), NPR, and PBS. ...
Interactive Multi-Label CNN Learning with Partial Labels...CNN-RNN LSEP Wsabie Fast0Tag Latent Noise Latent Noise Curriculum IMCL (relevant) (visual) Labeling-0.6 0.1 0.1 0.0 0.9 1.0
Interactive Multi-Label CNN Learning with Partial Labels...CNN-RNN LSEP Wsabie Fast0Tag Latent Noise Latent Noise Curriculum IMCL (relevant) (visual) Labeling-0.6 0.1 0.1 0.0 0.9 1.0
Deep Learning CNN - RNN - Pytorch - University of Rochesterzduan/teaching/ece477/lectures...2016 : AlphaGo beatsLeeSedol on Go game 2016 : WaveNet synthesizes high-fidelity speech
Deep Learning CNN - RNN - Pytorch - University of Rochesterzduan/teaching/ece477/lectures...2016 : AlphaGo beatsLeeSedol on Go game 2016 : WaveNet synthesizes high-fidelity speech
Edition Cnn Com TRANSCRIPTS 1402 28 Ebo 01 HTML
Edition Cnn Com TRANSCRIPTS 1402 28 Ebo 01 HTML
JOURNAL OF LA An Investigation of Deep Learning Frameworks ...crss.utdallas.edu/Publications/spoof_jp.pdf · The most common and effective ... CNN+RNN: To further explore the advancement
JOURNAL OF LA An Investigation of Deep Learning Frameworks ...crss.utdallas.edu/Publications/spoof_jp.pdf · The most common and effective ... CNN+RNN: To further explore the advancement
Edge Intelligence: Convergence of Edge Compute …acm-ieee-sec.org/2018/AR_edge.pdf‒Innovation in algorithms: DNN, CNN, DBN, RNN Modern NNs loosely based on neural network function
Edge Intelligence: Convergence of Edge Compute …acm-ieee-sec.org/2018/AR_edge.pdf‒Innovation in algorithms: DNN, CNN, DBN, RNN Modern NNs loosely based on neural network function
Neural Networks Convolutional and Recurrentai-nlp-ml/course/dnlp/CNN-RNN-LSTM-Attenti… · Convolutional Neural Networks (CNN) A CNN consists of a series (≥ 1) of convolution layer
Neural Networks Convolutional and Recurrentai-nlp-ml/course/dnlp/CNN-RNN-LSTM-Attenti… · Convolutional Neural Networks (CNN) A CNN consists of a series (≥ 1) of convolution layer
Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é
Introduction to Deep LearningIntroduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN I2DL: Prof. Niessner, Prof. Leal-Taix é
Transformers and CNNskanmy/courses/6101_1810/w8-transformer.pdf · • Motivation • CNN is faster since the computations can be more parallelized compared to RNN • RNN needs to
Transformers and CNNskanmy/courses/6101_1810/w8-transformer.pdf · • Motivation • CNN is faster since the computations can be more parallelized compared to RNN • RNN needs to
Introduction to Deep Learning - TUM · Introduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN Prof. Leal-Taixé and Prof. Niessner
Introduction to Deep Learning - TUM · Introduction to Deep Learning Optimization CNN Introduction to NN Machine Learning basics Back-propagation RNN Prof. Leal-Taixé and Prof. Niessner
Patient MoCap: Human Pose Estimation under Blanket ... · Keywords: pose estimation, motion capture, occlusion, CNN, RNN, random forest 1 Introduction Human motion analysis in the
Patient MoCap: Human Pose Estimation under Blanket ... · Keywords: pose estimation, motion capture, occlusion, CNN, RNN, random forest 1 Introduction Human motion analysis in the
Edition Cnn Com TRANSCRIPTS 1401 21 Acd 01 HTML
Edition Cnn Com TRANSCRIPTS 1401 21 Acd 01 HTML