Vocal Biomarkers for Monitoring Neurological Disorders

Thomas F. Quatieri

Senior Technical Staff, MIT Lincoln Laboratory Faculty Harvard-MIT Health Science Technology

May 7, 2015

JAC Review- 2 TFQ 02/27/15

MIT Lincoln Laboratory Federally Funded Research and Development Center

Massachusetts Institute of Technology MIT Lincoln Laboratory, Lexington, Massachusetts

Structure: Ten Divisions (e.g., Homeland Protection, Communication Systems, Cyber Security) with about eight groups within each division

Bioengineering Systems and Technology Group: Preserve and enhance human health and performance through monitoring, analysis, and interventions

•  New group ~3 years: Highly interdisciplinary •  Staff: ~50 scientists, engineers, students, support •  Funding sources: DoD, NIH, Internal •  Broad technical areas: Biomedical research, synthetic biology, bioinformatics,

biometrics

Speech, hearing, and neuro-cognitive analysis

JAC Review- 3 TFQ 02/27/15

Depression

Traumatic Brain Injury Cognitive Overload

Parkinson’s

Lou Gehrig’s Disease

Alzheimer’s Disease

Post Traumatic Stress Disorder

Fragmented Sleep

Environmental Hot, Cold,

Altitude

Psychological Fear

Danger

Speech, Hearing and Neuro-cognitive Analysis Motivation, Objective, Approach

Objective •  Simple, sensitive method to detect and

monitor a condition •  Distinguish across conditions

Gait

Approach: Vocal biomarkers!•  Reflect underlying

neurophysiological changes that alter speech motor control !

•  Reflect coordination changes across speech production components, as well with other modalities!

!

Motivation!•  Many conditions that effect

cognitive performance!•  Includes neurological and stress

conditions!

!

JAC Review- 4 TFQ 02/27/15

Prediction of cognitive status in elderly

0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1Imbalanced Data ROC for Averaged Dataset

False Positive Rate

True

Pos

itive

Rat

e

Equal Error Rate(EER) = 18%

Data: audio from 200 elderly

Research in vocal biomarkers

Articulatory coordination

Phonetic timing

MIT Lincoln Research Focus

Cognitive Load

Evolving research areas

Subjects Ages Sport

8 female 14–18

24 male 14–18

Data: Full Season Athlete Collection (Purdue)

Timing

Coordination

Reaction Time

False Alarm 0 0.1 0.2 0.3 0.4 0.5

0.2

0.4

0.6

0.8

1.0

True

Pos

itive

Mild Traumatic Brain Injury

ALS

Parkinson’s

8

10 -

RM

SE

Team Rank

AVEC 2013

9 - MIT LL

1

A

2

B

3

11 Data (2013 AVEC Depression Challenge):

•  Audio from 50 train/50 test subjects

Objective:

•  Predict BECK depression assessment score from audio

Depression

Vocal Fatigue

Effectiveness of drug treatment: Interest in Apps for depression monitoring

Traumatic brain injury: Piloting Apps for NCAA

Depression: Launching worldwide Apps (with Satra Ghosh, MIT BCS)

From laboratory to mobile device

Objective:

•  Detect cognitive impairment

Dementia

Objective: Detect cognitive impairment (using IMPACT)

MIT LL/MIT BCS

JAC Review- 5 TFQ 02/27/15

Data Collections

Modeling

Making an Impact Research Areas

Clinical Acceptance

Advanced Feature

Extraction

JAC Review- 6 TFQ 02/27/15

Data Collections

Modeling

Making an Impact Research Areas

Advanced Feature

Extraction

Scientific Foundation Explain old and guide

new vocal biomarkers

Clinical Acceptance

JAC Review- 7 TFQ 02/27/15

Data Collections

Modeling

Making an Impact Research Areas

Advanced Feature

Extraction

Scientific Foundation Explain old and guide

new vocal biomarkers

Clinical Acceptance

JAC Review- 8 TFQ 02/27/15

Large-scale behavioral collections •  Audio databases with on-body platforms

–  Option to extract vocal features and remove audio •  Collections with related modalities (e.g., robust wireless EEG)

Imaging collections •  Brain, vocal tract, and vocal fold imaging

–  Improved real-time MRI; ultra high-speed 3D video •  Ultimate is simultaneous measurements during speaking

Improved protocols •  Speaking tasks that illicit specific parts of the brain and

speech motor processes •  Speaking tasks that bring out specific neural and motor

components effected by different neurological conditions

Research Areas Databases

Gait

JAC Review- 9 TFQ 02/27/15

Need for model-based approaches to enhance scientific foundation for use of vocal biomarkers

Computational neural modeling •  Basic neural circuitry of speech production •  Modulation by non-speech networks (e.g., limbic) •  Disturbances in the distressed brain •  Directions into Velocities of Articulators (DIVA)

model is one basis

Computational physiological modeling •  Understanding of multitude of muscles and their

coordination in speech production •  Disorders both in articulatory and laryngeal (vocal

fold) movement

Research Areas Modeling

Speech

Concept

Sentences and words

Syllables and phonemes Prosodics

Phonetic representation: Position/state of articulators/

folds

Timing and coordination of articulators and vocal folds

Neural signaling Muscle activation

Diff

eren

t bra

in r

egio

ns

Simple approximate view of speech production

Aud

itory

and

tact

ile s

elf-

mon

itori

ng

Limbic

Cognitive

JAC Review- 10 TFQ 02/27/15

•  Mapping of changes in neural and physiological models

to changes in the acoustic signal •  Robust and high-resolution signal processing to reflect

dynamic and subtle aspects of complex changes in neural and physiological systems, beyond standard features

Research Areas Advanced Feature Extraction

JAC Review- 11 TFQ 02/27/15

•  Objective measures as an aid, not replacement •  Early identification of neurologic disease onset •  Prediction of relapse or recovery •  Prediction should be specific as well as sensitive

– Many sub-classes of speech disorders common to a variety of neurological disorders

•  Monitoring should be personalized with biofeedback

Clinical Acceptance

JAC Review- 12 TFQ 02/27/15

Acknowledgments

Daryush Mehta and Bob Hillman – MGH Voice Rehabilitation Center Jordan Green – MGH Speech and Feeding Disorders Lab Satra Ghosh – MIT Brain and Cognitive Science Dept. Visar Berisha – Arizona State, Speech and Hearing Science

JAC Review- 13 TFQ 02/27/15

Publications

Trevino, A., Quatieri, T. F. and Malyska, N., “Phonologically-based biomarkers for major depressive disorder,” EURASIP Journal on Advances in Signal Processing: Special Issue on Emotion and Mental State Recognition from Speech, 42:2011–2042, 2011. Williamson, J.R., Quatieri, T.F., Helfer, B.S., Horwitz, R., Yu, B., and Mehta, D.D., “Vocal biomarkers of depression based on motor incoordination,” in Proceedings of the 3rd ACM international workshop on Audio/visual emotion challenge, 2013, pp. 41–48. Helfer, B.S., T. F. Quatieri, Williamson, J.R., Mehta, D.D., Horwitz, R ., and B. Yu, “Classification of depression state based on articulatory precision.,” in Interspeech, 2013, pp. 2172–2176. Quatieri, T. F. and Malyska, N., “Vocal-source biomarkers for depression: A link to psychomotor activity,” Proceedings of Interspeech, 2012. Horwitz, R., Quatieri, T.F., Helfer, B.S., Yu, B., Williamson, J.R., and Mundt, J., “On the Relative Importance of Vocal Source, System, and Prosody in Human Depression.” IEEE Body Sensor Network Conference, Cambridge, MA, May 2013. Helfer, B.S., Quatieri, T.F., Williamson, J.R., Keyes, L., Evans, B., Greene, W.N., Vian, T., Lacirignola, J., Shenk, T., Talavage, T., Palmer, J., and Heaton, K., "Articulatory dynamics and coordination in classifying cognitive change in preclinical mTBI," Interspeech 2014. Yu, B., Quatieri, T.F., Williamson, J.R., and Mundt, J., "Prediction of cognitive performance in an animal fluency task based on rate and articulatory markers," Interspeech 2014. L. Keyes, J. Su, T. Quatieri, B. Evans, J. Lacirignola, T. Vian, W. Greene, D. Strom, and A. Dai, “FY12 Line-Supported Bio-Medical Initiative Program: Multi-modal Early Detection Interactive Classifier (MEDIC) for Mild Traumatic Brain Injury (mTBI) Triage,” MIT Lincoln Laboratory Project Report LSP-41, 2012.