AMANDA L. MILLERCORNELL UNIVERSITY AND THE UNIVERSITY OF

BRITISH COLUMBIAAMANDAMI@INTERCHANGE.UBC.CA

Using Ultrasound for Language Documentation

Advantages of Ultrasound

Ultrasound is safe, portable and non-invasive.The first safe tool that allows us to image the

back of the tongue, including the tongue root.Allows us to view the majority of the tongue

(difficult to image the tongue tip), and thus will allow us to understand tongue shape better.

Difficulties with ultrasound

What we see is a “floating tongue” with no reference to hard bony structures of the head, neck or spine, which make it difficult to interpret images.

This means that tongue movement seen can be due to head movement or ultrasound probe movement.

It is difficult to image the tongue tip with ultrasound (though with some speakers it is possible).

Two types of lingual ultrasound

Mid-sagittal ultrasound – have the probe aligned lengthwise under the chin, to view a central slice of the tongue length – can be used to determine the part of the tongue used (tip, blade, front, dorsum, root).

Coronal ultrasound – have the probe turned sideways to view a coronal section of the tongue (difficult to get the same slice on different occasions, difficult to know where in the tongue the image has been taken) – can be used to determine if a sound is lateral or not.

Imaging the palate

The ultrasound rays reflect at the first interface with air, so a mid-saggital ultrasound image shows a white line which is the upper edge of the tongue, from the tongue tip to the root.

The palate can typically not be seen in the images, except when the tongue touches the palate.

In order to trace the palate, you can record an ultrasound movie of a swallow (have the subject drink water, hold it in his / her mouth, and swallow).

Different points of the palate can be seen in different frames of the movie, but one composite palate trace is made.

One issue is that the soft palate can move during the production of uvular sounds.

Ultrasound of a swallow from a Grootfontein !Xung speaker

(/Xai Tsubeb)

The “floating tongue” problem

You can either stabilize the head and probe to avoid any movement other than that due to speech , or track movements of the head and probe and correct the tongue trace for these movements.

Types of ultrasound machines

Sonosite Titan the smallest, most portable machine Limited to scanning at 30 fps Fairly good image quality.

Types of ultrasound machines

GE Logiq E scans at frame rates up to 125 fps Stellar image quality – incorporates many

image quality buttons found in larger machines into a portable machine.

Wide scan angle allows researchers to view the whole tongue

Types of ultrasound machines

Terason T3000 machine allows collection of ultrasound data and audio at frame rates up to 70 Hz

Uses a script to align the audio signal with the ultrasound video

Types of ultrasound machines

Interson’s USB probe called “See More”http://www.interson.com/Products/SeeMore153USBProbes/tabid/79/Defaul

t.aspxUltra portable (just need the probe and plug

it into your laptop)Affordable ($5-6K)Only 15 fps frame rate (can do sonorants,

vowels)

Head and probe stabilization: HATS

Probe Stabilization

Magic arm holds probe in place under the person’s head.

Microphone stand with telescoping boom arm holds probe in place under the person’s head.

Ultrasound stabilization headset holds probe in place under the person’s head, and locks in imaging sweet spot over the recording session.

Head Stabilization

Moldable Head Stabilizer from the Comfort Company

Board on chair with pillow – ask the subject to keep their head on the pillow

Head and Probe Movement Correction

Haskins Optically Corrected Ultrasound System (HOCUS) – not portable.

Palatoglossatron method and software – use two blue sticks with pink dots on them. Software tracks the movement of the pink dots, and corrects the tongue image for the movements in each frame.

Palatoglossatron method and Ultrasound stabilization headset

Mixing with the audio signal

In most systems, there is no way to record the audio signal into the ultrasound machine.

The Terason T3000 can do this, because it is a laptop.

30 fps ultrasound

Use an Audio – Video Mixer, such as the Canopus TwinPact100, which has frame locking, to mix the audio speech signal and the ultrasound video signal

The ultrasound video signal comes out of the machine either through the S-video port (Sonosite Titan), or the External monitor port (GE Logiq E)

Can be used for monophthongs, sonorants and fricatives, which have fairly stable articulations.

There is a delay in the mixing between the video and audio signals (2-4 frames).

CHAUSA

CorrectedHigh frame rateAnchoredUltrasoundSoftware Alignment

CHAUSA Hardware Architecture20

CHAUSA Software21

Ultrasound Stabilization Headset, without head movement correction

Post-head movement correction

Alignment24

• Articulatory to Acoustic Alignment achieved manually, by aligning 3 click bursts, and 3 [k] bursts in each file.

• Alignment was verified using a custom built Tri-Modal 3 ms pulse generator (designed by Engineer, Kenneth Finch, specifications achieve simultaneity to thousands of a millisecond)

Proof of Alignment25

Grootfontein !Xung Alveolar click

Introduction27

• Maddieson (1993) found tongue dorsum retraction following labial-velars in Ewe using EMA data (but sensor coils could not be placed far enough back to view the tongue root with EMA)

Hypothesis28

• Ingressive airstream that is involved in the articulation of Dagbani labial-velars results in tongue dorsum / tongue root retraction.

• Labial-velar fronting in Dagbani is the repair strategy found for the constraint

*C [TD/TR retraction] V [coronal]

Experimental Goals29

• To document labial-velar fronting in Dagbani

• To see if there is tongue dorsum and tongue root retraction as part of the production of labial-velars in Dagbani in the [ɨ] context.

Dagbani labial-velars30

ISP

Mangetti Dune !Xung clicks

Needed High frame rate ultrasound, as 30 fps ultrasound showed high degree of under-sampling (aliasing)

Ultrasound is perfect because it allows us to view the tongue dorsum / tongue root.

Experimental Goals32

• We investigate the rarefaction gestures in all four clicks in Mangetti Dune !Xung (Miller, Scott, Sands and Shah 2009), as well as the C-V transitions in click- high front vowel sequences, to determine whether tongue root retraction is present in the rarefaction gestures, and whether this is carried over into the following vowel.

Dental click – Rarefaction gesture33

Rarefaction gesture displays tongue body lowering, but not tongue rootretraction.

Alveolar click – Rarefaction gesture34

Rarefaction gesture involves simultaneous tongue body lowering &Tongue root retraction between Trace 3 and Trace 5

Lateral click – Rarefaction gesture35

Rarefaction gesture displays both tongue body lowering and tongueroot retraction.

Palatal click – rarefaction gesture36

There is tongue body lowering. Although the posterior constriction is far back, there is no retraction of the root proper during the rarefaction gesture.