Synthesis of Walking Sounds for Alleviating Gait Disturbances in Parkinson's Disease

6
IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING, VOL. 22, NO. 3, MAY 2014 543 Synthesis of Walking Sounds for Alleviating Gait Disturbances in Parkinson’s Disease Matthew W. M. Rodger, William R. Young, and Cathy M. Craig Abstract—Managing gait disturbances in people with Parkinson’s disease is a pressing challenge, as symptoms can contribute to injury and morbidity through an increased risk of falls. While drug-based interventions have limited efcacy in alleviating gait impairments, certain nonpharmacological methods, such as cueing, can also induce transient improvements to gait. The approach adopted here is to use computationally-gen- erated sounds to help guide and improve walking actions. The rst method described uses recordings of force data taken from the steps of a healthy adult which in turn were used to synthe- size realistic gravel-footstep sounds that represented different spatio-temporal parameters of gait, such as step duration and step length. The second method described involves a novel method of sonifying, in real time, the swing phase of gait using real-time motion-capture data to control a sound synthesis engine. Both approaches explore how simple but rich auditory representations of action based events can be used by people with Parkinson’s to guide and improve the quality of their walking, reducing the risk of falls and injury. Studies with Parkinson’s disease patients are reported which show positive results for both techniques in re- ducing step length variability. Potential future directions for how these sound approaches can be used to manage gait disturbances in Parkinson’s are also discussed. Index Terms—Audio user interfaces, biomedical acoustics, non- invasive treatment, patient rehabilitation, sensory aids. I. INTRODUCTION G AIT disturbances are a major symptom in Parkinson’s disease (PD), restricting mobility and quality of life [3], and contributing to disability and morbidity through an increased risk of falling [14]. Some of the different types of impairments experienced by patients include freezing-of-gait (FOG) [24], reduced step length [25], increased spatio-tem- poral variability [8], and difculties with gait initiation [19]. While there are a number of pharmacological solutions to try and help manage the symptoms of Parkinson’s disease, such as levodopa [16], some gait-related symptoms appear resistant to such treatments [17], and may therefore benet from alternative/complementary forms of gait management. It is our interest to explore nonpharmacological, psychologi- cally-motivated interventions that could help improve activities Manuscript received May 29, 2013; revised September 10, 2013; accepted October 03, 2013. Date of publication October 25, 2013; date of current version April 28, 2014. This work was supported by the TEMPUS-G project funded by the European Research Council (210007 StIG). M. W. M. Rodger and C. M. Craig are with the School of Psychology, Queen’s University Belfast, BT9 5BN Belfast, U.K. (e-mail: [email protected]; [email protected]). W. R. Young was with theSchool of Psychology, Queen’s University Belfast, BT9 5BN Belfast, U.K. He is now with the Centre for the Study of Expertise, Brunel University, UB8 3PH London, U.K. (e-mail: [email protected]). Digital Object Identier 10.1109/TNSRE.2013.2285410 of daily living in people suffering from PD, either by compli- menting the effects of drug-based treatments or by recruiting nondopaminergic neural pathways to achieve improvements in motor performance [1]. The approach described here involves the use of movement-based sounds that are generated using computational synthesis and are presented to help improve the walking actions of PD patients. Existing research would suggest that management of gait im- pairments in PD should focus on the spatio-temporal consis- tency of walking actions. Increased variability in stride-to-stride timing is associated with higher risk of falls [24], while FOG is often preceded by progressive reductions in stride length [5]. Furthermore, step consistency is important for stable gait initi- ation [19]. In response to these ndings, we are looking to de- velop auditory guides that specify spatio-temporal information to get PD patients walking, keep them walking, and keep them walking steadily. Previous attempts at cueing gait in PD can be categorized ac- cording to whether they specify temporal information relating to stepping frequency (mostly acoustic cues), and spatial infor- mation relating to step length (mostly visual cues). However, re- cent work has suggested that there is an attentional cost of using a combination of cues to convey both temporal and spatial in- formation separately [2]. Hence, there is a clear need to develop novel sensory cueing strategies that specify both temporal and spatial information within a single cue modality. Previous work has shown that gait performance is compromised when PD pa- tients walk to music [4]. The assumption that walking whilst concurrently listening to music will place increased cognitive demands on the walker is likely to have discouraged attempts to develop acoustic guides for enhancing gait in PD patients. How- ever, this approach represents a signicant oversight. When pa- tients select their favorite musical track to walk to [4], the infor- mation presented in the acoustic information has no relevance to the required locomotor actions (aside from the possible trans- lation between the musical beat and cadence). This problem can be addressed by presenting acoustic information that is both temporally and spatially relevant to the required action, which can be achieved using a number of methods involving compu- tational synthesis and presentation of sounds to guide gait in PD patients. A key principle for the present research is the phenomenon of paradoxical kinesia; a term describing a suppression, although often brief, of Parkinsonian motor symptoms. For example, pa- tients who typically show bradykinesia or slowness of move- ment when asked to reach as fast as possible to grasp a ball placed on a table in front of them can suddenly move more quickly and stably when intercepting a ball in the same spot 1534-4320 © 2013 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

Transcript of Synthesis of Walking Sounds for Alleviating Gait Disturbances in Parkinson's Disease

IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING, VOL. 22, NO. 3, MAY 2014 543

Synthesis of Walking Sounds for AlleviatingGait Disturbances in Parkinson’s Disease

Matthew W. M. Rodger, William R. Young, and Cathy M. Craig

Abstract—Managing gait disturbances in people withParkinson’s disease is a pressing challenge, as symptoms cancontribute to injury and morbidity through an increased riskof falls. While drug-based interventions have limited efficacyin alleviating gait impairments, certain nonpharmacologicalmethods, such as cueing, can also induce transient improvementsto gait. The approach adopted here is to use computationally-gen-erated sounds to help guide and improve walking actions. Thefirst method described uses recordings of force data taken fromthe steps of a healthy adult which in turn were used to synthe-size realistic gravel-footstep sounds that represented differentspatio-temporal parameters of gait, such as step duration and steplength. The second method described involves a novel methodof sonifying, in real time, the swing phase of gait using real-timemotion-capture data to control a sound synthesis engine. Bothapproaches explore how simple but rich auditory representationsof action based events can be used by people with Parkinson’s toguide and improve the quality of their walking, reducing the riskof falls and injury. Studies with Parkinson’s disease patients arereported which show positive results for both techniques in re-ducing step length variability. Potential future directions for howthese sound approaches can be used to manage gait disturbancesin Parkinson’s are also discussed.

Index Terms—Audio user interfaces, biomedical acoustics, non-invasive treatment, patient rehabilitation, sensory aids.

I. INTRODUCTION

G AIT disturbances are a major symptom in Parkinson’sdisease (PD), restricting mobility and quality of life

[3], and contributing to disability and morbidity through anincreased risk of falling [14]. Some of the different types ofimpairments experienced by patients include freezing-of-gait(FOG) [24], reduced step length [25], increased spatio-tem-poral variability [8], and difficulties with gait initiation [19].While there are a number of pharmacological solutions totry and help manage the symptoms of Parkinson’s disease,such as levodopa [16], some gait-related symptoms appearresistant to such treatments [17], and may therefore benefitfrom alternative/complementary forms of gait management.It is our interest to explore nonpharmacological, psychologi-cally-motivated interventions that could help improve activities

Manuscript received May 29, 2013; revised September 10, 2013; acceptedOctober 03, 2013. Date of publication October 25, 2013; date of current versionApril 28, 2014. This work was supported by the TEMPUS-G project funded bythe European Research Council (210007 StIG).M.W.M. Rodger and C.M. Craig are with the School of Psychology, Queen’s

University Belfast, BT9 5BN Belfast, U.K. (e-mail: [email protected];[email protected]).W. R. Young was with theSchool of Psychology, Queen’s University Belfast,

BT9 5BN Belfast, U.K. He is now with the Centre for the Study of Expertise,Brunel University, UB8 3PH London, U.K. (e-mail: [email protected]).Digital Object Identifier 10.1109/TNSRE.2013.2285410

of daily living in people suffering from PD, either by compli-menting the effects of drug-based treatments or by recruitingnondopaminergic neural pathways to achieve improvements inmotor performance [1]. The approach described here involvesthe use of movement-based sounds that are generated usingcomputational synthesis and are presented to help improve thewalking actions of PD patients.Existing research would suggest that management of gait im-

pairments in PD should focus on the spatio-temporal consis-tency of walking actions. Increased variability in stride-to-stridetiming is associated with higher risk of falls [24], while FOG isoften preceded by progressive reductions in stride length [5].Furthermore, step consistency is important for stable gait initi-ation [19]. In response to these findings, we are looking to de-velop auditory guides that specify spatio-temporal informationto get PD patients walking, keep them walking, and keep themwalking steadily.Previous attempts at cueing gait in PD can be categorized ac-

cording to whether they specify temporal information relatingto stepping frequency (mostly acoustic cues), and spatial infor-mation relating to step length (mostly visual cues). However, re-cent work has suggested that there is an attentional cost of usinga combination of cues to convey both temporal and spatial in-formation separately [2]. Hence, there is a clear need to developnovel sensory cueing strategies that specify both temporal andspatial information within a single cue modality. Previous workhas shown that gait performance is compromised when PD pa-tients walk to music [4]. The assumption that walking whilstconcurrently listening to music will place increased cognitivedemands on the walker is likely to have discouraged attempts todevelop acoustic guides for enhancing gait in PD patients. How-ever, this approach represents a significant oversight. When pa-tients select their favorite musical track to walk to [4], the infor-mation presented in the acoustic information has no relevanceto the required locomotor actions (aside from the possible trans-lation between the musical beat and cadence). This problem canbe addressed by presenting acoustic information that is bothtemporally and spatially relevant to the required action, whichcan be achieved using a number of methods involving compu-tational synthesis and presentation of sounds to guide gait in PDpatients.A key principle for the present research is the phenomenon of

paradoxical kinesia; a term describing a suppression, althoughoften brief, of Parkinsonian motor symptoms. For example, pa-tients who typically show bradykinesia or slowness of move-ment when asked to reach as fast as possible to grasp a ballplaced on a table in front of them can suddenly move morequickly and stably when intercepting a ball in the same spot

1534-4320 © 2013 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

544 IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING, VOL. 22, NO. 3, MAY 2014

that has been rolled down a ramp [18]. Moreover, startle sounds(high intensity auditory stimuli) can cause PD patients to gen-erate arm forces with equivalent magnitude and speed comparedto healthy adults, which otherwise would be weaker and slower[1]. These results have been interpreted as showing that motorsymptoms of PD may be caused by degeneration of neuronsin the basal ganglia that are responsible for the generation ofself-guided actions, and that when patients are presented withaction-relevant external sensory information, or cues, such af-fected neural areas may be bypassed [22]. The current researchaims to utilize this phenomenon in the context of developing au-ditory guides that assist walking in people with Parkinson’s.Previous efforts to use sound to manage gait impairments in

PD patients, have typically involved rhythmic auditory stimu-lation [7], [13], [26]. This type of intervention generally pro-vides PD patients with an auditory metronome, or markedlyrhythmic music, and asks them to match consecutive footfallswith the onset of each beat. In spite of some success with this ap-proach in reducing gait variability, other research suggests thatthe technique of using metronomic stimuli is limited in its ef-fect [29]. Moreover, rhythmic sounds only specify step durationof gait, with no information relating to spatio-temporal proper-ties of walking actions. While the sounds used in the presentresearch are rhythmic in the sense that they repeat regularly (tospecify healthy walking actions), they are more complex thantraditional rhythmic auditory stimuli, as they sonify intra-strideinformation relating to additional gait parameters on top of stepduration.

II. SYNTHESIZING ACTION SOUNDS

The sounds created by real-world events can specify prop-erties of the event itself, such as the physical properties ofsounding objects, and interactions between such objects [12].This is particularly salient for action sounds, that is, soundsgenerated through human interaction with the world. Theperceptual resonance of action sounds offers a potentiallyrich source of cueing information to assist motor control inindividuals with Parkinson’s disease, as the action-specifyinginformation in the sounds may guide movements that patientswould normally find difficult to initiate or maintain. For thepresent purposes, the affected action that we are interested inrepresenting sonically is gait.Walking sounds are generated by the force interaction be-

tween the foot and the ground surface, due to the impulse im-parted onto the surface throughout the stance phase. On uncon-solidated or compliant surfaces, such as gravel or dry foliage,sounds are also generated by displacement of surface elementsunder the sole of the foot as pressure is dynamically impartedacross the cycle of the stance phase from heel contact throughto toe off. Hence, walking sounds may contain kinetic informa-tion about the force profile of the stance phase of gait and itsassociated spatio-temporal parameters. There is some evidenceto suggest that people are sensitive to this information. In ad-dition to walking speed, in some instances people can pick upcharacteristics of the walker, such as gender or emotional state,from the sounds of footsteps [27]. Moreover, complex walkingsounds, such as footsteps on gravel, may convey both temporal

(step duration) and spatial (step length) properties of gait. Evi-dence has shown that both young healthy adults [30] and peoplewith PD [31] can appropriately adjust step duration and steplength to the recorded sounds of different gravel footsteps.We were interested in using computational audio synthe-

sizing techniques to create artificial gravel walking sounds,which would have the same dynamic properties as recordedsounds, and theoretically the same perceptual cueing effectsfor PD patients. This project is based on the principle, adoptedby Foley artists, that if one can recreate the relevant invariantproperties of the type of interaction that causes a real-worldsound event, then the artificial sound will be heard as thoughit were the real one. To achieve this, an existing gravel soundsynthesis program [11] was adapted to use an approximationof the ground-reaction force (GRF) between the foot and thegravel surface as input control data for the sound synthesis.The advantages of using this sound synthesis technique overrecordings are greater control over the spatio-temporal pa-rameters of the presented gait, as the input vector parameterscan theoretically be adjusted to any step length or duration;the memory required to create the synthesized sound is lessthan that required to store and playback recorded audio files[10], making the synthesis approach easier to implement ondifferent devices; the timbre of the sounds can also be alteredto create perception of different walking surfaces, which mayhelp maintain patients’ interest, without the need to create newrecordings.To create synthesized gravel sounds based on real action

dynamics, standard gait ground reaction forces (GRFs) wererecorded. One healthy male adult age years walked intime with a metronome across a 6 m length walkway, in whichan 8-channel AccuGait force-plate was embedded in such away that the upper surface of the force plate was level withthe surface of the walkway. Force data from the plate sensorswere converted from analog to digital using an analog dataacquisition unit (National Instruments Inc., Austin, TX, USA)connected to a Dell (Precision T1650 Workstation) PC runningQualisys Track Manager (QTM) software (Qualisys, Sweden).Force data were sampled at 1000 Hz.Step length was specified to the walker using paper strips

attached to the walkway. GRF vectors were recorded at threestride lengths (50, 70, and 90 cm) and two step durations (600and 700 ms), producing kinetic data for six different spatio-tem-poral gait patterns. Each raw GRF profile was curve-fitted to aninth-order polynomial function using MATLAB (Mathworks,Natick, MA, USA)

(1)

where is the stance duration (time in which the foot is incontact with the ground) of the in samples.The coefficients of the polynomials for each gait param-

eter combination were used to generate input vectors of thegravel-sound synthesis patch, created in the PureData envi-ronment (http://puredata.info/). This patch, which was adaptedfrom [11], takes as input the 10 coefficients from the ninth-orderpolynomial function, the duration (T) of the stance phase ofthe step, and the inter-step interval (ISI), corresponding to

RODGER et al.: SYNTHESIS OF WALKING SOUNDS FOR ALLEVIATING GAIT DISTURBANCES IN PARKINSON’S DISEASE 545

Fig. 1. Synthesis of gravel footstep sounds using GRF recordings. a) Groundreaction data was recorded for steps made with specific step lengths and stepdurations. b) GRF data from steps for each gait parameter were curve-fitted to aninth-order polynomial function. indicates heel contact with surface and T isthe duration of the stance phase between heel-contact and toe-off from surface.c) The coefficients of these functions were used to control the input parame-ters (sound intensity, I; central filter frequency, F) of a gravel step synthesisprogram, generating sounds that correspond to particular step lengths and dura-tions. These sounds are played back to users who are asked to try to match thesounds heard with their own steps.

the cadence of the synthesized gravel walking sounds. Thepolynomial curve of duration, , controls the intensity leveland band-pass filter central frequency of a randomly generatednoise impulse train. The impulse train is generated by dividinga 300 Hz low-passed noise signal by a 2000 Hz low-passednoise signal and clipping the output to lie between 0.9 and0.9 of peak output level. The band-pass filter frequency isgenerated by a random value generator (0–30000) weightedby the control signal and clipped between 500 and 10 000 Hz.An intensity envelope, derived from the polynomial functioncontrol vector, was applied to the filtered impulse train. Audiodata was only processed for the stance phase duration, T, ofeach gravel footstep sound. Syntheses of consecutive footstepsounds were separated by the ISI duration, and consecutivefootstep sounds were alternately panned between left and rightstereo output channels.Audio output was processed in the PC’s Realtek HD sound-

card. Sounds were presented over Sennheiser HDR180 wirelessstereo headphones. This process is shown in Fig. 1.

A. Testing Synthesized Gravel Sounds

An initial study with young healthy adults showed that theywere able to adjust the spatial and temporal parameters of theirgait in accordance with the parameters prescribed by the syn-thesized gravel sounds, and that performance in this task wascomparable to using real recorded gravel step sounds [30].A pilot study was conducted on 10 idiopathic PD

patients (on medication) (males ; mean ageyears; mean diesase duration years;

mean diesase duration years; motor impairmentwas assessed using the Unified Parkinson’s Disease RatingScale where participant’s mean score was 28.4 11.6 (rangingbetween 15 and 46), to test the efficacy of using synthesizedgravel footstep sounds to improve step length and reduce vari-ability. Two conditions were used: control walk (no instructionsor guides); synthesized gravel sounds (70 cm). In the gravelsound condition, the participants were asked to try to imitate thesound they heard by walking in a way that would reproduce this

Fig. 2. Percentage changes in CoV of both step length and step duration whenwalking with synthesized gravel footstep sounds, compared with a control walk.Error bars denote 95% confidence intervals.

sound if they were walking on a gravel path. The step durationof gravel footstep sounds was set at 600 ms, which was takenfrom averaging control walk step durations to the nearest 100ms. The participants made two complete walks (up and down) a12 m path in each condition. Three-dimensional gait kinematicswere recorded using reflective markers attached to the toe andheel of each foot, recorded by 16 Qualisys Oqus-3 camerasconnected to a Dell PC running QTM software, sampling at200 Hz.The main measure taken was the relative change in co-ef-

ficient of variation (CoV) of the participants’ step length andstep duration between the control walks and the walks in thesynthesized gravel footstep condition. Baseline CoV measuresof step length and step duration were 9.20% (s.d. = 2.96%)and 5.28% (s.d. = 2.73%), respectively. When listening tosynthesized gravel footsteps while walking step length CoVwas reduced to 7.46% (s.d. = 2.64%) and step duration CoVwas slightly higher at 5.32% (s.d. = 2.68%). Statistical analysisshowed that the participants’ mean percentage change in CoVof step length was significantly lower than zero ( ,

, Cohen’s ), although change in CoV of stepduration did not significantly differ from zero ( ,

, Cohen’s ), as seen in Fig. 2. This indicatesthat variability in the spatial component of gait was reducedfor patients while walking with synthesized gravel soundscompared to no sounds.This pilot study shows promising results that walking to syn-

thesized gravel footsteps sounds can improve gait stability forindividuals with PD.

III. REAL-TIME SONFICIATION OF GAIT

While action-tailored sound synthesis offers one promisingnonpharmacological means to manage gait disturbances in PD,there are also a number of potential avenues in using real-timemovement sonification to provide patients direct augmentedfeedback of their actions with the goal of assisting walking.Real-time sonification of gait movements creates an additionalperceptual channel of information about walking, in additionto proprioception, kinesthetic, and optic flow [9]. This may aidgait by focusing perceptual attention on movement of the lower

546 IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING, VOL. 22, NO. 3, MAY 2014

Fig. 3. Real-time gait swing-phase sonification setup. Automatic detection of foot marker structures is used to generate positional data for each foot, which is thenstreamed to an audio synthesis program. This program triggers increasing pitch tones over displacement intervals once a velocity threshold is exceeded. Thesesounds are played directly back to users, creating a closed-loop, online gait sonification interface.

limbs [21], and may also be used in the context of providingtarget or guide sounds for patients to attempt to recreate throughtheir own movements.There exist a number of projects that use sonification to

enhance movement co-ordination, either as a training tool forsports, or in a rehabilitation context [15]. As stated above,our present purpose was to develop an interactive sonificationframework and technique specifically to enhance gait perfor-mance in PD. This is based on the following propositions.• Sonification of the “silent” part of a movement—sonifyingthe swing-phase of gait provides feedback about the com-ponent of gait that would not otherwise generate a spe-cific auditory profile. This is different from the previousapproach highlighted in this paper which utilizes naturallyoccurring auditory feedback associated with walking. Fur-thermore, sonifying the swing-phase when walking is theonly means to specify step length through sound usingkinematic, rather than kinetic, parameters.

• Provision of spatio-temporal sensory information—soni-fying the swing-phase can provide information about thedynamics of the action (changing-pitch), as well as the spa-tial extent of the action (end-pitch), thus providing bothadditional online sensory feedback, and goal-relevant in-formation.

• Guidance, not restriction—in existing sonificationprojects, auditory feedback is often used to indicatedeviation from a “typical” movement range [6], [28]. It isour intention not to restrict movements made by patientsby indicating “errors.” PD patients gait movement maynot necessarily be able to match a healthy template, butwe can provide a goal stride length and cadence and tailorthis to a patient’s own action capabilities.

Initial developments undertaken in theMovement InnovationLab at Queen’s University Belfast have implemented these prin-ciples in a real-time gait swing-phase sonification interface.An initial phase involved setting up real-time motion capture

of gait. L-shaped structures were built so they could be attachedto the shoe worn on each foot, using rigid lightweight foamand plastic mounts to place reflective markers. Five reflectivemarkers were attached to each structure in a unique config-uration for real-time automatic identification of the markersby the QTM system. A motion capture volume was set up

and calibrated in the Movement Innovation Lab at Queen’sUniversity Belfast, comprising 18 Qualisys Oqus-3 camerascovering a m m m volume, connected to a Dell(Precision T1650Workstation) PC running QTM. Multiple mo-tion capture recordings were taken of a healthy adult walkingwithin the limits of the motion-capture volume whilst wearingthe L-shaped structures, from which Automatic Identificationof Markers (AIM) models were generated (Qualisys, Sweden).These allowed the position of the marker adjacent to the heelon each structure to be tracked in real-time. This positional datawas used to control the synthesis of the swing sounds.Positional data (X and Y axes) from both L-structures were

streamed at 200 Hz from QTM to a PureData synthesis patchusing the Open Sound Control (http://opensoundcontrol.org/)protocol. The first block of this patch resolves the X and Y dis-placement changes into a single vector, , for each heel marker.The absolute difference in displacement between sample, ,and sample, , was calculated as an approximate measureof instantaneous absolute velocity. The swing-phase of the stepwas taken as initiated when this value exceeded a thresholdof 0.5 ms , which triggered the sonification synthesis. Thissounded a four-partial sine tone with fundamental frequencyMIDI-number 60 (261.63 Hz). Each subsequent tone wassounded as the displacement from stride initiation increasedin 0.1 m intervals, with pitch increasing by 1 MIDI numbereach interval, resulting in a chromatic glissando for each stride.Digital data was converted to sound through the PC’s RealtekHD soundcard, and presented to users through SennheiserHDR180 wireless stereo headphones, with sounds generatedby left and right foot markers panned to left and right ears,respectively. This setup is illustrated in Fig. 3.

A. Testing Real-Time Sonification of Swing-Phase of Gait

A PD patient pilot study using real-time gait sonification wasconducted with nine participants with idiopathic PD. Thesewere the same patients as described in Section II above, exceptfor one participant who was unable to complete this study dueto scheduling problems. The control condition was identical tothat described above. In addition, gait kinematics from anothercondition were recorded. In this condition, participants listenedto real-time sonification (as described above) of their gait whilewalking two walks (up and down) the 12 m walkway in each

RODGER et al.: SYNTHESIS OF WALKING SOUNDS FOR ALLEVIATING GAIT DISTURBANCES IN PARKINSON’S DISEASE 547

Fig. 4. Percentage changes in CoV of both step length and step duration whenwalking with real-time sonfication, compared with a control walk. Error barsdenote 95% confidence intervals.

condition. Movements were recorded in the same manner asdescribed above.As with the study using synthesized footstep sounds, the in-

dependent variable calculated was the relative change in CoVof step length and step duration between the control walks andthe walks while listening to real-time sonified step.Baseline measures of step length and step duration CoVs in

this study were 9.65% and 5.66%, respectively. When the present real-time sonification

technique was used during walking, patients’ step length CoVreduced to 7.08% , while step length CoV re-duced very slightly to 5.32% . Using this tech-nique, mean percentage change in step length CoV was signif-icantly lower than zero ( , , Cohen’s

), although as in the previous study, the mean per-centage change in step duration CoV was not found to signif-icantly differ from zero ( , , Cohen’s

). Mean percentage changes in step length CoV andstep duration CoV are shown in Fig. 4. These results suggestthat for PD patients the real-time sonification of their steps canlead to a significant reduction in the variability of spatial factorsof their gait.

IV. CONCLUSION AND FUTURE STEPS

The approaches described here use computationally gener-ated sounds to provide flexible, spatio-temporal information,and offer a promising nonpharmacological solution to the man-agement of gait disturbances in Parkinson’s disease. The firstapproach used ground-reaction force recordings to synthesizethe sounds of footsteps on gravel with specific step lengths anddurations, which can be picked up by users to adjust the spatio-temporal parameters of their gait. The second approach involvedreal-time sonifiaction of the swing-phase of gait, using motion-capture and audio processing software. Preliminary patient pilotstudies conducted in our laboratory (Movement Innovation Lab)suggest that both methods can be effective in reducing gait vari-ability in PD patients. These effects, if validated by testing largerpatient samples, provide the basis for novel, nonpharmacolog-ical tools to manage gait disturbances in PD.

It is relevant to note that both types of auditory gait cueingtechniques described here led to improvements in step lengthvariability, but not step duration variability, for these partici-pants. This suggests that the effects of auditory action cues actonly to improve consistency of spatial characteristics of gait andnot temporal parameters. Moreover, the baseline variability ofstep length of participants in both studies was greater than thatof step duration, which might suggest that there is less poten-tial to improve consistency in step duration compared with steplength. However, it has previously been shown that rhythmicauditory stimulation of gait in PD patients is only effective inreducing step duration variability if the sound is set to be fasterthan the patients’ preferred walking speed [13]. As these studiesdescribed here used sounds that fell within patients’ normal stepduration range, this may explain why reductions in step durationvariability were not obtained. Future research should examineany additional benefits from the present techniques when tem-poral parameters are set to cue gait faster than control walkingspeeds.Future work on this project will involve tailoring sounds

to suit patients. This can involve exploring different types ofsounds and timbres within types, as patients may have dif-ferent preferences and find some examples more aestheticallypleasing than others. Moreover, in conveying spatio-temporalinformation, some sound types may be more efficacious thanothers. Another step will involve matching sound playback topatients’ preferred cadence. Healthy elderly adults find it easierto synchronize with auditory rhythms when the tempo is closeto their preferred gait step duration [20], and this may also beimportant for PD patients in recruiting intact neural sensori-motor timing circuits for stabilizing gait performance. Finally,technological developments will be pursued to implement thesound approaches described here into portable devices for usein everyday life by patients. A major difficulty of the presenttechnology is the use of force plates and 3-D optical motioncapture techniques which are expensive and immobile. Futurework should research the possibilities of using lightweightaccelerometers to sense ground-reaction force and motion datafor controlling synthesis parameters in a mobile device. Suchdevelopments would allow the current techniques and findingsto have real-world clinical benefits for patients in everyday life.

ACKNOWLEDGMENT

The authors would like to thankN. Gillian and S. O’Modhrainfor their advice during early stages of the audio synthesis andexperiment design processes.

REFERENCES

[1] A. Anzak, H. Tan, A. Pogosyan, A. Djamshidian, H. Ling, A. Lees,and P. Brown, “Improvements in rate of development and magnitudeof force with intense auditory stimuli in patients with Parkinson’s dis-ease,” Eur. J. Neurosci., vol. 34, no. 1, pp. 124–132, Jul. 2011.

[2] K. Baker, L. Rochester, and A. Nieuwboer, “The immediate effect ofattentional, auditory, and a combined cue strategy on gait during singleand dual tasks in Parkinson’s disease,” Arch. Phys. Med. Rehab., vol.88, no. 12, pp. 1593–1600, 2007.

[3] B. R. Bloem, J. M. Hausdorff, J. E. Visser, and N. Giladi, “Falls andfreezing of gait in Parkinson’s disease: A review of two interconnected,episodic phenomena,” Movement Disord., vol. 19, no. 8, pp. 871–884,2004.

548 IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING, VOL. 22, NO. 3, MAY 2014

[4] L. A. Brown, N. de Bruin, J. B. Doan, O. Suchowersky, and B. Hu,“Novel challenges to gait in Parkinson’s disease: The effect of concur-rent music in single- and dual-task contexts,” Arch. Phys. Med. Rehab.,vol. 90, no. 9, pp. 1578–1583, 2009.

[5] R. Chee, A. Murphy, M. Danoudis, N. Georgiou-Karistianis, and R.Iansek, “Gait freezing in Parkinson’s disease and the stride length se-quence effect interaction,” Brain, vol. 132, pt. 8, pp. 2151–2160, 2009.

[6] L. Chiari, M. Dozza, and A. Cappello, “Audio-biofeedback for balanceimprovement: An accelerometry-based system,” IEEE Trans. Biomed.Eng., vol. 52, no. 12, pp. 2108–2111, Dec. 2005.

[7] M. F. del Olmo and J. Cudeiro, “Temporal variability of gait inParkinson disease: Effects of a rehabilitation programme based onrhythmic sound cues,” Parkinsonism. Relat. D., vol. 11, no. 1, pp.25–33, 2005.

[8] L. E. Dibble, D. E. Nicholson, B. Shultz, B. a MacWilliams, R. L.Marcus, and C. Moncur, “Sensory cueing effects on maximal speedgait initiation in persons with Parkinson’s disease and healthy elders,”Gait Posture, vol. 19, no. 3, pp. 215–225, 2004.

[9] A. Effenberg, U. Fehse, and A. Weber, “Movement sonification: Au-diovisual benefits on motor learning,” presented at the BIOWeb Conf.,Dec. 2011 [Online]. Available: http://www.bio-conferences.org/arti-cles/bioconf/pdf/2011/01/bioconf_skills_00022.pdf, unpublished

[10] A. Farnell, “Marching onwards: Procedural synthetic footstepsfor video games and animation,” presented at the Pure DataConvent., Montreal, Canada, Aug. 2007 [Online]. Available:http://obiwannabe.co.uk/html/papers/pdcon2007/PDCON-2007-FAR-NELL.pdf, unpublished

[11] A. Farnell, Designing Sound. Cambridge, MA: MIT Press, 2008.[12] W. Gaver, “What in the world do we hear?: An ecological approach

to auditory event perception,” Ecol. Psychol., vol. 5, no. 1, pp. 1–29,1993.

[13] J. M. Hausdorff, J. Lowenthal, T. Herman, L. Gruendlinger, C. Peretz,and N. Giladi, “Rhythmic auditory stimulation modulates gait vari-ability in Parkinson’s disease,” Eur. J. Neurosci., vol. 26, no. 8, pp.2369–2375, 2007.

[14] J. M. Hausdorff, D. a Rios, and H. K. Edelberg, “Gait variability andfall risk in community-living older adults: A 1-year prospective study,”Arch. Phys. Med. Rehab., vol. 82, no. 8, pp. 1050–1056, Aug. 2001.

[15] T. Hermann, J. Neuhoff, and A. Hunt, The Sonification Handbook.Berlin: Logos Publishing House, 2011.

[16] P. A. LeWitt, “Levodopa for the treatment of Parkinson’s disease,”NewEng. J. Med., vol. 359, no. 23, pp. 2468–2476, Dec. 2008.

[17] S. Lord, K. Baker, A. Nieuwboer, D. Burn, and L. Rochester, “Gaitvariability in Parkinson’s disease: An indicator of non-dopaminergiccontributors to gait dysfunction?,” J. Neurol., vol. 258, no. 4, pp.566–572, 2011.

[18] M. Majsak, T. Kaminski, A. Gentile, and J. Flanagan, “The reachingmovements of patients with Parkinson’s disease under self-determinedmaximal speed and visually cued conditions,” Brain, pp. 755–766,1998.

[19] R. T. Roemmich, J. R. Nocera, S. Vallabhajosula, S. Amano, K. M.Naugle, E. L. Stegemöller, and C. J. Hass, “Spatiotemporal variabilityduring gait initiation in Parkinson’s disease,” Gait Posture, vol. 36, no.3, pp. 340–343, Jul. 2012.

[20] M. Roerdink, P. J. M. Bank, C. L. E. Peper, and P. J. Beek, “Walkingto the beat of different drums: Practical implications for the use ofacoustic rhythms in gait rehabilitation,” Gait Posture, vol. 33, no. 4,pp. 690–694, Apr. 2011.

[21] M. D. Sage and Q. J. Almeida, “Symptom and gait changes after sen-sory attention focused exercise vs aerobic training in Parkinson’s dis-ease,” Movement Disord., vol. 24, no. 8, pp. 1132–1138, 2009.

[22] S. V. Sarma, M. L. Cheng, U. Eden, Z. Williams, E. N. Brown, andE. Eskandar, “The effects of cues on neurons in the basal ganglia inParkinson’s disease,” Front. Integrative Neurosci., vol. 6, p. 40, Jul.2012.

[23] J. D. Schaafsma, N. Giladi, Y. Balash, A. L. Bartels, T. Gurevich, andJ. M. Hausdorff, “Gait dynamics in Parkinson’s disease: Relationshipto Parkinsonian features, falls and response to levodopa,” J. Neurol.Sci., vol. 212, no. 1–2, pp. 47–53, 2003.

[24] J. M. Shine, S. L. Naismith, and S. J. G. Lewis, “The pathophysiolog-ical mechanisms underlying freezing of gait in Parkinson’s Disease,”J. Clin. Neurosci., vol. 18, no. 9, pp. 1154–1157, 2011.

[25] M. Svehlík, E. B. Zwick, G. Steinwender,W. E. Linhart, P. Schwingen-schuh, P. Katschnig, E. Ott, and C. Enzinger, “Gait analysis in patientswith Parkinson’s disease off dopaminergic therapy,” Arch. Phys. Med.Rehab., vol. 90, no. 11, pp. 1880–1886, Nov. 2009.

[26] M. H. Thaut, G. C. McIntosh, R. R. Rice, R. a Miller, J. Rathbun,and J. M. Brault, “Rhythmic auditory stimulation in gait training forParkinson’s disease patients,” Movement Disord., vol. 11, no. 2, pp.193–200, 1996.

[27] Y. Visell, F. Fontana, B. L. Giordano, R. Nordahl, S. Serafin, and R.Bresin, “Sound design and perception in walking interactions,” Int. J.Hum-Comput. Stu., vol. 67, no. 11, pp. 947–959, 2009.

[28] K. Vogt, D. Pirró, I. Kobenz, R. Höldrich, and G. Eckel, “Phys-iosonic—Movement sonification as auditory feedback,” presented atthe Int. Conf. Auditory Display, Copenhagen, May 2009 [Online].Available: http://iem.kug.ac.at/fileadmin/media/iem/projects/2009/physio.pdf, unpublished

[29] J. E. Wittwer, K. E. Webster, and K. Hill, “Music and metronome cuesproduce different effects on gait spatiotemporal measures but not gaitvariability in healthy older adults,” Gait Posture, pp. 8–11, 2012.

[30] W. Young, M. Rodger, and C. M. Craig, “Perceiving and reenactingspatiotemporal characteristics of walking sounds,” J. Exp. PsycholHuman, vol. 39, no. 2, Apr. 2013.

[31] W. R. Young, M. W. M. Rodger, and C. M. Craig, “Using ecologicalevent-based acoustic guides to cue gait in Parkinson’s disease patientsabstract,” Movement Disord., vol. 27, p. 119, Jun. 2012.

Matthew W. M. Rodger received the M.A. (Hons.)degree in philosophy and psychology from the Uni-versity of Edinburgh, Edinburgh , U.K., in 2006, andthe Ph.D. degree from the School of Music and Cre-ative Arts, Queen’s University Belfast, Belfast, U.K.,in 2010.He is currently a Lecturer in the School of Psy-

chology at Queen’s University Belfast, Belfast, U.K.His research interests are the use of auditory guidesto facilitate movement rehabilitation and skill acqui-sition, and the perception and production of musical

performance.

William R. Young received the B.Sc. degree, in2004, and the Ph.D. degree, in 2007, from TheSchool of Sport and Exercise Sciences, The Univer-sity of Birmingham, Birmingham, U.K., where hestudied interactions between gaze behavior and gaitparameters in older adults.In 2007, he joined the School of Psychology at

Queen’s University Belfast, developing tools forassisting mobility in older adults and Parkinson’sdisease patients. After a three-month visit to StanfordUniversity, he joined Brunel University, London,

U.K., in May 2013, as a Research Fellow. His current work examines howanxiety can compromise balance control and increase fall risk in older adults.

Cathy M. Craig received the M.A. and Ph.D. de-grees in psychology from Edinburgh University, Ed-inburgh, U.K., in 1993 and 1997, respectively.She worked as a Lecturer for eight years at the

Sports Science Faculty, the University of Aix-Mar-seille 2 France, where she obtained her H.D.R. in2006. She became a Senior Lecturer in psychologyat Queen’s University Belfast, Belfast, U.K., andwas promoted to Professor in 2010. She is nowHead of the School of Psychology and Director ofthe Perception Action and Communication research

group. Her research interests focus on how temporal patterns of sound and lightcan be exploited to help improve movement performance.