MR Pharmacy: Adaptive User Interfaces and Biofeedback for ...
2
MR Pharmacy: Adaptive User Interfaces and Biofeedback for Therapy in Mixed Reality Environments Fariba Mostajeran * Human-Computer Interaction Group Universit ¨ at Hamburg ABSTRACT Virtual and augmented realities (VR/AR) can provide a broad range of possibilities for therapeutic applications. Virtual implementation of classical therapy methods can offer many advantages to both patients and caregivers. My research focuses on validating and improving the use of VR and AR in psycho-and physiotherapy. Moreover, I will employ several physiological measures in order to provide different forms of biofeedback and make the virtual environment (VE) adaptive to the user’s affective states. Index Terms: Human-centered computing—Human computer interaction (HCI)—;——Applied Computing—Life and medical sciences—consumer health 1 I NTRODUCTION My research has a general aim at using mixed reality (MR), in par- ticular, virtual reality (VR) and augmented reality (AR) for mental and physical health. This paper will describe some of my work on practical usage of VR for psychotherapy (cf. Section 1) and will summarize my research interest in AR applications in physiotherapy. 2 VIRTUAL PSYCHOTHERAPY VR has enormous potential to be used in psychotherapeutic ap- plications. Exposure therapy (ET) is a typical example of such applications. It is a widely used method in the treatment of sev- eral psychological disorders, such as various forms of phobias and anxiety disorders [7]. ET relies on this assumption that phobias are conditioned (i. e., learned) reactions and exposure provides an opportunity for deconditioning (i.e., corrective learning) of phobic responses [3]. Therefore, patients are carefully exposed to the anxi- ety stimuli, i. e. situations, people, objects, or places which trigger their anxiety, in order to enable them to gradually bring their phobia reactions under control. In this context, VR can facilitate the therapy by providing a safe, controlled environment for VR exposure therapy (VRET). For example, exposing to a virtual airplane and virtual sense of flying for treating the fear of flying is much safer and less expensive compared to in vivo (i.e., real life) exposure. VR provides also a broader range of control over the course of treatment. For instance, to treat the fear of public speaking one can easily change the number of the virtual agents (VAs) representing the audience from one to 100 or change their behavior. Having the same conditions in real life requires hiring and training of an enormous amount of actors for every single experiment. In my opinion, the most ambitious goal of the virtual therapy could probably be the standardization of its virtual prescriptions and treatment methods. This way, similar to taking, for example, a painkiller for a headache, one could receive prescribed ”virtual medications” for their needs. Equally compared with availability * e-mail: [email protected] of the standard pharmacological medications all around the world, virtual medications shall become standard and available all around the world as well. This part of my research focuses on validating and improving the use of VR in psychotherapy and making virtual environments (VEs) more adaptive to the user’s affective and biological states. Since physiological measurements are able to represent the internal affective states of the users [12], several bio-signals (e.g., heart rate (HR), heart rate variability (HRV), galvanic skin response (GSR), etc.) will be recorded and analyzed as well. In addition, bio-signals can provide a better understanding of the user’s internal states for the users themselves, since they often under- or overestimate their experience during the experiments. The physiological recordings can show, for example, how much a person perceives stress by measuring, for instance, the variability in their heart rate or hormone level. Feeding this information in real-time back to the user (as biofeedback [14]) can help them learn about their reactions to the conditions and hopefully, overcome their discomfort by getting control over their psychological behaviors. Furthermore, biofeedback can be integrated into the VR, so that the virtual environment changes according to the user’s affective states. For example, in an adaptive VR for treating the fear of public speaking, the VA might show gradually positive behavior (such as giving a smile) according to the user’s stress level. This hypothesis relies on the function of mirror neurons in pri- mates (including humans) which are activated not only during one’s own sensations and emotions but also during the observation of these acts in others [4]. In other words, when humans see for example that someone is sad or happy, the same neural structures in their brain get activated as if they were themselves sad or happy. Therefore, I will test whether detecting virtual positive stimuli (e.g. a smile in virtual avatars) can trigger the same emotion in humans (i.e. positive mood). And if so, I will use the results for making an affective and adaptive VE for some therapeutic applications such as stress relief and positive mood induction. In order to examine my hypotheses, I have been conducting some experiments on social anxiety and mood disorders in cooperation with University Medical Center Hamburg-Eppendorf (UKE). In the next section I will briefly describe one of my studies on social anxiety disorder. 2.1 Virtual Trier Social Stress Test Trier Social Stress Test (TSST) [8] is a widely used protocol for inducing moderate social stress in laboratory settings. It consists of an anticipation and a test period for delivering a free speech and performing mental arithmetic in front of three neutral audiences. Prior studies [6, 9, 11] support the use of VR for provoking public speaking anxiety. Yet several questions have not been answered. For example, all studies so far have experimented with a fixed number of VA. Therefore, the effects of any change in the number of VA has not been reported yet. I implemented several virtual versions of the TSST (VTSST) for my study, thereby I would like to address the following research questions: Q 1 Can the proposed VTSST evoke the user’s social anxiety as
Transcript of MR Pharmacy: Adaptive User Interfaces and Biofeedback for ...
MR Pharmacy: Adaptive User Interfaces and Biofeedbackfor Therapy in Mixed Reality Environments
Fariba Mostajeran*
Human-Computer Interaction GroupUniversitat Hamburg
ABSTRACT
Virtual and augmented realities (VR/AR) can provide a broad rangeof possibilities for therapeutic applications. Virtual implementationof classical therapy methods can offer many advantages to bothpatients and caregivers. My research focuses on validating andimproving the use of VR and AR in psycho-and physiotherapy.Moreover, I will employ several physiological measures in orderto provide different forms of biofeedback and make the virtualenvironment (VE) adaptive to the user’s affective states.
Index Terms: Human-centered computing—Human computerinteraction (HCI)—;——Applied Computing—Life and medicalsciences—consumer health
1 INTRODUCTION
My research has a general aim at using mixed reality (MR), in par-ticular, virtual reality (VR) and augmented reality (AR) for mentaland physical health. This paper will describe some of my work onpractical usage of VR for psychotherapy (cf. Section 1) and willsummarize my research interest in AR applications in physiotherapy.
2 VIRTUAL PSYCHOTHERAPY
VR has enormous potential to be used in psychotherapeutic ap-plications. Exposure therapy (ET) is a typical example of suchapplications. It is a widely used method in the treatment of sev-eral psychological disorders, such as various forms of phobias andanxiety disorders [7]. ET relies on this assumption that phobiasare conditioned (i. e., learned) reactions and exposure provides anopportunity for deconditioning (i.e., corrective learning) of phobicresponses [3]. Therefore, patients are carefully exposed to the anxi-ety stimuli, i. e. situations, people, objects, or places which triggertheir anxiety, in order to enable them to gradually bring their phobiareactions under control.
In this context, VR can facilitate the therapy by providing asafe, controlled environment for VR exposure therapy (VRET). Forexample, exposing to a virtual airplane and virtual sense of flying fortreating the fear of flying is much safer and less expensive comparedto in vivo (i.e., real life) exposure. VR provides also a broaderrange of control over the course of treatment. For instance, to treatthe fear of public speaking one can easily change the number ofthe virtual agents (VAs) representing the audience from one to 100or change their behavior. Having the same conditions in real liferequires hiring and training of an enormous amount of actors forevery single experiment.
In my opinion, the most ambitious goal of the virtual therapycould probably be the standardization of its virtual prescriptionsand treatment methods. This way, similar to taking, for example,a painkiller for a headache, one could receive prescribed ”virtualmedications” for their needs. Equally compared with availability
*e-mail: [email protected]
of the standard pharmacological medications all around the world,virtual medications shall become standard and available all aroundthe world as well.
This part of my research focuses on validating and improvingthe use of VR in psychotherapy and making virtual environments(VEs) more adaptive to the user’s affective and biological states.Since physiological measurements are able to represent the internalaffective states of the users [12], several bio-signals (e. g., heart rate(HR), heart rate variability (HRV), galvanic skin response (GSR),etc.) will be recorded and analyzed as well.
In addition, bio-signals can provide a better understanding ofthe user’s internal states for the users themselves, since they oftenunder- or overestimate their experience during the experiments. Thephysiological recordings can show, for example, how much a personperceives stress by measuring, for instance, the variability in theirheart rate or hormone level. Feeding this information in real-timeback to the user (as biofeedback [14]) can help them learn about theirreactions to the conditions and hopefully, overcome their discomfortby getting control over their psychological behaviors.
Furthermore, biofeedback can be integrated into the VR, so thatthe virtual environment changes according to the user’s affectivestates. For example, in an adaptive VR for treating the fear of publicspeaking, the VA might show gradually positive behavior (such asgiving a smile) according to the user’s stress level.
This hypothesis relies on the function of mirror neurons in pri-mates (including humans) which are activated not only during one’sown sensations and emotions but also during the observation of theseacts in others [4]. In other words, when humans see for example thatsomeone is sad or happy, the same neural structures in their brainget activated as if they were themselves sad or happy. Therefore, Iwill test whether detecting virtual positive stimuli (e.g. a smile invirtual avatars) can trigger the same emotion in humans (i.e. positivemood). And if so, I will use the results for making an affective andadaptive VE for some therapeutic applications such as stress reliefand positive mood induction.
In order to examine my hypotheses, I have been conducting someexperiments on social anxiety and mood disorders in cooperationwith University Medical Center Hamburg-Eppendorf (UKE). Inthe next section I will briefly describe one of my studies on socialanxiety disorder.
2.1 Virtual Trier Social Stress TestTrier Social Stress Test (TSST) [8] is a widely used protocol forinducing moderate social stress in laboratory settings. It consistsof an anticipation and a test period for delivering a free speech andperforming mental arithmetic in front of three neutral audiences.Prior studies [6, 9, 11] support the use of VR for provoking publicspeaking anxiety. Yet several questions have not been answered. Forexample, all studies so far have experimented with a fixed numberof VA. Therefore, the effects of any change in the number of VA hasnot been reported yet. I implemented several virtual versions of theTSST (VTSST) for my study, thereby I would like to address thefollowing research questions:
Q1 Can the proposed VTSST evoke the user’s social anxiety as
Figure 1: 15 neutral VAs
Figure 2: Three VAs provide positive feedback by gradually smiling athigh stressful moments
good as a real TSST (i.e. with real audiences)?
Q2 Do the number (three, six, or 15 (see Figure 1)), gender, age,and costume (formal suit vs. laboratory white coat) of the VAvary the intensity of the perceived anxiety?
Q3 Can giving biofeedback in the form of an overlay progress barwhich shows the user’s level of stress help users understandand control their social anxiety reaction?
Q4 Can receiving positive feedback from VA i.e. gradually smilingif the user’s stress level exceeds a threshold (see Figure 2) helpusers understand and control their social anxiety reaction?
In order to extract the user’s stress level, several questionnaires(e.g. State-Trait Anxiety Inventory) and physiological measures(e.g. salivary cortisol) are employed. The online extraction of thestress magnitude and duration is done using HR, HRV, and GSRrecordings. Moreover, I track the user’s head position and direction.This is particularly helpful for distinguishing the objects or avatars,at which the user looks or avoids looking during the test. This part ofthe experiment is based on those studies which suggest individualswith social phobia avoid mutual gaze in social interactions [13].
3 VIRTUAL PHYSIOTHERAPY
In this part of my research, I will explore various forms of correctivebiofeedback in AR for virtual physiotherapy. For example, I wouldlike to conduct an experiment in AR in which the real shaking handsof a patient in their early stages of Parkinson’s Disease (PD) arereplaced with virtual firm hand models. My hypothesis is that pro-viding gradual corrective feedback using such virtual hands (while
the real ones are not visible to the patient) during an intense physicaltraining period could improve the patient’s motor functions.
Several studies support different aspects of this hypothesis. Forinstance, the classic rubber hand illusion (RHI) experiment hasdemonstrated that a fake hand can be perceived as own hand [1].The RHI has been also shown in several VR studies such as [5].Furthermore, Lanier [10] has used the term ”homuncular flexibility”to explain the ability of the brain for adapting to and learning tomanipulate different body configurations. Besides proper integra-tion of haptic and visual feedback can enhance the illusion [15] andtherefore its application in physiotherapy. A successful exampleof combining various technologies (e.g. VR, visual-tactile feed-back, brain-computer interface) for neurorehabilitation is reportedin [2], where eight chronic spinal cord injury paraplegics could gainrecovery following 12 months of training.
In brief, the English aphorism ”fake it till you make it” can depictvery well my hypothesis: we might achieve our desired therapy goalsby providing positive and corrective feedback (e.g. bio-visual-hapticfeedback) in virtual training programs.
REFERENCES
[1] M. Botvinick and J. Cohen. Rubber hands’ feel’touch that eyes see.Nature, 391(6669):756, 1998.
[2] A. R. Donati, S. Shokur, E. Morya, D. S. Campos, R. C. Moioli,C. M. Gitti, P. B. Augusto, S. Tripodi, C. G. Pires, G. A. Pereira, et al.Long-term training with a brain-machine interface-based gait protocolinduces partial neurological recovery in paraplegic patients. Scientificreports, 6:30383, 2016.
[3] I. C. Fedoroff and S. Taylor. Psychological and pharmacologicaltreatments of social phobia: a meta-analysis. Journal of clinical psy-chopharmacology, 21(3):311–324, 2001.
[4] V. Gallese. The’shared manifold’hypothesis. from mirror neurons toempathy. Journal of consciousness studies, 8(5-6):33–50, 2001.
[5] W. A. IJsselsteijn, Y. A. W. de Kort, and A. Haans. Is this my handi see before me? the rubber hand illusion in reality, virtual reality,and mixed reality. Presence: Teleoperators and Virtual Environments,15(4):455–464, 2006.
[6] P. Jonsson, M. Wallergard, K. Osterberg, A. M. Hansen, G. Johansson,and B. Karlson. Cardiovascular and cortisol reactivity and habituationto a virtual reality version of the trier social stress test: a pilot study.Psychoneuroendocrinology, 35(9):1397–1403, 2010.
[7] A. N. Kaczkurkin and E. B. Foa. Cognitive-behavioral therapy foranxiety disorders: an update on the empirical evidence. Dialogues inclinical neuroscience, 17(3):337, 2015.
[8] C. Kirschbaum, K.-M. Pirke, and D. H. Hellhammer. The trier socialstress test–a tool for investigating psychobiological stress responses ina laboratory setting. Neuropsychobiology, 28(1-2):76–81, 1993.
[9] O. D. Kothgassner, A. Felnhofer, H. Hlavacs, L. Beutl, R. Palme,I. Kryspin-Exner, and L. M. Glenk. Salivary cortisol and cardiovascularreactivity to a public speaking task in a virtual and real-life environment.Computers in Human Behavior, 62:124–135, 2016.
[10] J. Lanier. You are not a gadget: A manifesto. Vintage, 2010.[11] E. Montero-Lopez, A. Santos-Ruiz, M. C. Garcıa-Rıos, R. Rodrıguez-
Blazquez, M. Perez-Garcıa, and M. I. Peralta-Ramırez. A virtualreality approach to the trier social stress test: Contrasting two distinctprotocols. Behavior research methods, 48(1):223–232, 2016.
[12] R. W. Picard and R. Picard. Affective computing, vol. 252. MIT pressCambridge, 1997.
[13] L. Schulze, B. Renneberg, and J. S. Lobmaier. Gaze perception in socialanxiety and social anxiety disorder. Frontiers in human neuroscience,7, 2013.
[14] M. S. Schwartz and F. Andrasik. Biofeedback: A practitioner’s guide.Guilford Publications, 2017.
[15] B. Spanlang, J.-M. Normand, D. Borland, K. Kilteni, E. Giannopoulos,A. Pomes, M. Gonzalez-Franco, D. Perez-Marcos, J. Arroyo-Palacios,X. N. Muncunill, et al. How to build an embodiment lab: achievingbody representation illusions in virtual reality. Frontiers in Roboticsand AI, 1:9, 2014.
