JOURNAL OF LATEX CLASS FILES, VOL. XX, NO. X, XXX XXXX 1

Effect of Avatar Appearance on DetectionThresholds for Remapped Hand Movements

Nami Ogawa, Takuji Narumi and Michitaka Hirose

Abstract—Hand interaction techniques in virtual reality often exploit visual dominance over proprioception to remap physical hand movementsonto different virtual movements. However, when the offset between virtual and physical hands increases, the remapped virtual handmovements are hardly self-attributed, and the users become aware of the remapping. Interestingly, the sense of self-attribution of abody is called the sense of body ownership (SoBO) in the field of psychology, and the realistic the avatar, the stronger is the SoBO.Hence, we hypothesized that realistic avatars (i.e., human hands) can foster self-attribution of the remapped movements better thanabstract avatars (i.e., spherical pointers), thus making the remapping less noticeable. In this paper, we present an experiment in whichparticipants repeatedly executed reaching movements with their right hand while different amounts of horizontal shifts were applied.We measured the remapping detection thresholds for each combination of shift directions (left or right) and avatar appearances(realistic or abstract). The results show that realistic avatars increased the detection threshold (i.e., lowered sensitivity) by 31.3% thanthe abstract avatars when the leftward shift was applied (i.e., when the hand moved in the direction away from the body-midline). Inaddition, the proprioceptive drift (i.e., the displacement of self-localization toward an avatar) was larger with realistic avatars for leftwardshifts, indicating that visual information was given greater preference during visuo-proprioceptive integration in realistic avatars. Ourfindings quantifiably show that realistic avatars can make remapping less noticeable for larger mismatches between virtual and physicalmovements and can potentially improve a wide variety of hand-remapping techniques without changing the mapping itself.

Index Terms—Virtual Reality, Body Ownership, 3D Interaction, Hand Interaction, Hand Redirection, Hand Retargeting, Avatar

�

1 INTRODUCTION

OUR perceptual system inherently integrates multisen-sory information from different sensory receptors to

obtain a robust and coherent perception of the environmentand our own bodies. When vision signals conflict withother sensory signals, visual information often becomesdominant, even without subjective awareness of the conflict(i.e., visual dominance or visual capture [1]).

In the field of virtual reality (VR), a number of hand in-teraction techniques, such as retargeting (e.g., [2]–[8]), redi-rection (e.g., [9], [10]), pseudo-haptics (e.g., [11]), and controlto display (C/D) ratio techniques (e.g., [12]), leverage visualdominance over proprioception. Typically, physical handmovements are remapped onto different virtual movements;thus, the position of the virtual hand is often displaced fromthe position of the actual hand. However, when the displace-ment increases, vision and proprioception are no longerintegrated. Consequently, the remapped movements cannotbe considered as one’s own movements, that is, they cannotbe self-attributed [13]. In these hand remapping techniques,larger displacement while ensuring that the remapping isless noticeable is a common challenge.

Interestingly, multisensory integration of body parts,which is the basis of the hand remapping techniques,also constructs bodily self-consciousness, specifically thesense of body ownership (SoBO, i.e., the self-attribution ofbody) [14]. SoBO is also elicited by a user’s hand and thevirtual hand’s synchronous movements as long as the dis-

• Nami Ogawa and Michitaka Hirose: The University of Tokyo

• Takuji Narumi: The University of Tokyo / JST PREST

Manuscript received XX XX, 2018; revised XX XX, 20XX.

Fig. 1. Participants execute a reaching movement with visual feedbackof the (a) human hand (realistic) or (b) spherical pointer (abstract) towhich horizontal shits are incrementally applied. The semi-transparentvirtual hand represents the physical location of the participant’s hand(not displayed in the actual experimental scene).

placement of the virtual hand is under a threshold [15]; how-ever, it is weakened when the displacement increases [16],as in the case of hand remapping techniques. However, onlya few studies have associated the SoBO with hand remap-ping techniques despite the fundamental link between them(e.g., [7], [17]). More importantly, SoBO is not only in-fluenced by visuomotor/visuo-proprioceptive congruencybut also by the appearances of an avatar (i.e., virtual self-representation); it is attenuated or even eliminated using ab-stract avatars than using realistic avatars [18]–[20] Therefore,we hypothesized that realistic avatars can better foster theself-attribution of remapped virtual hand movements thanabstract avatars, thus making the remapping less noticeable.

To the best of our knowledge, this study is the first toinvestigate the unexplored effect of avatar appearance onthe self-attribution of remapped movements. To this end,we conducted an experiment to measure the threshold ofself-attribution when one’s hand movements are remapped

This is the author's version of an article that has been published in this journal. Changes were made to this version by the publisher prior to publication.The final version of record is available at http://dx.doi.org/10.1109/TVCG.2020.2964758

Copyright (c) 2020 IEEE. Personal use is permitted. For any other purposes, permission must be obtained from the IEEE by emailing pubs-permissions@ieee.org.

JOURNAL OF LATEX CLASS FILES, VOL. XX, NO. X, XXX XXXX 2

Fig. 2. A virtual hand is displayed at the position either in the left orright direction (from a first person perspective) of the physical hand. Theparticipants adjusted their trajectory to compensate for the shift so thatthe virtual hand could reach the virtual target.

onto different virtual movements. In the experiment, wecompared two kinds of the avatar appearances: realistic andabstract (i.e., a realistic human hand vs. a spherical pointer;Figure 1). Participants executed reaching movements withtheir right hand while horizontal (left or right) shifts wereincrementally applied at different angles in the trials (Fig-ure 2). They were asked to discriminate if the remappedmovement that they observed matched the movement oftheir own hand after each reaching movement. The resultsshow that the threshold for remapping detection is ap-proximately 31.3% higher when using realistic avatars thanwhen using abstract avatars when shifts were applied inthe left direction (i.e., when the physical hand moved ina direction away from the body-midline). In addition, wemeasured proprioceptive drift (i.e., the displacement of pro-prioceptive self-localization toward a perceived virtual handfrom the actual hand position), which is a measurementof the strength of the body ownership illusion (BOI) [21].The results show that the proprioceptive drift was largerwith realistic avatars than with abstract avatars in the caseof leftward shifts. This finding indicates that visual infor-mation is given greater importance in visuo-proprioceptiveintegration when realistic avatars are used, as comparedto abstract avatars. For both measurements, no significantdifferences were observed between the two avatars whenthe rightward shift was applied. To summarize, the resultsshow that compared with an abstract avatar, the realisticavatar makes remapping less noticeable and brings theproprioceptive hand position closer to that of the virtualhand. The use of realistic avatars instead of abstract avatarscan potentially improve a wide variety of hand interactiontechniques because it can mitigate a range of unnoticeabledisplacements without changing the mapping model itself.

2 RELATED WORK

2.1 Visual Dominance in Hand Interaction TechniqueOwing to the nature of human multisensory integration,when vision and other sensory signals conflict moderately,these signals are integrated to generate a single estimate.”For instance, in the well known ventriloquist effect, the

speech sounds is perceived as coming from the location ofthe dummy’s moving mouth rather than the ventriloquist’sunmoving mouth [22]. In such case, vision often dominatesother sensory information; thus, the integrated perceptionrelies more on visual information (i.e., visual dominance orvisual capture [1]). Visual dominance holds true, particu-larly over the senses that have lower spatial resolution thanvision, such as proprioception [23]–[27], although this is notalways the case, depending on sensory combinations andtheir relative reliability [28]–[31].

In the field of VR, visual dominance over proprioceptionis often exploited in hand interaction techniques, where themovement of a virtual hand is remapped for a number ofpurposes (i.e., hand remapping technique). For example, ina series of retargeting techniques, the reaching movementsof a virtual hand are mapped to guide the user’s actualhand toward haptic props for providing passive haptics formultiple virtual objects [2]–[5]. On the other hand, handredirection techniques enable modification of the perceivedproperties, such as shape, of a physical object when theuser continuously touches and explores surfaces [9], [10].These techniques can also be used to prevent a virtualhand from interpenetrating other virtual objects [4], [32]and to improve the perceived performance of shape dis-plays [8]. Although movement remapping is often combinedwith passive haptics, it can also be used without hapticsto make virtual targets more easily accessible [6] and tomake overhead interactions less tiring [7]. Furthermore,pseudo-haptics simulate or enrich haptic sensations suchas stiffness, the presence of bumps and holes, and shapewithout necessarily requiring a haptic interface (for a reviewsee [11]). Lastly, the C/D ratio interaction techniques breakthe 1:1 mapping between the actual and virtual hands (i.e.,control to display ratio) to improve interaction by increasingprecision or speed [12].

In hand remapping techniques, making movementremapping less noticeable is a common challenge. Inessence, a large discrepancy between the position of thereal and virtual hands causes the user to become aware ofthe remapping (e.g., up to ≈ 4.5◦ [17]). Hence, to applylarger shifts, researchers attempted to find a better mappingbetween the virtual and real coordinate systems. For in-stance, in the haptic retargeting [2], the entire virtual worldis rotated, as in the redirected walking technique [33]–[35]as well as the mapping of the bodies. Similarly, Feuchtnerand Muller proposed Ownershift, which gradually applies ashift after ballistic movements [7]. However, these solutionshave limitations in that they limit the user movements orsituations unnaturally. In this study, we aim to alleviate therange of unnoticeable offset by focusing on how we perceivethe virtual hand movements as our own, instead of findingbetter mappings.

2.2 Illusory Self-Attribution of Remapped MovementsStudies in the field of psychology and cognitive neuro-science have also demonstrated that we can self-attributevisual hand movements, which are different from realhand movements. The sense of self-attribution of handmovements has two aspects with respect to bodily self-consciousness: the sense of agency (SoA, i.e., the self-attribution of action) and SoBO (i.e., the self-attribution

This is the author's version of an article that has been published in this journal. Changes were made to this version by the publisher prior to publication.The final version of record is available at http://dx.doi.org/10.1109/TVCG.2020.2964758

Copyright (c) 2020 IEEE. Personal use is permitted. For any other purposes, permission must be obtained from the IEEE by emailing pubs-permissions@ieee.org.

JOURNAL OF LATEX CLASS FILES, VOL. XX, NO. X, XXX XXXX 3

of body). Generally, visuomotor inconsistencies caused byspatial or temporal remapping of movements attenuate bothSoA and SoBO [36]–[40]. However, as in the case of handremapping techniques, it has been shown that SoA andSoBO are elicited as long as the displacement is under acertain threshold [41]. Indeed, SoA and SoBO can occur overdrifting avatars [15], movements made by someone else [42],[43], and even over an avatar’s autonomous movementswithout the user’s actual movements [44], [45].

Hand remapping techniques in VR are often explainedthrough visual dominance alone. However, as the tech-niques inevitably use virtual bodies, the knowledge ofbodily self-consciousness should also be incorporated intoits theory and evaluation. Nevertheless, only a few studieshave addressed the concept of SoA and SoBO in handremapping techniques. Only recently, Debarba et al. [13]investigated the effect of the type of remapping (i.e., helpingor hindering) on self-attribution of hand movements, andthey discussed their results in terms of SoA. In contrast, thefirst study which evaluated the users’ SoBO with a handremapping technique was recently conducted by Feuchtnerand Muller [7], who proposed a novel remapping techniquefor overhead interaction. Zenner and Kruger [17] also eval-uated the SoBO in conjunction with detection thresholds forhand redirection techniques as one of various indicators ofsubjective impressions.

In contrast, we hypothesize that the induction of astronger SoA or SoBO can effectively foster the self-attribution of virtual hand movements, despite visuo-proprioceptive discrepancies. That is, the detection thresh-old for discrepancy can be increased. In this study, weattempted to manipulate the strength of SoBO rather thanSoA, as SoBO is greatly influenced by the semantic featuresof the visual body [14], which can be easily manipulatedin VR. In contrast to SoBO, the main cue for SoA is thespatiotemporal contiguity between one’s own and observedmovements or outcomes [36]–[38] ; this reduces according tothe increase in the shift of remapping. Nevertheless, studiessuggest that SoBO and SoA may strengthen each other ifthey co-occur [46]–[49], although some studies have indi-cated that both experiences can double dissociate (for a re-view, see [50]). Hence, we consider that strengthening SoBOmay influence even the detection thresholds of discrepancybetween the virtual and actual movements, although it canbe related to the sense of self-attribution of action (i.e., SoA)rather than that of body (i.e., SoBO).

2.3 Body Ownership Illusion (BOI)

The experience of SoBO over an artificial (i.e., rubber handillusion; RHI [21]) or a virtual (i.e., virtual hand illusion; VHI[15], [39]) body is called the BOI. Typically, synchronousvisuo-tactile stimuli are given to the artificial and real un-seen hand in RHI, whereas visuomotor congruency is thekey in VHI. Similar to the hand remapping techniques,BOI is weakened by the displacement between the real andartificial or virtual hand [16], [40], [51].

In this section, we first introduce the effect of avatarappearance as an important factor of BOI. Then, we reviewthe proprioceptive drift as one of the measurements of BOI.Lastly, we introduce interoceptive sensitivity, which is said

to be related to individual differences in susceptibility ofBOI.

2.3.1 Effect of Avatar AppearanceThe induction of BOIs depends critically on bottom-upfactors, that is, the spatiotemporal congruency betweenmultisensory/sensorimotor stimuli. Nevertheless, the BOIis driven by an interplay between bottom-up and top-downfactors; the other principle underlying BOIs is that the closerthe appearance of the self-avatar is to our own bodies interms of structural and morphological aspects, the strongerthe SoBO (for review, see [14], [52]). In fact, a realistic virtualhuman hand elicits a stronger SoBO than nonhuman hands(e.g., robotic, cartoon) [18]–[20] and non-anthropomorphicobjects (e.g., sphere, block, and arrow) [15], [18], [19] in VHI.Interestingly, Yuan and Steed [15] reported that realisticvirtual hand elicited SoBO even when the virtual hand wasgradually shifted to 10 cm, but not for an abstract arrowcursor.

Recently, a few studies revealed the effect of the appear-ance of virtual hands on how sensory inputs are processed.Ogawa et al. [53] showed that a realistic avatar fosters theeffect of body-based scaling, in which object size perceptionis influenced by the body size. Moreover, Schwind et al. [20]revealed that the appearance could affect how participantsintegrate conflicting visual and haptic signals when theydetect surface irregularities (bumps and holes). Hence, inthe case of visuo-proprioceptive integration, the realisticavatar may foster the integration of conflicting information,resulting in further leveraging of visual dominance.

2.3.2 Proprioceptive DriftIn BOIs, subjective strength of SoBO is typically measuredthrough a questionnaire. In addition, a phenomenon knownas proprioceptive drift, in which the perceived locationof one’s own hand drifts toward the displaced artificialhand, has been classically considered an implicit, objectivemeasure of BOI [14], [52]. Indeed, several studies have con-firmed that the magnitude of proprioceptive drift correlateswith the questionnaire scores (e.g., [21], [46]), although therelationship between them is a matter of debate [54]–[58].The avatar appearance also affects the proprioceptive drift;a non-anthropomorphic object reduces the amount of thedrift, compared to realistic virtual [58] and artificial [59]hands. Hence, we hypothesized that the proprioceptive driftas well as the thresholds of self-attribution of remappedmovements was influenced by the avatar appearance. Theproprioceptive drift can not only be used as a measurementof SoBO but also of how vision dominates proprioception.

2.3.3 Interoceptive SensitivityThere are individual differences in detecting visuomotor orvisuo-proprioceptive discrepancies over remapped move-ments. Yet, the cause of these differences has not beenwell understood. In contrast, individual differences in thesusceptibility to BOIs (i.e., malleability of body representa-tions) are shown to be related to interoceptive sensitivity,which refers to an individual’s ability to sense the internalphysiological conditions of the body [60], [61]. Interoceptivesensitivity is usually considered an individual trait and

This is the author's version of an article that has been published in this journal. Changes were made to this version by the publisher prior to publication.The final version of record is available at http://dx.doi.org/10.1109/TVCG.2020.2964758

Copyright (c) 2020 IEEE. Personal use is permitted. For any other purposes, permission must be obtained from the IEEE by emailing pubs-permissions@ieee.org.

JOURNAL OF LATEX CLASS FILES, VOL. XX, NO. X, XXX XXXX 4

Fig. 3. (a) In the remapping detection task, participants answeredwhether the movements of the avatar corresponded to their own move-ments. (b) In the self-localization task, participants estimated their actualhand position (represented by a semi-transparent hand; not displayed inthe actual experimental scene) by shifting the position of a thin boardby using a controller. The difference between the estimated and actualpositions is referred to as proprioceptive drift.

measured by a task wherein a user detects or counts ownheartbeat [62]. Tsakiris et al. [60] showed that individualswith low interoceptive sensitivity experience stronger SoBO,indicated by both questionnaire and proprioceptive drift. Itis considered that they rely mainly on exteroceptive signals(e.g., vision and touch) during body perception; hence, theyare easily misled by the BOI. Thus, in the experiment, wemeasured interoceptive sensitivity to determine if it couldalso predict the individual differences in the self-attributionof remapped movements.

2.4 HypothesesOur hypotheses were as follows:H1 The detection threshold of the movement remapping is

higher with realistic avatars than with abstract ones.H2 The amount of proprioceptive drift is larger with real-

istic avatars than with abstract ones.H3 The participants with lower interoceptive sensitivity

tend to exhibit higher thresholds.H4 The participants with lower interoceptive sensitivity

tend to be well influenced by avatar appearance interms of the thresholds.

3 EXPERIMENT

The main objective of the experiment was to investigatewhether 1) a realistic avatar makes the movement remap-ping less noticeable compared with the abstract avatar(remapping detection task; Figure 3 (a)). To further supportthis objective, we also studied whether 2) a realistic avatargives greater importance to the visual information for visuo-proprioceptive integration than an abstract avatar (self-localization task; Figure 3 (b)). In both tasks, participantsreached for a virtual button with visual feedback of theirright hand; this movement was remapped by different shiftsfor each trial. The visual feedback was either a realistic or anabstract self-avatar (i.e., realistic hand or spherical pointer;Figure 1). In the remapping detection task, we asked theparticipants to discriminate whether the visual movementscorresponded to their own movements to estimate the de-tection threshold for remapping. In the self-localization task,we asked the participants to estimate the position of theiractual index fingertip without observing it to measure theproprioceptive drift. In addition, we investigated whether3) individual differences in the perception of remapped

Fig. 4. Top view of the retargeting model used in the experiment,illustrating the case with a right-shift and realistic avatar. The abscissaand ordinate represent the horizontal and depth axes, respectively. Theposition of virtual fingertip (p’) was obtained by horizontally shiftingthe position of physical fingertip (p) according to the shift (s) during areaching movement. Smax, which is the maximum shift applied whenthe virtual fingertip reaches the target, changed for every trial.

movements can be explained by interoceptive sensitivity, anindex of a personality trait that reportedly predicts the sus-ceptibility to BOIs (heartbeat counting test). In the heartbeatcounting test, participants silently counted their heartbeatwithout tracking their pulse to measure interoceptive sensi-tivity. Finally, we measured the subjective SoBO through aquestionnaire.

3.1 Remapping Model

We used a basic retargeting model that linearly remapsa hand movement by a fixed angle. During a reachingmovement, a horizontal shift is incrementally applied tothe virtual hand position depending on its distance tothe target. The participants were exposed to a continuousdirectional discrepancy between the virtual and physicalhand movements. Therefore, they adjusted their trajectoryto compensate for the shift such that the virtual hand cansuccessfully reach the virtual target. This type of remappingmodel is often accompanied by a physical object to providepassive haptic feedback (e.g., [2]–[4]). Nevertheless, we didnot provide the participants with haptic feedback so as toverify the effect of avatar appearance by using the simplestmodel possible.

As shown in Figure 4, the model maps the physicalposition of an index fingertip (p) onto a virtual position(p′) according to a shift (s) when the depth position of pis between the initial and target positions (Oz < pz < Tz).Smax represents the maximum shift applied when p reachesT, and it changed for every trial. Smax takes a positive valuefor a rightward shift and a negative value for a leftwardshift. If D and d are the distance between O and T and

This is the author's version of an article that has been published in this journal. Changes were made to this version by the publisher prior to publication.The final version of record is available at http://dx.doi.org/10.1109/TVCG.2020.2964758

Copyright (c) 2020 IEEE. Personal use is permitted. For any other purposes, permission must be obtained from the IEEE by emailing pubs-permissions@ieee.org.

JOURNAL OF LATEX CLASS FILES, VOL. XX, NO. X, XXX XXXX 5

Fig. 5. (a) Front view of physical setup. (b) Arrangement of virtualstimulus (side view). The starting point and target button were placed10 cm to the right of the participants’ body mid-line.

between p and O along the depth direction, respectively(D = Tz − Oz , d = pz − Oz), s is given as s = d

D · Smax.In addition, to prevent the virtual fingertip from penetratingthe button, pz is restricted to below Tz , whereas the physicaland virtual positions are collocated when pz is smaller thanOz . To summarize, the following equations are used tocalculate p′:

(p′x, p′z) =

(px + Smax, Tz) (pz ≥ Tz)

(px + s, pz) (Oz < pz < Tz)

(px, pz) (pz ≤ Oz)

During the experiment, we manipulated the value ofSmax for every trial while O, T, and D remained constant(D = 20 cm). Hereafter, we refer to Smax as shift.

3.2 ParticipantsBefore the experiment, we performed a power analysis toestimate the necessary sample size using G*Power withregards to the main response variable (i.e., detection thresh-old). Based on the prior analysis of the informal pilotexperiment, we expected the effect size to be large. Foran effect size of 0.8, a significance level of 0.05, and apower of 0.8, the minimum sample size was computed as15. Thus, we set the sample size to 16 and conducted theexperiment with 16 participants. However, as we had toexclude the data from one participant (see Section 4.1 fordetails), we recruited another participant. Consequently, atotal of 17 individuals participated (14 males and 3 females;24.53 ± 4.42 (SD) years old; 169.94 ± 7.77 (SD) cm tall). Allparticipants were recruited through social media, unawareof the true purpose of the experiment, and had normal orcorrected-to-normal vision. We did not recruit participantswith glasses because they could have difficulties in correctlywearing a head-mounted display (HMD). All participantsexcept one self-reported as being right-handed. Nine partic-ipants had limited experience with VR, whereas eight werefamiliar with VR. They signed an approved statement ofconsent, and they were compensated with an Amazon giftcard amounting to approximately $10. The experiment wasapproved by the local ethical committee.

3.3 Materials3.3.1 ApparatusThe experimental apparatus included a Windows-basedcomputer (with an Intel i7-8750H, 16 GB RAM, and an

NVIDIA GTX1060), a hand-tracking sensor (Leap MotionController), an HMD set (Oculus Rift CV1), a controller(Oculus Remote), and a pulse oximeter (KONICA MI-NOLTA PULSOX-Lite). The experimental program was de-veloped using Unity 3D.

3.3.2 Hand AppearanceIn line with the previous studies (e.g., [18]–[20]), we adoptedthe Leap Motion Controller as the hand-tracking device.This controller can track the forearm, hand, and fingers ofthe users within a 1-m radius without the need to weargloves or markers. Its estimations deviate from the actualpositions by 1.2 mm on average [63]. Therefore, we con-sidered it to be sufficiently accurate when tracking handsin this study. Nevertheless, the following two steps wereconsidered to ensure optimal tracking conditions for theleap motion. First, we placed an antireflective cloth on thephysical table to limit infrared interference (Figure 5). Inaddition, we asked the participants to hold the controllerwith their left hands and place the hand on their laps duringthe tasks. This instruction was intended to prevent detectionartifacts by keeping their left hand away from the field ofview of the leap motion. In Figure 5, the left hand is placedon the table only for clarity.

We used two types of avatar appearances, namely real-istic and abstract (Figure 1). For the realistic condition, weused a realistic gender-neutral virtual hand. The forearmwas not included in the model, as in previous retargetingstudies (e.g., [3], [4], [8], [32], [64], [65]). A 3D model of therealistic hand was obtained from the Leap Motion SoftwareDevelopment Kit (SDK). For the abstract condition, we chosea sphere that moved in correspondence with the tip positionof the participant’s index finger as a non-anthropomorphicavatar. One of the reasons for this choice was that propri-oceptive drift is commonly measured using the positionof the tip of an index finger (e.g., [21], [46]). Thus, it wasnecessary to use a model for which the position of the tip ofan index finger was as obvious as that of the realistic handmodel. In addition, task performance is known to influenceaction-effect integration in SoA [13], [66]. Considering Fitts’law [67], the difference in the cursor size (i.e., the avatarsize for pushing the button) could influence the task perfor-mance. Thus, for a fair comparison, we had to ensure thatboth avatars maintained the same functionality in pushingthe button despite their different appearances. Thus, wechose a spherical model, the diameter and end position ofwhich corresponded with the width and end position of theindex fingertip of the realistic hand model.

3.3.3 Experimental SceneThe experimental scene was designed to display a realisticroom composed of a wooden table and shelves. A virtualbutton with a diameter of 8 cm and a semitransparentvirtual sphere with a diameter of 2.5 cm represented thetarget and initial positions, respectively. Their spatial con-figurations are shown in Figure 5 (b). They floated 10 cm tothe right of the body mid-line so that the participants’ rightarm should be natural and comfortable to touch the initialpoint while keeping their elbow on the table. To personalizethe height of the initial point depending on the arm length,the participants wore the HMD and were asked to adjust

This is the author's version of an article that has been published in this journal. Changes were made to this version by the publisher prior to publication.The final version of record is available at http://dx.doi.org/10.1109/TVCG.2020.2964758

Copyright (c) 2020 IEEE. Personal use is permitted. For any other purposes, permission must be obtained from the IEEE by emailing pubs-permissions@ieee.org.

JOURNAL OF LATEX CLASS FILES, VOL. XX, NO. X, XXX XXXX 6

Fig. 6. Illustrations of a trial. The semi-transparent virtual hand represents the physical location of the participant’s hand (not displayed in theactual experimental scene). 1a) The participants placed the index fingertip of the hand inside the marker of the initial point. 1b) They kept thefingertip at this point for 1 s. 2a) They start a reaching movement with remapped visual feedback of the human hand (realistic) or spherical pointer(abstract). 2b) They press the target button. At that moment, the avatar’s displacement from the actual hand is the exact amount of shift, whichchanges for each trial. 2c) They returned the fingertip to the initial point with remapped visual feedback only for the remapping detection task. 3a)The participants answered whether the visual movements corresponded to their own movements. 3b) The position of the board was adjusted tocorrespond with the perceived location of their invisible index finger.

the height of the initial point by pushing the buttons of thecontroller to reach a comfortable posture when touchingthe initial point with the tip of their index finger at thebeginning of the experiment. The height of the button wasalso changed according to the adjusted height of the startingpoint, to keep the relative positions identical. The above-mentioned values and procedures were determined throughthe informal pilot study to retain the participants’ comfortand eliminate the influence of fatigue.

3.4 Remapping Detection Task

This task was aimed at estimating the detection thresholds(i.e., the lowest levels of discrepancy that can be detected)of the movement remapping. To this end, we used a two-alternative forced choice (2AFC) task with an adaptivestaircase method [68]. The task followed a 2 × 2 factorial de-sign (within-subjects), and the independent variables wereavatar appearance (realistic and abstract) and shift direction(left and right). Further, the dependent variable was thedetection threshold.

The participants first executed a reaching movementwith their right hand in each trial. To start each trial, theparticipants were asked to place their right index fingertipat an initial position (Figure 6, 1). Before starting the trial,they could view the same sphere model as in the abstractcondition without any movement remapping. When thetrial started, the program proceeded to beep every 1 s. Thiswas introduced to ensure that the movement speed wasalmost constant among trials and to avoid the influenceof speed on the task. Participants were asked to maintaina still posture at the initial position for 1 s until theyheard the next beep. Then, the mark at the initial positiondisappeared, and a button appeared at the target position.Simultaneously, either the realistic or abstract avatar, which

moved according to the remapping model, was displayed(Figure 6, 2a). The shift of the model changed for everytrial. The participants reached the button in 1 s such thatthey could push it with the virtual tip of the index fingerimmediately upon hearing the beep (Figure 6, 2b). Whenthey successfully pushed the button to a depth of 2 cm, thebutton disappeared.

In the remapping detection task, the participants in-stantly retracted their hand in another second so that theycould move back to the initial position when they heard thenext beep (Figure 6, 2c). During the retraction, the self-avatarwas still visible, and its movement followed the retargetingmodel. The self-avatar disappeared when the participantssuccessfully touched the marker at the initial point. Then,they answered a 2AFC task (yes/no) (Figure 6, 3a). Thedisplayed question was a Japanese-translation of ”Did themovement of the virtual hand (sphere) exactly correspondto your own movement?”. They could select the answerby pressing the left or right button on the controller heldin their left hand, and they confirmed it by pressing thecenter button. They proceeded with the trials consecutively,although they could freely rest during the interval of trialsif they needed.

3.4.1 Staircase ProcedureIn the remapping detection task, the shift amount was var-ied between trials according to a simple up–down staircaseprocedure (Figure 7). When the participant was aware thatthe movement was remapped (i.e., a ”no” response) at atrial, the staircase reduced the shift amount by 1.5 cm forthe next trial. Likewise, in the case of response ”yes,”, theprocedure raised the shift amount by 1.5 cm. This step-ping rule converged to the 50% point of the psychometricfunction, which corresponds to the detection threshold atwhich remapping can barely be detected. As typically used

This is the author's version of an article that has been published in this journal. Changes were made to this version by the publisher prior to publication.The final version of record is available at http://dx.doi.org/10.1109/TVCG.2020.2964758

Copyright (c) 2020 IEEE. Personal use is permitted. For any other purposes, permission must be obtained from the IEEE by emailing pubs-permissions@ieee.org.

JOURNAL OF LATEX CLASS FILES, VOL. XX, NO. X, XXX XXXX 7

Fig. 7. (a) The staircases increase or decrease the shift amount by 1.5cm for every trial according to the answer. (b) An example panel rep-resents the trial histories for two interleaved staircases that convergedfrom above and below the expected threshold region (descending andascending staircases, respectively). The dashed lines indicate the meanof staircase reversals (represented by square markers), which approx-imates the detection threshold (blue: ascending, red: descending, andblack: mean of both staircases).

in the staircase method, to prevent participants from beingaware of the scheme and to exclude any possible biasesresulting from this awareness, two staircases were inter-leaved for each condition. The descending and ascendingstaircases began sufficiently above (10 cm) and below (0cm) the expected threshold region, respectively, the valuesof which were estimated during the informal pilot study.In any trial, the shift amount cannot take a negative value(i.e., remapping to the opposite direction of the condition).For example, in case the participant’s response was ”no”when the shift amount was 0 cm (i.e., the movement wasnot remapped), the shift amount did not change to -1.5 cmbut remained 0 cm for the next trial. Similarly, when theirresponse was ’no’ to the 1-cm-shift trial in the descendingstaircase, the shift changed to 0 cm rather than −0.5 cm forthe next trial. In this case, after they answered ”yes” to the0-cm-shift trial, the shift was increased to 1 cm and not 1.5cm.

Eight unique types of staircases (two levels of avatarappearance × two shift directions × two initial shifts)were alternately performed in a randomized order until14 trials were completed for all the staircases. That is, theparticipants proceeded through eight staircases in parallel;they first completed the first trials from all eight staircasesin a randomized order and then moved to the next eighttrials in another randomized order. This design is aimed atpreventing the influence of adaptation. Each participant per-formed a total of 112 trials. Note that when the participantcompleted the 14th trial, the staircase could determine theshift amount for the 15th trial. The average of the reversalpoints (i.e., the point at which the response changes from”yes” to ”no” or vice versa) is typically used to estimate thedetection threshold.

3.5 Self-Localization TaskThis task was aimed at assessing proprioceptive drift, thatis, the degree to which the perceived hand location driftedtoward the self-avatar. The proprioceptive drift was mea-sured through the visual estimations of the perceived fingerposition, as in previous studies (e.g., [57], [69]–[71]). Weused an adjustment method; the participants continuouslychanged the position of the virtual thin board until itsposition was perceived as the same as their finger position.To eliminate the influence of anchoring effects, we used two

directions of adjustment: the board appeared at either 25 cmto the left or right from the center of the virtual button foreach trial. We assumed that the positions were sufficientlyfar from the participants’ perceived hand location.

The task followed a 2 × 5 factorial design(within-subjects). The independent variables were theavatar appearance (realistic and abstract) and shift(−10cm,−5cm, 0cm, 5cm, and 10cm). Moreover, the neg-ative and positive values indicate leftward and rightwardshifts, respectively. Based on our informal pilot study, weanticipated that the thresholds with respect to the shiftamount in the remapping detection task would be approxi-mately 5 cm on average and seldom exceed 10 cm. Hence, bysetting the shift factor to ±5 and ±10, we expected that wecould collect data of proprioceptive drifts for both cases inwhich the participants easily noticed the discrepancy (±10cm) and experienced difficulty in realizing the discrepancy(±5 cm). The dependent variable was the proprioceptivedrift, which was measured by recording the horizontalposition of the virtual board from the actual finger position.

In each trial, the participants first executed a reachingmovement as in remapping detection task. The differencewas that they kept their arm still after completing the one-way reaching movement. When they pushed the target but-ton (Figure 6, 2b), the button and self-avatar disappeared.Instead, a virtual thin board appeared (Figure 6, 3b). Then,the participants adjusted the horizontal position of theboard to match the position with the perceived location oftheir own index finger. They could continuously move theboard with a precision of 1 mm by pressing the left andright buttons on the controller held in their left hand. Whenthey were satisfied, they pressed the center button to finalizethe answer. While estimating their perceived index fingerposition, they were not allowed to move their arm. Then,they retracted their arm to the initial position for the nexttrial. During the retraction, the self-avatar was still invisiblebecause the visible position would provide feedback forthe correct answer. When the tip of the actual index fingerapproached the starting position within a 5-cm radius fromthe center, the sphere model was displayed at the tip of theindex finger.

Twenty unique types of trials (two levels of avatar ap-pearances × five levels of shift amounts × two adjustmentdirections) were successively performed in a randomized or-der. These 20 unique trials constituted a block, and the taskconstituted of two blocks. Thus, each participant performed40 trials.

3.6 Heartbeat Counting Test

We conducted this test to assess individual interoceptivesensitivity, which reportedly predicts the susceptibility toBOIs. The susceptibility to BOI has two aspects in thisexperiment: how much the threshold is increased, i.e., de-crease in the sensitivity to the visuomotor discrepancies,irrespective of avatar appearance, and how susceptible oneis to the avatar appearance, that is, how one’s responsein each measurement differed between the levels of avatarappearance. Hence, we expected that interoceptive sensitiv-ity may account for one’s sensitivity to the remapping orsusceptibility to avatar appearance.

This is the author's version of an article that has been published in this journal. Changes were made to this version by the publisher prior to publication.The final version of record is available at http://dx.doi.org/10.1109/TVCG.2020.2964758

Copyright (c) 2020 IEEE. Personal use is permitted. For any other purposes, permission must be obtained from the IEEE by emailing pubs-permissions@ieee.org.

JOURNAL OF LATEX CLASS FILES, VOL. XX, NO. X, XXX XXXX 8

We used the mental tracking method [72], which hasbeen widely used to assess interoceptive sensitivity (e.g.,[60], [62]). The test was conducted without using the HMD.The participants were asked to count their heartbeat as fol-lows: ”Without manually checking, e.g., touching your heartor vein, silently count your heartbeat from the time youhear ’start’ until you hear ’stop’. At the end of each interval,please verbally report the number of counted heartbeats.”This process was repeated for three time intervals of 25, 35,and 45 s, presented in a random order. The participant wasinformed of the length of the time interval. No feedbackregarding their performance was given to the participant.After one brief training session (15 s), the actual experimentstarted. There were 20-s breaks after every interval. Duringthe task, the actual heart rate was monitored with a pulseoximeter attached to the participant’s left index finger.

The interoceptive sensitivity was estimated by calculat-ing the normalized difference between their estimated andactual heart rates. Each participant’s interoceptive sensitiv-ity score was calculated as the mean score of three trialsaccording to the following transformation [72], [73]:

1

3

3∑i=0

(1− |recorded− counted|recorded

) (1)

The score could vary between 0 and 1, with higher scoresindicating small differences between recorded and countedheartbeats (i.e., higher interoceptive sensitivity).

3.7 QuestionnaireThe subjective evaluation of embodiment for virtual avatarswas assessed through a questionnaire. Although we wereinterested in SoBO in particular, we measured the senseof embodiment [52], which consists of SoA, SoBO, andthe sense of self-location (i.e., one’s spatial experience ofbeing inside a body), as they are closely related to eachother. We used Gonzalez–Franco and Peck’s VR-specificavatar embodiment questionnaire [74], supplemented by aJapanese translation. We omitted items that were not appli-cable to our study context, such as the question regardingtactile sensation. Consequently, the questionnaire consistedof nine items and three subsets of questions: body owner-ship (Ownership), agency and motor control (Agency), andbody location (Location) (Table 1). Each response was scoredon a seven-point Likert scale (−3 = strongly disagree; +3= strongly agree). The evaluation was performed for eachlevel of avatar appearance.

The questionnaire was performed within the HMD. Dur-ing the questionnaire, the participants could see the avatarof their right hands. The avatar appearance was eitherrealistic or abstract, and the avatar moved in exact corre-spondence with their real movements. They could freelymove their right hand. A question from the questionnaireand a slider with numbers from −3 to +3 were displayed.The questionnaire was simultaneously shown in English(original) and in Japanese (translated). The participants wereinstructed to answer each question based on the impressionof the avatar they were observing (i.e., the movement-correspondent avatars) but not based on the recall of themain tasks. Each question was answered twice for the twotypes of avatar alternately in a randomized order. They

TABLE 1Questionnaire to measure the sense of embodiment based on [74].

Items in italics represent control questions.

Subscale QuestionOwnership 1) I felt as if the virtual hand (sphere) was my

hand.2) It felt as if the virtual hand (sphere) I saw wassomeone else’s.3) It seemed as if I might have more than one hand.

Agency 4) It felt like I could control the virtual hand(sphere) as if it was my own hand.5) The movements of the virtual hand (sphere)were caused by my movements.6) I felt as if the movements of the vir-tual hand (sphere) were influencing my ownmovements.7) I felt as if the virtual hand (sphere) was movingby itself.

Location 8) I felt as if my hand (fingertip) was locatedwhere I saw the virtual hand (sphere).9) I felt out of my body.

could choose the answer by pressing the buttons on thecontroller held in their left hand. Once they answered thequestion regarding the observed virtual avatar, the appear-ance changed to the other, while the question remained thesame. After they answered the same question for two typesof avatar appearances, the next question was shown, whichwas randomly chosen from the nine items.

3.8 Procedure

The basic procedure of the experiment was (a) remap-ping detection task, (b) heartbeat counting test, (c) self-localization task, and (d) questionnaire. The whole exper-iment took approximately 1 h to complete. The overall flowof each experiment is described as follows. The steps initalics were conducted while the participants’ wearing theHMD. The participants

1) read and signed an experimental consent form.2) received instructions and an overview of the task.3) completed the training and the main trials of remapping

detection task (10–15 min).4) completed the heartbeat counting test (3 min).5) completed the training and main trials of self-localization

task (10–15 min).6) completed the embodiment questionnaire (5–10 min).7) took off the HMD and filled out a demographic ques-

tionnaire.

During training of remapping detection task, the partic-ipants completed a minimum of eight trials with the samestimulus type as the first trial for each unique staircase. Foreach training trial, the expected answer was given to theparticipant as feedback. Through the course of training, allthe participants were able to correctly distinguish betweennonremapped (i.e., 0-cm shift) and remapped (i.e., 10-cmshift) movements. For self-localization task, the trainingtrials consisted of a few trials without feedback to practicethe procedure.

This is the author's version of an article that has been published in this journal. Changes were made to this version by the publisher prior to publication.The final version of record is available at http://dx.doi.org/10.1109/TVCG.2020.2964758

Copyright (c) 2020 IEEE. Personal use is permitted. For any other purposes, permission must be obtained from the IEEE by emailing pubs-permissions@ieee.org.

JOURNAL OF LATEX CLASS FILES, VOL. XX, NO. X, XXX XXXX 9

Fig. 8. Results of remapping detection task. (Left) Bar plot of detectionthreshold of remapping determined by shift amount considering avatarappearance and shift direction. (Right) Bar plot of the rate of change ofthe threshold from abstract to realistic considering shift direction. Theerror bars indicate 95% confidence intervals. *: p < .05, ****: p < .0001.

4 RESULTS

For continuous variables, analysis of variance (ANOVA)was conducted when the normality assumption (Shapiro–Wilk’s normality test) was not violated (p > .05). Whenthe sphericity assumption was violated (Mauchly’s spheric-ity test), the degrees of freedom were corrected using theGreenhouse–Geisser correction. In addition, η2p were pro-vided to quantitatively compare the effect size. Finally,Tukey’s post-hoc tests (α = .05) were conducted to checkthe significance of pairwise comparisons of the parametricdata. When the normality assumption was violated, or themeasurement did not use continuous variables, a Wilcoxonsigned rank test was conducted.

4.1 Detection Threshold of RemappingTo test [H1], i.e., the threshold for detecting if the remappingof movements is higher with the realistic avatar than withthe abstract avatar, we analyzed the data from remappingdetection task. As described in Subsection 3.4 and Figure 7,the average of the reversal points, including the data pointfrom the 15th trial, was used to estimate the threshold foreach staircase. We excluded three data points from reversalsas they were possibly mistakes and outliers; two partici-pants answered “no” in the first trial of one of the ascendingseries (i.e., a 0-cm shift) and one participant answered “yes”for a descending series (i.e., a 10-cm shift). The averagevalue of the thresholds of the ascending and descendingseries for each condition was used as a representative valuefrom each participant because they were expected to con-verge to the same values. As described in Section 3.2, thedata from one male participant was excluded from the anal-ysis because his thresholds for two out of four conditionsexceeded 10 cm. Considering he understood the instructionduring the training session that the expected response to 10-cm shift was ”no” (see Subsection 3.8), it is rather unlikelythat he actually could not notice the remapping with a 10-cm shift. Rather, it is reasonable to suppose that he didnot properly complete the task. Thus, we used a total of16 participant datasets for the following analysis.

Finally, a two-way ANOVA with repeated measureswas conducted considering the within-subjects factors of

avatar appearance (two levels: realistic and abstract) andshift direction (two levels: left and right). As shown in theleft part of Figure 8, ANOVA showed a significant two-way interaction effect between avatar appearance and shiftdirection [F (1, 15)=17.39, p< .001, η2p =0.54]. Thus, weconducted Tukey’s post-hoc tests for each avatar appearancelevel and shift direction. When comparing the values foreach shift direction, the threshold was significantly smallerfor the abstract avatar than for the realistic avatar only forthe leftward shift (left: p < .0001, right: p = .67). In contrast,when comparing the value of each avatar appearance level,the threshold was significantly smaller in the leftward thanin the rightward shift only in the case of the abstract avatar(abstract: p < .05, realistic: p = .42).

These results indicate that the participants were less sen-sitive to remapping with realistic avatars than with abstractavatars when leftward shifts are applied. To reveal howmuch the realistic avatar increased the detection thresholdsof visuomotor discrepancy accompanied by leftward shifts,we calculated the threshold ratio of realistic to abstract foreach shift direction, for each participant (Figure 8 right).Note that the average of the ratios can be different from theratio of the averages. As a result, the threshold was foundto be 31.3±12.5% higher and 0.5±8.2% lower (95%CIs) forrealistic avatars than abstract avatars on average for the left-and right-shift conditions, respectively.

To summarize, these results partially supported [H1] inthe sense that the detection threshold of the remapping washigher when using realistic avatar than when using abstractavatar, only in the case of the leftward shifts.

4.2 Proprioceptive Drift

To test [H2], i.e., the amount of proprioceptive drift is largerfor the realistic avatar than for the abstract avatar, the resultsof self-localization task were analyzed. The value of thedrift indicates how far the proprioceptive self-location wasperceived toward visual self-location (i.e., where the self-avatar was located) from the actual hand position. Theaverage of the four repeated measurements (including bothadjustment starting points of −25 cm and 25 cm) for eachcondition was taken as the data point for each participant.

A two-way ANOVA with repeated measures was con-ducted considering the within-subjects factors of avatarappearance (two levels: realistic and abstract) and shift (fivelevels: −10, −5, 0, 5, and 10 cm) (Figure 9). Note that therightward and leftward shifts take a positive and nega-tive value, respectively. ANOVA revealed a significant two-way interaction effect between avatar appearance and shift[F (2.99, 47.84)=3.40, p< .05, η2p =0.18]. Thus, we con-ducted Tukey’s post-hoc tests for each avatar appearancelevel and shift (Figure 9). When comparing the value foreach shift, the proprioceptive drift was significantly smallerwith respect to the abstract avatar than that with respectto the realistic avatar in the −5- and −10-cm conditions(p < .05 and p < .01, respectively). For the realistic avatar,the proprioceptive drifts were not significantly differentbetween 5 and 10 cm (p = .96) or between −5 and −10cm (p = .84). The other comparisons produced significantdifferences (p < .05 for 0 cm vs 10 cm; p < .01 for theothers). For the abstract avatar, the proprioceptive drifts

This is the author's version of an article that has been published in this journal. Changes were made to this version by the publisher prior to publication.The final version of record is available at http://dx.doi.org/10.1109/TVCG.2020.2964758

Copyright (c) 2020 IEEE. Personal use is permitted. For any other purposes, permission must be obtained from the IEEE by emailing pubs-permissions@ieee.org.

JOURNAL OF LATEX CLASS FILES, VOL. XX, NO. X, XXX XXXX 10

Fig. 9. Point plot of the mean proprioceptive drift (i.e., the localizationbias of own hand toward the virtual hand) considering avatar appear-ance and shift during self-localization task. Error bars indicate 95% CIs.The shift takes a positive/negative value for rightward/leftward shift Thesolid/dotted black lines indicate the relative position of the actual/virtualhands from the actual hand, respectively, for readability. *: p < .05, **:p < .01.

Fig. 10. Box plots of the mean subjective ratings of ownership (left),agency (center), and location (right) for each avatar appearance leveland gender obtained through the questionnaires (from −3 to +3).

were not significantly different among 0, −5, and −10 cm(p = .27 for 0 cm vs −5 cm, p = .34 for 0 cm vs −10 cm,p = 1.00 for −5 cm vs −10 cm) or between 5 cm and 10cm (p = .60). Moreover, the other comparisons producedsignificant differences (p < .01 for all).

To summarize, these results partially support [H2] inthe sense that the extent of proprioceptive drift was largerwith respect to the realistic avatar than with respect to theabstract avatar, only for leftward shifts.

4.3 Sense of EmbodimentTo test if the realistic avatar induced a stronger sense ofembodiment, including SoBO, than the abstract avatar, thesubjective ratings of the questionnaire were analyzed. Thesubscales of ownership, agency, and location ratings in thequestionnaire were aggregated and averaged (answers forcontrol items were inverted) to compute the scores for eachavatar appearance per participant.

As the Likert scale is regarded as an ordinal scale,Wilcoxon signed-rank tests with the within-subjects factorof avatar appearance (two levels: Realistic, Abstract) wereperformed for each subscale. As shown in Figure 10, no sig-nificant difference was found between the scores of realistic

Fig. 11. Bar plots of the threshold of remapping (upper left) and theproprioceptive drift ratio (upper right), and box plots of the subjectiveratings of ownership (bottom) for each avatar appearance level andinteroceptive sensitivity. Error bars of the bar plots indicate 95% CIs.

and abstract avatars for Ownership (p = .20) or Agency(p = .72). The Location score with the abstract avatarwas marginally smaller than that with the realistic avatar(p = .07). Contrary to our expectations drawing on previousstudies that the realistic avatar induces the stronger SoBO(e.g., [15], [18]–[20]), there was no significant differencebetween the sense of embodiment with the realistic andabstract avatars.

4.4 Interoceptive SensitivityWe tested [H3] and [H4], i.e., the participants with lowerinteroceptive sensitivity tend to have higher thresholds(H3) or tend to be well influenced by avatar appearance(H4). In addition, we tested if the proprioceptive drift andquestionnaire scores of ownership subscale, which are bothmeasurements of BOI, can be explained according to intero-ceptive sensitivity to validate the previous study [60]. Pre-vious studies have suggested that interoceptive sensitivitypredicts the malleability of body representation; that is, peo-ple with low interoceptive sensitivity experience a strongerBOI [60]. Therefore, we investigated whether interoceptivesensitivity predicted the individual differences of detectionthreshold of remapping.

As a result of the heartbeat counting test, interoceptivesensitivity scores were derived from Equation 1 in 3.6. Thescores did not satisfy the normality assumption, and themedian value was 0.93. As with the previous studies oninteroceptive sensitivity [60]–[62] , a median split methodwas used to divide the participants into high- and low-interoceptive-sensitivity groups (median = 0.96 and 0.78,respectively).

For the thresholds, we used the average values in theleft- and right-shift conditions for each avatar appearanceas an index. We then calculated the indices of the thresh-old and proprioceptive drift for each avatar appearance,which can represent the values regardless of shift. For

This is the author's version of an article that has been published in this journal. Changes were made to this version by the publisher prior to publication.The final version of record is available at http://dx.doi.org/10.1109/TVCG.2020.2964758

Copyright (c) 2020 IEEE. Personal use is permitted. For any other purposes, permission must be obtained from the IEEE by emailing pubs-permissions@ieee.org.

JOURNAL OF LATEX CLASS FILES, VOL. XX, NO. X, XXX XXXX 11

the proprioceptive drift, we first calculated the normalizedratio of the proprioceptive drift to the shift (i.e., the ratioof the subjective displacement of the self-location to theactual displacement) for each condition (avatar appearance× shift). This ratio can vary from 0 to 1, except for thecase of a 0-cm shift. The drift was not meant to occur withthis shift because the virtual and actual hands were presentat the exact same position. Nevertheless, we observed theproprioceptive drift with a 0-cm shift (p < .05 for bothavatar appearances; see Discussion). Hence, we eliminatedthe bias for each participant by subtracting the value in the0-cm shift as a baseline from the values in each shift foreach avatar appearance. We used these adjusted values tocalculate the ratio for each condition. Finally, we used theaverage values of the ratio in all shift conditions, except forthe 0-cm shift, for each avatar appearance as an index.

As we were interested in whether the values irrespectiveof avatar appearance differed according to the interoceptivesensitivity (i.e., the main effect of interoceptive sensitivity)and whether the effect of avatar appearance differed accord-ing to the interoceptive sensitivity (i.e., the interaction effect)for each index, we conducted a two-way ANOVA withrepeated measures considering the within-subjects factor ofavatar appearance (two levels: realistic and abstract) and thebetween-subjects factor of the interoceptive sensitivity (twolevels: high and low) for each index except for the subjectiveSoBO. For the subjective SoBO, we conducted the Wilcoxonsigned-rank test instead of the ANOVA as the data wasnonparametric. We did not report the main effect of avatarappearance for all the indices as it had been already reportedin the analyses in the previous sections.

For the thresholds, ANOVA revealed that neither themain effect of interoceptive sensitivity nor the interactioneffect were significant [F (1, 14)=1.20, p= .29, η2p =0.08;F (1, 14)=1.67, p= .21, η2p =0.11]. For the propriocep-tive drift ratio, the ANOVA also revealed that neitherthe main effect of interoceptive sensitivity nor the in-teraction effect were significant [F (1, 14)=0.01, p= .91,η2p =0.0008; F (1, 14)=0.27, p= .61, η2p =0.002]. For thesubjective SoBO, the Wilcoxon signed-rank test showed thatthe SoBO score tended to be lower in the high-interoceptivesensitivity group than in the low-interoceptive sensitivitygroup (W = 83, p = .09), although it was not significant.The other comparisons equivalent to the test of the interac-tion effect were not significant for the subjective SoBO.

To summarize, we did not find any relationship betweenone’s interoceptive sensitivity and detection thresholds orthe proprioceptive drift. Instead, we marginally validatedthe previous studies, which showed that the participants oflower interoceptive sensitivity tended to score higher SoBOratings. These results did not support [H3] and [H4].

4.5 Summary of Findings

The main findings of our experiment are as follows:• The detection threshold of the movement remapping

was approximately 31.3% higher with realistic (i.e.,virtual hands) than abstract (i.e., spherical pointers)avatars when the leftward shift was applied.

• The proprioceptive drift (i.e., displacement of the per-ceived self-location toward the virtual hand) was larger

with realistic than abstract avatars when the leftwardshift was applied.

• For both measurements, there were no significant dif-ferences between the two avatars when the rightwardshift was applied.

• The sense of embodiment including SoBO did not differbetween 1:1 mapped realistic and abstract avatars.

• The relationship of the interoceptive sensitivity with theindividual differences of the threshold and propriocep-tive drift was not found.

5 DISCUSSION

We revealed, for the first time, that avatar appearance in-fluences the self-attribution of the remapped movements inhand interaction techniques that exploit visual dominance.Specifically, the result of remapping detection task showedthat compared to abstract avatars, realistic avatars can makeremapping less noticeable. With the realistic avatar, the par-ticipants could not notice that the movement of the virtualhand was different from that of their own for a leftward shiftof up to 4.5 cm (i.e., 12.7◦ redirection), which is 31.3% higherthan that with abstract avatar. In addition, the result of self-localization task showed that a greater importance is givento visual information for visuo-proprioceptive integrationwhen realistic avatars are used than when abstract avatarsare used. The finding that the same results were shown withtwo complementary measurements not only confirms thereliability of our findings but also reinforces our argumentthat the BOI is closely related to the self-attribution of theremapped movement. This is remarkable because in remap-ping detection task, we asked the participants about the self-attribution of movements, which is related to SoA and notSoBO. Conventionally, the limitation of the self-attributionof a remapped movement is considered to depend on themismatch between actual control and visual feedback (i.e.,visuo-motor or visuo-proprioceptive discrepancy). To thebest of our knowledge, this is the first study which showedthat avatar appearance changes how one perceives the iden-tical discrepancy between visual and real hand movements.

Our contributions, with respect to the limitation of unno-ticeable offset of the remapped movement exploiting visualdominance, can be summarized as follows:

1) We expect that the avatar appearance can influence thesense of self-attribution by associating with BOI

2) We provide quantified evidence that realistic avatarscan make the remapping less noticeable for larger mis-matches between visual and actual movements.

Given these findings, we recommend that VR designers userealistic rather than abstract avatars to widen the range ofunnoticeable displacement for the hand remapping tech-niques. The use of realistic rather than abstract avatarscan potentially improve a wide variety of hand interactiontechniques because it can mitigate a range of unnoticeabledisplacement without changing the mapping model itself.

Nevertheless, there are some unexpected results thatmust be discussed. First, with both the detection thresholdand proprioceptive drift, the effect of avatar appearance wasobserved only when the leftward shift was applied, i.e.,his/her own right hand moves to the right and forward. Thefollowing questions were raised. 1. “What is the difference

This is the author's version of an article that has been published in this journal. Changes were made to this version by the publisher prior to publication.The final version of record is available at http://dx.doi.org/10.1109/TVCG.2020.2964758

Copyright (c) 2020 IEEE. Personal use is permitted. For any other purposes, permission must be obtained from the IEEE by emailing pubs-permissions@ieee.org.

JOURNAL OF LATEX CLASS FILES, VOL. XX, NO. X, XXX XXXX 12

between the rightward and leftward shifts?” and 2. “Why isthe avatar appearance only effective in the leftward shift?”.The first question may be hinted at by the notable anisotropyobserved in self-localization task. As shown in Figure 9,the proprioception was almost fully captured by visionin the 5 cm-shift condition (i.e., rightward shift) for bothavatars, and the location was scarcely captured with the−10- and −5-cm shifts (i.e., leftward shift), especially forabstract avatars. A possible explanation for the first questioncan be the difference in the muscle execution: flexion andextension. As the target was placed 10 cm to the right ofthe participant’s body midline (i.e., where the right handis naturally located), the arm should be somewhat flexedto touch it. The actual arm was also flexed with rightwardshift. However, it was extended in the case of leftward shift.It can be considered that it is difficult to deceive that the armwas flexed when the actual arm was extended. In addition,when we extend the arm to the right, maintaining theposture was more difficult than when reaching for the frontof the body. Consequently, it could foster the awarenessof proprioception and might result in higher reliability ofproprioception in the case of the leftward shift.

Furthermore, similar anisotropy is shown in BOI; therelative distance of the fake hands to the body midlineas well as distance between the real and artificial handsmodulate the proprioceptive drift [75]. In our cases, therelative distances of the actual hand from the body midlinelargely differed when the leftward and rightward shiftswere applied. The distance from body midline influencesBOI because the peripersonal space, a representation ofthe space immediately surrounding the body, plays a keyrole in BOI and visual-proprioceptive integration [40], [75]–[77]. Outside this space (i.e., extra-personal space), multisen-sory integration is considered to be diminished. Althoughperipersonal space is considered to be determined by thereaching extent, evidence has been found for a graduatedtransition between the two spaces, and this transition beginswithin the reaching space [78]. The theory of peripersonalspace explains the first point and helps in the considerationof the second point. Our results indicate that the avatarappearance has an effect on the visuo-proprioceptive inte-gration only when integration scarcely occurs. We conjecturethat if the situation already allows for integration (i.e.,within peripersonal spaces), the impact of the top–downprocess is relatively low. Rather, the avatar appearance canfoster the integration when the integration hardly occurs.Nevertheless, the above-mentioned studies mostly inves-tigated the effect of the position of artificial hands ratherthan that of the actual hands. In addition, the actual hand isplaced away from body midline in a typical RHI (e.g., [56]).Hence, further investigation would be needed to validate ifthe peripersonal space could well explain the anisotropy.

Second, in self-localization task, we observed the pro-prioceptive drift with the 0-cm-shift, where the drift wasnot supposed to occur because the virtual and actual handswere present at the exact same positions (Figure 9). Thisbias is considered to be a result of proprioceptive recali-bration caused by the unexpectedly fast, strong visuomotoradaptation (e.g., [79]) that persisted in the subsequent trial.In particular, the estimated drift to the right in the 0-cm-shift condition might have been influenced by the large

proprioceptive drift under the preceding right-shift condi-tion. Another possible explanation for the bias is an artifactof the absolute error in the estimation of hand positionsby Leap Motion. Optical imperfections such as perspectivedistortion could produce the constant horizontal offsetsbetween actual and estimated hand positions.

Third, contrary to the expectation, the results of thequestionnaire indicated that avatar appearance does notinfluence SoBO. The visuomotor synchrony from the first-person perspective enabled by VR was considered to func-tion as a powerful tool for embodiment and resulted in theceiling effect, regardless of avatar appearance. Nevertheless,the score of SoBO over the spherical pointer in our studywas rather high even compared to that of previous studieson SoBO over abstract avatars (e.g., block, board and sphere)in VR [18]–[20], [53]. This may be because of our experimen-tal protocol. In our experiment, participants evaluated thesense of embodiment over avatars whose movements werecompletely synchronized with their own after they usedthe avatars whose movements were remapped in the maintasks. Hence, it is possible that in the questionnaire, SoBOwas estimated to be high as completely synchronous visuo-motor information was provided compared to the previousavatar. In addition, the proprioceptive drift and subjectivescores of SoBO were shown to not always correspond [54]–[58]. Therefore, we considered that the unexpected resultsof the questionnaire do not immediately undermine thevalue of our other results. Furthermore, the sample sizeof this study was chosen based on the power analysis forthe detection threshold. Hence, the supports for the nullhypothesis of the questionnaire results might be attributableto the relatively low statistical power for non-parametrictests, which increased the probability of committing a typeII error. Thus, further investigations are needed in terms ofthe mechanism of the effect of avatar appearance.

Lastly, we did not find the relationship between intero-ceptive sensitivity and threshold of remapping, contrary tothe expectation (Figure 11). In addition, we could not vali-date the previous finding that the susceptibility of BOI, mea-sured by proprioceptive drift and questionnaire, is predictedby interoceptive sensitivity. One of the reasons for this isthat the interoceptive sensitivity scores indicated the ceilingeffect and did not sufficiently vary between participants. Infact, the median was 0.93. Some participants reported inopen-ended questions after the experiment that the pulsebecame apparent owing to the finger being pressured inthe pulse oximeter. Hence, finding an alternative methodto precisely detect the participant’s actual heartbeat wouldbe needed for future studies. If the individual differencesare predicted through interoceptive sensitivity, the levelof remapping to each user can be adjusted based on theindividual traits of how sensitive they are to the sensoryconflict without directly measuring it. As a method of pre-dicting one’s tendency of visual dominance, it may also beapplicable to a wide variety of interaction techniques.

5.1 Limitations

Herein, we adopted two types of avatar appearances toverify the hypothesis that the more realistic the avatar,the more visual dominance can be exploited. Our results

This is the author's version of an article that has been published in this journal. Changes were made to this version by the publisher prior to publication.The final version of record is available at http://dx.doi.org/10.1109/TVCG.2020.2964758

Copyright (c) 2020 IEEE. Personal use is permitted. For any other purposes, permission must be obtained from the IEEE by emailing pubs-permissions@ieee.org.

JOURNAL OF LATEX CLASS FILES, VOL. XX, NO. X, XXX XXXX 13

supported the hypothesis for the leftward shift at least in ourexperimental condition and showed for the first time thatthe appearance of virtual hands matters in hand remappingtechnique; nevertheless, it remains unclear how far thisholds true for different avatar appearances. Hence, a furtherinvestigation from at least two aspects may be necessaryin future studies. The first being the consideration of howthe self-attribution would be affected if the avatar is ofmedium level of realism, such as robotic hand. Schwind etal. [20] used five different appearances for virtual hands toinvestigate the effect on visuo-haptic integration and foundthat massive violations of the human-likeness of a virtualhand (i.e., robot, invisible, or cartoon hand) can affect tactileexperiences, but do not seem to affect tactile sensitivitysystematically. This indicate a nonlinear effect of virtualhand appearances; hence, further investigation would beneeded. The other is whether a more realistic virtual hand(e.g., personalized hand in terms of size and texture anda realistic full-body including arm) fosters self-attributionmore than a realistic virtual hand without an arm, whichwe used in this study. It is known that a stronger SoBOcan be elicited with a virtual upper limb that appears tobe connected with one’s torso than a virtual hand withoutan arm [54]. However, when remapping the movements ofthe virtual upper limb, the arm may sometimes appear tobe obviously detached from the user’s torso or protrudeunnaturally from the torso, which breaks the connectivityto the body. Indeed, Azmandian et al. [2] considered itto be an issue and proposed a body-friendly adjustmentmethod, wherein the virtual hand rotates and shifts to avoiddetaching the arm from the torso. Thus, when using a full-body avatar, a technique to maintain the connectivity as wellas to investigate the effect of SoBO must be realized.

5.2 Future Work

Herein, we focused on the aspect of SoBO with respect tothe self-attribution to identify if the avatar appearance couldaffect the threshold of the remapping. Similarly, focusing onSoA, which is another aspect of the self-attribution, mayalso contribute to identifying other factors that could affectthe threshold. In fact, higher-level cognitive processes, suchas background beliefs and contextual knowledge relatingto the action, also influence the induction of SoA [80],[81], although spatial and temporal contiguity betweenone’s own and observed movements are the main cuesfor SoA [36]–[38]. Thus, controlling the context may helpincrease the threshold without changing the mapping. Forexample, priming, a method that is often used to modulatethe SoA by manipulating prior conscious thought about anoutcome [82]–[84], would be a promising technique to fosterself-attribution. Moreover, task performance is known toinfluence action-effect integration in SoA [13], [66]. In thisstudy, we used the avatars with the same functionality orcontrollability in pushing the button so as to investigate theeffect of avatar appearance fairly. Conversely, the change inavatar representation including the functionality or control-lability as well as its appearance may influence both SoAand SoBO; hence, it may further foster self-attribution.

Furthermore, evidence from several studies indicatesthe effectiveness of haptic feedback. For instance, Caola

et al. [44] found that synchronous visuo-tactile stimulationcounteracted the visuomotor inconsistencies (i.e., virtualarms moved while real arms kept still) to induce SoBO. Leeet al. [85] also showed that finger-based cutaneous hapticfeedback increased the detection threshold of the visuo-proprioceptive conflict arising from tracking error. To applythese findings to hand remapping technique in terms ofSoBO would be an interesting future prospect.

This study dealt with a basic retargeting model thatlinearly remaps a hand movement according to a fixedangle as an example of a hand remapping technique ex-ploiting visual dominance. Nevertheless, considering thetheory of SoBO, the avatar appearance may influence theself-attribution in more complex remapping techniques aswell. Moreover, it may be applicable to visuo-proprioceptivediscrepancies, which is unintentionally caused by trackingerrors and latency. Finally, the investigation of the effective-ness of avatar appearance in combinations of multimodalintegration other than visuo-proprioceptive, such as visuo-vestibular (i.e., used in redirected walking techniques), is apotentially fruitful area of future work.

6 CONCLUSION

Taking advantage of visual dominance, previous researchhas developed a variety of hand interaction techniqueswhich remap physical hand movements onto different vir-tual movements. However, there is a problem that whenthe offset between virtual and physical hands increases,the users become aware of the remapping. We hypothe-sized that realistic avatars (i.e., human hands) can makethe remapping less noticeable than abstract avatars (i.e.,spherical pointers) by fostering the SoBO over the remappedhands. The results showed that realistic avatars increasedthe detection threshold (i.e., lowered sensitivity) by 31.3%than the abstract avatars when the leftward shift was ap-plied (i.e., when the hand moved in the direction awayfrom the body-midline), supporting the hypothesis. In ad-dition, we also measured the proprioceptive drift (i.e., thedisplacement of self-localization toward an avatar). It waslarger, indicating a greater weight being given to visualinformation, when realistic avatars are used than abstractavatars are used for leftward shifts. Our findings showthat the more visual dominance over proprioception can beexploited by simply manipulating avatar appearance.

ACKNOWLEDGMENTS

REFERENCES

[1] I. Rock and J. Victor, “Vision and touch: An experimentally createdconflict between the two senses,” Science, vol. 143, no. 3606, pp.594–596, 1964.

[2] M. Azmandian, M. Hancock, H. Benko, E. Ofek, and A. D. Wilson,“Haptic Retargeting: Dynamic Repurposing of Passive Hapticsfor Enhanced Virtual Reality Experiences,” in Proceedings of theSIGCHI Conference on Human Factors in Computing Systems (CHI’16)., 2016, pp. 1968–1979.

[3] L.-P. Cheng, E. Ofek, C. Holz, H. Benko, and A. Wilson, “SparseHaptic Proxy: Touch Feedback in Virtual Environments Using aGeneral Passive Prop,” in Proceedings of the SIGCHI Conference onHuman Factors in Computing Systems (CHI ’17), 2017, pp. 3718–3728.

This is the author's version of an article that has been published in this journal. Changes were made to this version by the publisher prior to publication.The final version of record is available at http://dx.doi.org/10.1109/TVCG.2020.2964758

Copyright (c) 2020 IEEE. Personal use is permitted. For any other purposes, permission must be obtained from the IEEE by emailing pubs-permissions@ieee.org.

JOURNAL OF LATEX CLASS FILES, VOL. XX, NO. X, XXX XXXX 14

[4] Y. Zhao and S. Follmer, “A Functional Optimization Based Ap-proach for Continuous 3D Retargeted Touch of Arbitrary, ComplexBoundaries in Haptic Virtual Reality,” in Proceedings of the SIGCHIConference on Human Factors in Computing Systems (CHI ’18), 2018,pp. 1–12.

[5] B. J. Matthews, B. H. Thomas, S. Von Itzstein, and R. T. Smith,“Remapped Physical-Virtual Interfaces with Bimanual Haptic Re-targeting,” in Proceedings of the IEEE Conference on Virtual Reality(IEEE VR ’19), 2019, pp. 19–27.

[6] R. A. Montano Murillo, S. Subramanian, and D. Martinez Plasen-cia, “Erg-O: Ergonomic Optimization of Immersive Virtual En-vironments,” in Proceedings of ACM Symposium on User InterfaceSoftware and Technology, 2017, pp. 759–771.

[7] T. Feuchtner and J. Muller, “Ownershift: Facilitating OverheadInteraction in Virtual Reality with an Ownership-Preserving HandSpace Shift.” in Proceedings of ACM Symposium on User InterfaceSoftware and Technology, 2018, pp. 31–43.

[8] P. Abtahi and S. Follmer, “Visuo-Haptic Illusions for Improvingthe Perceived Performance of Shape Displays,” in Proceedings ofCHI Conference on Human Factors in Computing Systems, 2018, pp.1–13.

[9] Y. Ban, T. Kajinami, T. Narumi, T. Tanikawa, and M. Hirose,“Modifying an identified curved surface shape using pseudo-haptic effect,” in Proceedings of Haptics Symposium. IEEE, 2012,pp. 211–216.

[10] L. Kohli, M. C. Whitton, and F. P. Brooks, “Redirected touching:The effect of warping space on task performance,” in Proceedingsof IEEE Symposium on 3D User Interfaces, 2012, pp. 105–112.

[11] A. Lecuyer, “Simulating haptic feedback using vision: A surveyof research and applications of pseudo-haptic feedback,” Presence:Teleoperators and Virtual Environments, vol. 18, no. 1, pp. 39–53,2009.

[12] S. Frees, G. D. Kessler, and E. Kay, “PRISM interaction for enhanc-ing control in immersive virtual environments,” ACM Transactionson Computer-Human Interaction, vol. 14, no. 1, pp. 2–es, 2007.

[13] H. Galvan Debarba, R. Boulic, R. Salomon, O. Blanke, and B. Her-belin, “Self-attribution of distorted reaching movements in immer-sive virtual reality,” Computers & Graphics, vol. 76, pp. 142–152,2018.

[14] K. Kilteni, A. Maselli, K. P. Kording, and M. Slater, “Over myfake body: body ownership illusions for studying the multisensorybasis of own-body perception.” Frontiers in human neuroscience,vol. 9, p. 141, 2015.

[15] Y. Yuan and A. Steed, “Is the Rubber Hand Illusion Induced byImmersive Virtual Reality?” in Proceedings of the IEEE Conferenceon Virtual Reality (IEEE VR ’10), 2010, pp. 95–102.

[16] A. Kalckert and H. H. Ehrsson, “The spatial distance rule in themoving and classical rubber hand illusions,” Consciousness andCognition, vol. 30, pp. 118–132, 2014.

[17] A. Zenner and A. Kruger, “Estimating Detection Thresholds forDesktop-Scale Hand Redirection in Virtual Reality,” in Proceedingsof the IEEE Conference on Virtual Reality (IEEE VR ’19), 2019, pp.47–55.

[18] F. Argelaguet, L. Hoyet, M. Trico, and A. Lecuyer, “The role ofinteraction in virtual embodiment: Effects of the virtual handrepresentation,” in Proceedings of the IEEE Conference on VirtualReality (IEEE VR ’16), 2016, pp. 3–10.

[19] L. Lin and S. Jorg, “Need a hand? How Appearance Affects theVirtual Hand Illusion,” in Proceedings of the ACM Symposium onApplied Perception. (SAP ’16), 2016, pp. 69–76.

[20] V. Schwind, L. Lin, M. Di Luca, S. Jorg, and J. Hillis, “Touchwith Foreign Hands: The Effect of Virtual Hand Appearance onVisual-Haptic Integration,” in Proceedings of the ACM Symposiumon Applied Perception. (SAP ’18). ACM, 2018, pp. 1–8.

[21] M. Botvinick and J. Cohen, “Rubber hands feel’ touch that eyessee,” Nature, vol. 391, no. 6669, p. 756, 1998.

[22] D. Alais and D. Burr, “The ventriloquist effect results from near-optimal bimodal integration,” Current biology, vol. 14, no. 3, pp.257–262, 2004.

[23] G. Calvert, C. Spence, and B. Stein, The handbook of multisensoryprocesses. MIT Press, 2004.

[24] J. C. Hay, H. L. Pick, and K. Ikeda, “Visual capture produced byprism spectacles,” Psychonomic Science, vol. 2, no. 1-12, pp. 215–216, 1965.

[25] N. P. Holmes, G. Crozier, and C. Spence, “When mirrors lie:”Visual capture” of arm position impairs reaching performance,”

Cognitive, Affective and Behavioral Neuroscience, vol. 4, no. 2, pp.193–200, 2004.

[26] M. Mon-Williams J. P. Wann, et al., “Synaesthesia in the normallimb,” Proceedings of the Royal Society of London, vol. 264, no. 1384,pp. 1007–1010, 1997.

[27] D. H. Warren and W. T. Cleaves, “Visual-proprioceptive inter-action under large amounts of conflict,” Journal of ExperimentalPsychology, vol. 90, no. 2, pp. 206–214, 1971.

[28] M. O. Ernst and M. S. Banks, “Humans integrate visual and hapticinformation in a statistically optimal fashion.” Nature, vol. 415, no.6870, pp. 429–433, 2002.

[29] H. B. Helbig and M. O. Ernst, “Optimal integration of shapeinformation from vision and touch,” Experimental Brain Research,vol. 179, no. 4, pp. 595–606, 2007.

[30] R. J. Van Beers, D. M. Wolpert, and P. Haggard, “When feeling ismore important than seeing in sensorimotor adaptation,” CurrentBiology, vol. 12, no. 10, pp. 834–837, 2002.

[31] R. J. van Beers, A. C. Sittig, and J. J. D. van der Gon, “Integrationof Proprioceptive and Visual Position-Information: An Experimen-tally Supported Model,” Journal of Neurophysiology, vol. 81, no. 3,pp. 1355–1364, 2017.

[32] E. Burns, A. T. Panter, M. C. Whitton, M. R. Mccallus, and F. P.Brooks, “The Hand Is More Easily Fooled than the Eye: UsersAre More Sensitive to Visual Interpenetration than to Visual-Proprioceptive Discrepancy,” Presence, vol. 15, no. 1, pp. 1–15,2006.

[33] S. Razzaque, Z. Kohn, and M. C. Whitton, “Redirected Walking,”in Eurographics, 2001, pp. 2–7.

[34] F. Steinicke, G. Bruder, J. Jerald, H. Frenz, and M. Lappe, “Estima-tion of detection thresholds for redirected walking techniques,”IEEE Transactions on Visualization and Computer Graphics, vol. 16,no. 1, pp. 17–27, 2010.

[35] L. Kohli, E. Burns, D. Miller, and H. Fuchs, “Combining passivehaptics with redirected walking,” in Proceedings of internationalconference on Augmented tele-existence, 2005, p. 253.

[36] P. Haggard and V. Chambon, “Sense of agency,” pp. R390–R392,2012.

[37] C. Farrer, M. Bouchereau, M. Jeannerod, and N. Franck, “Effect ofDistorted Visual Feedback on the Sense of Agency,” BehaviouralNeurology, vol. 19, no. 1-2, pp. 53–57, 2008.

[38] N. Franck, C. Farrer, N. Georgieff, M. Marie-Cardine, J. Dalery,T. D’Amato, and M. Jeannerod, “Defective recognition of one’sown actions in patients with schizophrenia,” American Journal ofPsychiatry, vol. 158, pp. 454–459, 2001.

[39] M. V. Sanchez-Vives, B. Spanlang, A. Frisoli, M. Bergamasco, andM. Slater, “Virtual hand illusion induced by visuomotor correla-tions,” PLoS ONE, vol. 5, no. 4, p. e10381, 2010.

[40] D. M. Lloyd, “Spatial limits on referred touch to an alien limb mayreflect boundaries of visuo-tactile peripersonal space surroundingthe hand,” Brain and Cognition, vol. 64, no. 1, pp. 104–109, 2007.

[41] E. Kokkinara, M. Slater, and J. Lopez-Moliner, “The Effects ofVisuomotor Calibration to the Perceived Space and Body, throughEmbodiment in Immersive Virtual Reality,” ACM Transactions onApplied Perception, vol. 13, no. 1, pp. 1–22, 2015.

[42] D. M. Wegner and T. Wheatley, “Apparent mental causation:Sources of the experience of will,” American Psychologist, vol. 54,no. 7, pp. 480–492, 1999.

[43] T. I. Nielsen, “VOLITION: A NEW EXPERIMENTAL AP-PROACH,” Scandinavian Journal of Psychology, vol. 4, no. 1, pp.225–230, 1963.

[44] B. Caola, M. Montalti, A. Zanini, A. Leadbetter, and M. Martini,“The Bodily Illusion in Adverse Conditions: Virtual Arm Owner-ship During Visuomotor Mismatch,” Perception, vol. 47, no. 5, pp.477–491, 2018.

[45] E. Kokkinara, K. Kilteni, K. J. Blom, and M. Slater, “First PersonPerspective of Seated Participants over a Walking Virtual BodyLeads to Illusory Agency over the Walking,” p. 28879, 2016.

[46] M. R. Longo, F. Schuur, M. P. Kammers, M. Tsakiris, and P. Hag-gard, “What is embodiment? A psychometric approach,” Cogni-tion, vol. 107, no. 3, pp. 978–998, 2008.

[47] D. Banakou and M. Slater, “Body ownership causes illusory self-attribution of speaking and influences subsequent real speaking,”Proceedings of the National Academy of Sciences, vol. 111, no. 49, pp.17 678–17 683, 2014.

[48] T. Dummer, A. Picot-Annand, T. Neal, and C. Moore, “Movementand the rubber hand illusion,” Perception, vol. 38, no. 2, pp. 271–280, 2009.

This is the author's version of an article that has been published in this journal. Changes were made to this version by the publisher prior to publication.The final version of record is available at http://dx.doi.org/10.1109/TVCG.2020.2964758

Copyright (c) 2020 IEEE. Personal use is permitted. For any other purposes, permission must be obtained from the IEEE by emailing pubs-permissions@ieee.org.

JOURNAL OF LATEX CLASS FILES, VOL. XX, NO. X, XXX XXXX 15

[49] T. Asai, “Agency elicits body-ownership: proprioceptive drift to-ward a synchronously acting external proxy.” Experimental brainresearch, vol. 234, no. 5, pp. 1163–74, may 2016.

[50] N. Braun, S. Debener, N. Spychala, E. Bongartz, P. Soros, H. H.Muller, and A. Philipsen, “The senses of agency and ownership: Areview,” p. 535, 2018.

[51] B. Nierula, M. Martini, M. Matamala-Gomez, M. Slater, and M. V.Sanchez-Vives, “Seeing an Embodied Virtual Hand is AnalgesicContingent on Colocation,” Journal of Pain, vol. 18, no. 6, pp. 645–655, 2017.

[52] K. Kilteni, R. Groten, and M. Slater, “The Sense of Embodimentin Virtual Reality,” Presence: Teleoperators and Virtual Environments,vol. 21, no. 4, pp. 373–387, 2012.

[53] N. Ogawa, T. Narumi, and M. Hirose, “Virtual Hand Realism Af-fects Object Size Perception in Body-Based Scaling,” in Proceedingsof the IEEE Conference on Virtual Reality (IEEE VR ’19), 2019, pp.519–528.

[54] D. Perez-Marcos, M. V. Sanchez-Vives, and M. Slater, “Is my handconnected to my body? The impact of body continuity and armalignment on the virtual hand illusion,” Cognitive Neurodynamics,vol. 6, no. 4, pp. 295–305, 2012.

[55] N. P. Holmes, H. J. Snijders, and C. Spence, “Reaching with alienlimbs: Visual exposure to prosthetic hands in a mirror biasesproprioception without accompanying illusions of ownership,”Perception and Psychophysics, vol. 68, no. 4, pp. 685–701, 2006.

[56] A. Haans, W. A. IJsselsteijn, and Y. A. W. de Kort, “The effect ofsimilarities in skin texture and hand shape on perceived owner-ship of a fake limb,” Body Image, vol. 5, no. 4, pp. 389–394, 2008.

[57] M. Rohde, M. Luca, and M. O. Ernst, “The rubber hand illusion:Feeling of ownership and proprioceptive drift Do not go hand inhand,” PLoS ONE, vol. 6, no. 6, p. e21659, 2011.

[58] K. Matsumiya, “Separate multisensory integration processes forownership and localization of body parts,” Scientific Reports 20199:1, vol. 9, no. 1, p. 652, 2019.

[59] M. Tsakiris, L. Carpenter, D. James, and A. Fotopoulou, “Handsonly illusion: Multisensory integration elicits sense of ownershipfor body parts but not for non-corporeal objects,” ExperimentalBrain Research, vol. 204, no. 3, pp. 343–352, 2010.

[60] M. Tsakiris, A. Tajadura-Jimenez, and M. Costantini, “Just a heart-beat away from one’s body: interoceptive sensitivity predicts mal-leability of body-representations.” Proceedings. Biological sciences /The Royal Society, vol. 278, no. 1717, pp. 2470–2476, 2011.

[61] K. Suzuki, S. N. Garfinkel, H. D. Critchley, and A. K. Seth,“Multisensory integration across exteroceptive and interoceptivedomains modulates self-experience in the rubber-hand illusion,”Neuropsychologia, vol. 51, no. 13, pp. 2909–2917, 2013.

[62] S. N. Garfinkel, A. K. Seth, A. B. Barrett, K. Suzuki, and H. D.Critchley, “Knowing your own heart: Distinguishing interoceptiveaccuracy from interoceptive awareness,” Biological Psychology, vol.104, pp. 65–74, 2015.

[63] F. Weichert, D. Bachmann, B. Rudak, and D. Fisseler, “Analysisof the accuracy and robustness of the Leap Motion Controller,”Sensors, vol. 13, no. 5, pp. 6380–6393, 2013.

[64] D. T. Han, M. Suhail, and E. D. Ragan, “Evaluating remappedphysical reach for hand interactions with passive haptics in virtualreality,” IEEE Transactions on Visualization and Computer Graphics,vol. 24, no. 4, pp. 1467–1476, 2018.

[65] E. Burns and F. P. Brooks, “Perceptual sensitivity to vi-sual/kinesthetic discrepancy in hand speed, and why we mightcare,” in Proceedings of the ACM symposium on Virtual reality softwareand technology, 2006, p. 3.

[66] W. Wen, A. Yamashita, and H. Asama, “The influence of perfor-mance on action-effect integration in sense of agency,” Conscious-ness and Cognition, vol. 53, pp. 89–98, 2017.

[67] T. Grossman, T. Grossman, R. Balakrishnan, and R. Balakrishnan,“The bubble cursor: enhancing target acquisition by dynamicresizing of the cursor’s activation area,” in Conference on HumanFactors in Computing Systems, no. c, 2005, pp. 281–290.

[68] M. R. Leek, “Adaptive procedures in psychophysical research,”Perception & Psychophysics, vol. 63, no. 8, pp. 1279–1292, 2001.

[69] M. Tsakiris and P. Haggard, “Experimenting with the acting self,”Cognitive Neuropsychology, vol. 22, no. 3-4, pp. 387–407, 2005.

[70] M. Kammers, F. de Vignemont, L. Verhagen, and H. Dijkerman,“The rubber hand illusion in action,” Neuropsychologia, vol. 47,no. 1, pp. 204–211, 2009.

[71] M. Tsakiris, P. Gita, and P. Haggard, “Having a body versus mov-ing your body: Neural signatures of agency and body-ownership,”Consciousness and Cognition, vol. 15, no. 2, pp. 423–432, 2006.

[72] R. Schandry, “Heart Beat Perception and Emotional Experience,”Psychophysiology, vol. 18, no. 4, pp. 483–488, 1981.

[73] O. Pollatos, A. L. Kurz, J. Albrecht, T. Schreder, A. M. Kleemann,V. Schopf, R. Kopietz, M. Wiesmann, and R. Schandry, “Reducedperception of bodily signals in anorexia nervosa,” Eating Behaviors,vol. 9, no. 4, pp. 381–388, 2008.

[74] M. Gonzalez-Franco and T. C. Peck, “Avatar Embodiment. To-wards a Standardized Questionnaire,” Frontiers in Robotics and AI,vol. 5, p. 74, 2018.

[75] C. Preston, “The role of distance from the body and distance fromthe real hand in ownership and disownership during the rubberhand illusion,” Acta Psychologica, vol. 142, no. 2, pp. 177–183, 2013.

[76] J. P. Noel, M. Samad, A. Doxon, J. Clark, S. Keller, and M. DiLuca, “Peri-personal space as a prior in coupling visual andproprioceptive signals,” Scientific Reports, vol. 8, no. 1, p. 15819,dec 2018.

[77] T. R. Makin, N. P. Holmes, and H. H. Ehrsson, “On the otherhand: Dummy hands and peripersonal space,” Behavioural BrainResearch, vol. 191, no. 1, pp. 1–10, 2008.

[78] M. R. Longo and S. F. Lourenco, “On the nature of near space:Effects of tool use and the transition to far space,” Neuropsychologia,vol. 44, no. 6, pp. 977–981, 2006.

[79] Y. Rossetti, G. Rode, L. Pisella, A. Farne, L. Li, D. Boisson, andM. T. Perenin, “Prism adaptation to a rightward optical deviationrehabilitates left hemispatial neglect,” Nature, vol. 395, no. 6698,pp. 166–169, 1998.

[80] J. W. Moore, “What is the sense of agency and why does it matter?”p. 1272, 2016.

[81] A. Desantis, C. Roussel, and F. Waszak, “On the influence of causalbeliefs on the feeling of agency,” Consciousness and Cognition,vol. 20, no. 4, pp. 1211–1220, 2011.

[82] J. W. Moore, D. M. Wegner, and P. Haggard, “Modulating the senseof agency with external cues,” Consciousness and Cognition, vol. 18,no. 4, pp. 1056–1064, 2009.

[83] D. Wenke, S. M. Fleming, and P. Haggard, “Subliminal primingof actions influences sense of control over effects of action,”Cognition, vol. 115, no. 1, pp. 26–38, apr 2010.

[84] K. Linser and T. Goschke, “Unconscious modulation of the con-scious experience of voluntary control,” Cognition, vol. 104, no. 3,pp. 459–475, sep 2007.

[85] Y. Lee, I. Jang, and D. Lee, “Enlarging just noticeable differencesof visual-proprioceptive conflict in VR using haptic feedback,” inProceedings of IEEE World Haptics Conference. IEEE, 2015, pp. 19–24.

Nami Ogawa received the M.A.Sc. degree ininformation studies from the University of Tokyo,Japan, in 2017. She is currently working towardthe Ph.D. degree in engineering at the Universityof Tokyo. Her research interests include avatarembodiment and embodied perception.

Takuji Narumi Takuji Narumi is a lecturer atthe Graduate School of Information Scienceand Technology, the University of Tokyo. His re-search interests include perceptual modificationand human augmentation with virtual and aug-mented reality technologies. More recently, asa JST PREST researcher, he is also directingthe Ghost Engineering Project which aims atutilizing the effect of virtual body on our mind andcognitive functions to encourage better commu-nication between people.

Michitaka Hirose Michitaka Hirose is a profes-sor of Human Interface and Virtual Reality inthe Graduate school of information science andtechnology, the University of Tokyo. He receivedBE, ME, PhD in Mechanical Engineering fromthe University of Tokyo, in 1977, 1979, 1982, re-spectively. He received The 2015 VGTC VirtualReality Career Award.

This is the author's version of an article that has been published in this journal. Changes were made to this version by the publisher prior to publication.The final version of record is available at http://dx.doi.org/10.1109/TVCG.2020.2964758

Copyright (c) 2020 IEEE. Personal use is permitted. For any other purposes, permission must be obtained from the IEEE by emailing pubs-permissions@ieee.org.