OpticalMarionette:GraphicalManipulationofHuman’sWalkingDirection

AkiraIshii,Ippei Suzuki,ShinjiSakamoto,KeitaKanai

Kazuki Takazawa,Hiraku Doi,YoichiOchiai

(DigitalNatureGroup,UniversityofTsukuba,Japan)

Background|RedirectedWalking

n Manipulationtechniqueofhuman’swalkingdirection

forVRenvironment

n Usesonly visualfeedback forthemanipulation

n Enablesuserstoexploreavirtualworldthat

isquitelargerthantherealenvironment

2Figureby

Matsumotoetal.2016

Introduction|Motivation

n Suspectedtherearepossibilitiesthathuman’swalking

directioncouldbe manipulatedusingonlyvisualfeedback

intherealenvironmentlikeaRedirectedWalkinginVR

3

Tomanipulateuser’swalkingdirection

usingonly visualfeedbackintherealenvironment

Ourgoal

Introduction|ToThisEnd

n Examinedvariousimage-processingmethods

tomanipulatewalkingpath

n Investigatedhowtomanipulatethewalkingpath

usingtheimage-processingmethods

4

Wefoundeffectivereorientingmethod

usinganHMDintherealenvironment

Implementation

n ThecameraisattachedtotheHMD

n Usersperceivetherealworldbyvideo

providedbyanHMDandstereocamera

n Sowecancoordinateusers’sightinthissetup5

�

HMD Stereocamera

=

See-through

Implementation|Hardware

n HMD:OculusRiftDK2

n Stereocamera:Ovrvision

6

HMD

Camera

Implementation|Hardware

7

User’ssightviaHMD+camera

whilewalking

Implementation|Flow

8

Implementation|Flow

9

Investigatedeffective

imageprocessing inapilotstudy

PilotStudy

n Purpose

- Todeterminewhichoftheimageprocessingmethods

havethemosteffectonahuman’swalkingpath

n Participants

- 5participants(1female);18-22ages

n 6imageprocessingmethods

n Task

-Walkstraight10mforeachimageprocessingmethods

10

PilotStudy|Result

11

Implementation|ChangingFocalRegion(CFR)

12

Rawvideo

fromcamera

Videothatuserssee

viaHMD

Cropstherawimage

Shiftsthecroppedarea

Implementation|ChangingFocalRegion(CFR)

13

Rawvideo

fromcamera

Videothatuserssee

viaHMD

Cropstherawimage

Shiftsthecroppedarea

UserisaskedtogototheB,however,actuallyheisgoingtotheA

Experiment

14

n Purpose

-Todeterminehowtocontrolthewalkingdirection

usingChangingfocalregionmethod

n Participants

-16participants-Hadnotpriorknowledgeofourexperiment

n Task

-Walkstraight24mforeachimageprocessing

Experiment|ExperimentalDesign

n Imageprocessingmethods

-Noprocessing(rawvideo)-Magnification

- Magnificationwasaddedtotheimage-processingmethodsforcomparison

becauseitisusedinChangingfocalregionmethod.

Thisenabledustoidentifytheeffectsofimagemagnificationand

torevealthepureeffectsofchangingthefocalregion(shiftingimages).

- Changingfocalregion(slow/fast)- Scrollspeedofthecroppedareaintheslowconditionwas0.5px/frame

- Inthefastcondition,thescrollspeedofthecroppedareawastwice

thescrollspeedintheslowcondition15

Magnification+Shifting

Changingfocalregion

Experiment|ExperimentalDesign

n Locations

-Hallway- Narrow

- Severalhintsforspatial

perception(suchaswalls)

-Outside- Largespace

- Nohint

forspatialperception

16

1. Completedpre-SSQ(SimulatorSickness

Questionnaire)

2. Walkedstraightfor24meachofimageprocessing

at2locations

-At4m,guidedtotheleft

-At14m,guidedtotheright

3. Completedpost-SSQ

Experiment|Procedure

17

Result|SimulatorSicknessQuestionnaire

n AnalyzedtheSSQscoreswitht-test

-Nosignificantdifferencebetweenpre-SSQandpost-SSQ-Pre-SSQ:4.7(SD=4.7)-Post-SSQ:6.7(SD=8.3)

18

Ididnotfeel

anymotionsicknessJ

Result|Noprocessing

19

Positionofparticipants[m]

Distancefromstartingpoint[m]

Result |Changingfocalregion(fast)

20

Positionofparticipants[m]

Distancefromstartingpoint[m]

Result

21

Noprocessing Changingfocalregion

Noprocessing

Changingfocalregion

Result|Hallway

n Significanteffect

-Noprocessing―Changingfocalregion(slow/fast)

-Magnification ―Changingfocalregion(slow/fast)

n Nosignificanteffect

-Noprocessing ―Magnification

22

Positionofparticipants[m]

Noprocessing Magnification CFR(slow) CFR(fast)

Result|Hallway

n AmountofthemanipulationforChangingfocalregion

-Slow:65.3mm/m

- Ifyou walk 2m,your position moves by10cmhorizontally

-Fast:75.3mm/m

23

No

Amountofmanipulation[mm]

* guidetoleft

** guidetoright

Result|Outside

n Significanteffect

-Noprocessing―Changingfocalregion(slow/fast)

-Magnification ―Changingfocalregion(slow/fast)

n Nosignificanteffect

-Noprocessing ―Magnification

24

Positionofparticipants[m]

Noprocessing Magnification CFR(slow) CFR(fast)

Result|Outside

n AmountofthemanipulationforChangingfocalregion

-Slow:105.2mm/m

-Fast:199.2mm/m

25

NoAmountofmanipulation[mm]

* guidetoleft

** guidetoright

Result|PureEffectofCFR

n Nosignificanteffect

betweenNoprocessingandMagnification

-Thisindicatesthatthecroppedimagebyitselfdidnot

affecttheparticipants’walkingpaths

- Itisclearthattheparticipants’walkingpathswereaffectedbymovementofthecroppedareaof

Changingfocalregion

26

Result|Hallwayvs.Outside

n Significanteffectwithinthelocations

-Theoutside participantswereaffectedmore

bythemanipulationmethod

- Inthehallway,theparticipantscouldnotmovemorethan1.1m

becausethewithofthehallwayis2.2m

- Thus,themeanvalueofthepositionchangeinthehallwaywas

smallerthanthatintheoutside

27

Result|Slowvs.Fast

n Changingfocalregion(fast)wassignificantlymore

effectivethantheslowcondition

-Slow:105.2mm/m

-Fast:199.2mm/m(atoutside)

28

Discussion|SpatialPerception

n PastresearchreportedChangingtheFOVhaseffects

onspatialperception[1,2]

n Nohorizontalspatialperceptioneffectonourresult

-becausenosignificantdifferencebetweenNoprocessingandMagnification

n Bycontrast,wecouldnotdetermine

whethertherewasanydepthperceptioneffect

29

Inthisstudy

[1]Campos,J.,Freitas,P.,Turner,E.,Wong,M.,andSun,H.Theeffectofopticalmagnification/minimizationondistanceestimationbystationaryand

walkingobservers.JournalofVision7,9(June2007),1028.

[2]Kuhl,S.A.,Thompson,W.B.,andCreem-Regehr,S.H.HMDcalibrationanditseffectsondistancejudgments.ACMTransactionsonApplied

Perception6,3(September2009),19:1–19:20.

Discussion|CognitiveResource

n Conventionalnavigationsrequireuserstorecognize

information(gototheright)andthenfollow directions

n Ourmethoddirectlyaffectsusers’bodies

sothatitcancontrolthem

withoutrequiringuserrecognitionprocess

30

Doesourmethodhavelighterburden

thanconventionalnavigations?

Discussion|CognitiveResource

n Significantamountofcognitiveresourcesisrequired

forredirectedwalkinginVR[3]

Similarly,ourmethodmightrequiresomecognitive

resources

31

Weplantoinvestigatehowmuchcognitive

resourcesarereallyrequired

byuserstofollowthemanipulation

[3]Bruder,G.,Lubas,P.,andSteinicke,F.Cognitiveresourcedemandsofredirectedwalking.IEEETransactionsonVisualizationandComputer

Graphics21,4(April2015),539–544.

Discussion|Feasibility

n DemonstratedourmethodatSIGGRAPH2016E-Tech

n Over700people weresuccessfullymanipulated

32

Direction:

GostraighttoB

Applications|ARContents

n Insee-throughARcontents,

tocontrolofhuman’swalkingpathisimportant

becauseusersmightconflicteachother

33

Applications|WalkerNavigation

n Thereispossibilityofwalkernavigationsystem,

ifwecanincrease

theamountofmanipulationofusers’walkingdirection

34

RelatedWork |Galvanicvestibularstimulation (GVS) [4]

n Administerselectricalstimulationtothebackofears

(thevestibules)

n Controlsthewalker’ssenseofbalance

35[4]Maeda,T.,Ando,H.,Iizuka,H.,Yonemura,T.,Kondo,D.,andNiwa,M.Parasitichumanoid:Thewearableroboticsasabehavioralassistinterface

likeonenessbetweenhorseandrider.InProc.AH’11,18:1–18:8.

RelatedWork|ElectricalMuscleStimulation (EMS)[5]

n EMS-basedwalkernavigationsystem

n Controlswalker’slegsusingEMS

36[5]MaxPfeiffer,TimDunte,StefanSchneegass,FlorianAlt,MichaelRohs.CruiseControlforPedestrians:ControllingWalkingDirectionusing

ElectricalMuscleStimulation.CHI2015,pp.2505-2514,2015.

RelatedWork

n Thesemethodsuseelectricalstimulation

n Ourmethodisvision-based manipulation

techniqueofhuman’swalkingdirectionusing

only visualfeedback

37[4]MaxPfeiffer,TimDunte,StefanSchneegass,FlorianAlt,MichaelRohs.CruiseControlforPedestrians:ControllingWalkingDirectionusing

ElectricalMuscleStimulation.CHI2015,pp.2505-2514,2015.

OpticalMarionette:

n Wefoundeffectiveimage-processingmethodsfor

walkermovementcontrolwithanHMD

n Weinvestigatedoftheeffectsof

theimage-processingmethodsviaauserstudy

n Changingfocalregionmethodwasmosteffective,

andchangedwalkingpathbyabout200mm/m

38

AkiraIshii,Ippei Suzuki,ShinjiSakamoto,KeitaKanai

Kazuki Takazawa,Hiraku Doi,YoichiOchiai (UniversityofTsukuba)

ishii@akira.io

GraphicalManipulationofHuman’sWalkingDirection