An Introduction toAn Introduction to Virtual Reality Virtual Reality

Iman Mohammad Rezazadeh

School of Biomedical Engineering

Science and Research Branch

Islamic Azad University

Spring 2010

Virtual Environment (VE)Virtual Environment (VE)A computer generated world with

which the user can interact

Interaction can vary from looking around to interactively modifying the world.

What is virtual reality(VR)?

A computer generated,immersive(or wide field),multi-sensory informationprogram which tracks a user in realtime.

HistoryHistory

Year Person(s) Responsible Why it was important

1965 Ivan Sutherland The beginnings of VR

1977Dan Sandin, Richard Sayre and Thomas Defanti

Interaction through body movement

1982 Bonnie MacBird (Writer) The first computer- generated movie

1983 Myron Krueger First virtual environment

1984 William Gibson (His assistant) The term 'Cyberspace'

1987 Michael Piller (Writer) The Holodeck, idea of immersive VR

1992 Stephen King (Official Website) A look at the possible negative side of VR

1995 Silicon Graphics Virtual Reality Modeling Language

1999 Larry and Andy Wachowski Virtual Reality movie grosses $750M worldwide

TypesTypes

ImmersiveAugmented Text-basedDesktop (Window on a World)Video Mapping

Computer Graphics vs. VRComputer Graphics vs. VRCG is concerned with the modeling, lighting

and dynamics of environments and in general how to build a medium – so forth called Virtual World.

But, designing a VE is something beyond designing a CG.

When we become concern about how people experience the simulated environment, (re)act within it and interact with its components through the designed human-machine interface (HMI), we are talking about VE or VR

What is Immersive VR?What is Immersive VR?

A type of VR in which the user becomes immersed (deeply involved) in a virtual world. it is also a form of VR that uses computer related components.

Types of Non-immersive Types of Non-immersive VRVRText-based VR: when a reader

of a certain text form a mental model of this virtual world in their head from the description of people , places and things.

Augmented VR: the idea of taking what is real and adding to it in some way so that user obtains more information from their environment.

ApplicationsApplicationsArchitectureTrainingMedicineEngineering and DesignE-CommerceEntertainmentManufacturing

DevicesDevices

Head Mounted Displays (HMD):

was the first device providing its wearer with an immersive experience. Evans and Sutherland demonstrated a head-mounted stereo display already in 1965.

3D VIDEO Eyewear3D VIDEO Eyewear

winner of a 2005 & 2006 Consumer Electronics Show Design

Award.

VR Price: $499.00

Binocular Omni-Orientation Binocular Omni-Orientation Monitor (BOOM)Monitor (BOOM)

BOOM is a head coupled stereoscopic display device. Screens and optical system are housed in a box that is attached to a multi-link arm.

VR… VE…VR… VE…

14

AR in teleconferencingAR in teleconferencing

• person works at real desk

• remote collaborator represented by picture or video or “talking head”

• objects of discussion; e.g. a patient’s brain image, might also be fused into visual field

• HOW IS THIS ACHIEVED?

From University of Washington HIT Lab

Cave Automatic Virtual Cave Automatic Virtual Environment (CAVE)Environment (CAVE)

A room with projections on all walls, floor and ceiling

The users wear shutter glasses to get a 3D view of the world.

The users are able to move and control the environment with some kind of input mechanism◦ Camera◦ Device in hand

Bam and its Cultural Bam and its Cultural LandscapeLandscape After the earthquake of the 26th December

2003 and the immense destruction of the Bam heritage in Iran, the reconstruction of the ruins especially of the citadel of Bam has become an important task for experts.first phase of the project were to develop the 3Dimensional models of 7 case studies inside the citadel of Bam (Main gate,Bazaar,Tekkiyeh (religious plaza), Mosque, Sistani house, barrack, stable, four season reception room

Cont….Cont…. The team of drawing the 3D models

consisting of in Japan from Waseda University and the team of Raaz Ahang architectural firm in Iran from University of Tehran has made a tremendous effort to build the 3D models.

How about… learning to hang-glide?

I’m more of the down to earth type...

How aboutlearning todrive a car?

© www.ozedweb.com

Hey, that wasn’t there a moment ago...

Things are easywith VR.

Real Estate Agents

use it every day.

© www.ozedweb.com

...And Architects love it!

Wait till Wait till youyou

WalkWalkthrough!through!

© www.ozedweb.com

At last… something real.

Sorry… Sorry…

That’s VRThat’s VRtoo.too.

© www.ozedweb.com

Ahh...

It goes here.It goes here.

© www.ozedweb.com

Princeton Video WallPrinceton Video Wall

Augmented RealityAugmented Reality

A nice Little ARA nice Little AR

Share Virtual EnvironmentShare Virtual EnvironmentYou are a Avatar!!!! Wow…You are a Avatar!!!! Wow…

Classroom is a VEClassroom is a VE

VR CAVEVR CAVE

VR WorkbenchVR Workbench

Norman’s AffordancesNorman’s AffordancesAffordances:

◦Have perceived properties that may or may not exist

◦Have suggestions or clues about to how to use these properties

◦Can be dependent on the Experience Knowledge Culture of the actor

◦Can make an action easy or difficult

From McGrenere & Ho, Proc of Graphics Interfaces, 2000

Affordances of a Teapot?Affordances of a Teapot?

Slide adapted from Saul Greenberg

Real vs. Perceived Real vs. Perceived AffordancesAffordancesIn product design, where one deals

with real, physical objects, there can be both real and perceived affordances, and the two need not be the same.

Slide adapted from Saul Greenberg

Based on slide by Saul Greenberg

Transfer EffectsTransfer EffectsPeople transfer their

expectations from familiar objects to similar new ones◦positive transfer: previous

experience applies to new situation◦negative transfer: previous

experience conflicts with new situation

Based on slide by Saul Greenberg

Cultural AssociationsCultural AssociationsGroups of people learn idioms

◦red = danger, green = goBut these differ in different

places◦Light switches

America: down is off Britain: down is on

◦Faucets America: counter-clockwise is on Britain: counter-clockwise is off

VR and CognitionVR and Cognition

VR in Medicine VR in Medicine

I think I’m feeling ill.

Well, that’s not

a problem.

We’ll find acure with theVirtualpatient...

© www.ozedweb.com

…now I’m sure I am.

…in our virtual

hospital.

Why not?

© www.ozedweb.com

Virtual Body vs. CadaverVirtual Body vs. CadaverTraining on cadavers has several drawbacks:

◦ if trainee cuts a nerve or a blood vessel in a cadaver nothing will happen

◦ no action can be reversed on cadavers (what is cut is cut)

◦ dead tissue is harder, color is changed, arteries do not pulsate

Advantages of computer simulations:◦ procedures can be repeated many times with no

damage to virtual body◦ virtual body does not have to be dead - many

functions can be simulated for realistic visualizations

◦ organs can be made transparent and modeled

VR in EndoscopyVR in Endoscopy

Virtual Fighting and Virtual Fighting and Rehab.Rehab.

VR in EndoscopyVR in Endoscopy

VR in EndoscopyVR in Endoscopy

endoScope - A VR ToolendoScope - A VR Tool

EndoScope is a Motif/Inventor based model viewer that runs on SGI workstations

Developed by Biomedical Imaging Resource at Mayo Clinic

VR in ColonoscopyVR in Colonoscopy

VR in LaparoscopyVR in Laparoscopy

VR in LaparoscopyVR in Laparoscopy

MIS Training at EPFLMIS Training at EPFL

Group for surgical robotics and instrumentation, Swiss

Gastro-intestinal organ modeled and tissue properties simulated (pushing, pulling)

Force feedback generated for realistic simulation

Laparoscopic InterfacesLaparoscopic Interfaces

Immersion Corp. devices for minimally invasive surgery simulators

Offers tracking in 5 degrees of freedom (left-right, up-down, in-out, rotation around axis, open-close)

Version with and without force-feedback are available

Price range: up to $8,000

3-D Stereo Visualization3-D Stereo VisualizationUniv. of Illinois Chicago, School of

Biomedical and Health Information Sciences

ImmersaDesk system for temporal bone visualization

Fetus VisualizationFetus Visualization

Univ. of North Carolina at Chapel Hill

A VR system is developed for visualization of a fetus in a pregnant woman’s abdomen

The ultrasound image of the fetus is superimposed on the video image of the woman’s abdomen

The system can be used for pregnancy check-ups

Fraunhofer Institute Fraunhofer Institute SimulatorsSimulators

Fraunhofer Institute medical training simulators

The arthroscopy training simulator

Nasal endoscopy simulator

The trainee is able to practice techniques before facing a real patient

Eye Surgery SimulatorEye Surgery Simulator

Interactive Media Technology Center at Georgia Tech

Eye surgery simulator for:◦ education and training of medical students◦ training of surgeons to cope with emergencies

Simulator provides force feedback information for more realistic simulation of tissue cutting

Simulation of the tissue includes elasticity of the eye surface tissue before a cut is made

Craniofacial Surgery Craniofacial Surgery SimulationSimulation

Erlangen Institute, Germany In craniofacial surgery it is important to plan

and predict the outcome of surgical interventionThe face can be visualized after reconstructive

plastic surgery

Augmented Reality for Augmented Reality for SurgerySurgery

Julesz, Harward Medical SchoolAugmented reality visualization has three phases:

1. 3D laser scanning of the patient’s head surface

2. 3D registration of the scanned and imaged surface

3. Augmented reality display of tumor (green)

Augmented Reality in Augmented Reality in NeurosurgeryNeurosurgery

Harvard Medical School

Combined neurosurgery planning and augmented reality

Phobia ProjectsPhobia Projects

GA Tech GVU Center Virtual Reality Exposure Therapy project

Demonstrated effectiveness Advantages:

◦ cost effective◦ effective therapy◦ patient acceptance◦ suitable for network delivery

(telemedicine)

Spider Phobia Spider Phobia DesensitizationDesensitization

Univ. of Washington Human Interface Technology Lab

VE environment designed that contains virtual spiders (a large brown spider with fur and a small one were used

Patients are encouraged to pick up spiders with their virtual hands

Spiders are unexpectedly dropped of the ceiling, patient could pull the spider legs off

Acrophobia ProjectAcrophobia Project

Univ. of Michigan Balcony view from high floors are

generated The patient gradually watches the

environment from higher and higher viewpoints

A multi-session therapy cures the acrophobia patient

Medical Education at Medical Education at UCSDUCSD

UCSD Applied Technologies Lab project on VR-multimedia system for education of medical students (anatomy)

UCSD Virtual Anatomy WorldAnatomic structures are

linked to supporting multimedia contents to provide VR-MM anatomy lessons

Medical Education at Medical Education at Fraunhofer InstituteFraunhofer Institute

Fraunhofer InstituteVirtual anatomy

system for medical students

Virtual patientStudents are able to

understand complex interrelationships of anatomical structures

Remote Palpation Remote Palpation SimulatorSimulator Interactive Technology Media Center at

Georgia TechThe idea is to allow doctors to examine

patients at a remote location IMTC developed a haptic lens - a sensor that

measures 3D surface under a specific pressure

The device is pressed against an object and 3D surfaces, deformed under the pressure, are recorded in real-time

OK! OK ! No Military UseOK! OK ! No Military UseAnkle TherapyAnkle Therapy

Medical RoboticsMedical Robotics

Introduction(2)Introduction(2)CIS

Information flow in CIS

Motivation:◦Started with the weaknesses and

strengths of minimally invasive surgery (MIS)

◦Smaller incisions, shorter post-operative time, reduced infection, faster rehabilitation, lesser pain, better cosmetics, ...

◦Eye-hand coordination, difficulty in moving arms, degree of motion

ClassificationClassificationDepending on the degree of surgeon interaction during the procedure: ◦Supervisory-controlled;◦Telesurgical;◦Shared-control;

Application of Medical Application of Medical RoboticsRoboticsTelesurgery

◦Surgeon sits at a console

◦Has controls to move the robotic arms

◦Does not operate on the patient directly

◦Mainly used in minimally invasive surgeries

Application of Medical Application of Medical RoboticsRoboticsTelemedicine and Teleconsultation◦Telecommunciation channels to

communicate with other physicians/patients

◦Control an external camera which in turn controls an endoscopic camera – used to share images with a remote surgeon

Application of Medical Application of Medical RoboticsRoboticsRehabilitation◦Assistive

robots◦Wheelchair

with intelligent navigational control system

Application of Medical Application of Medical RoboticsRoboticsRemote surgery

◦Surgeon can be anywhere in the world

◦Remotely controls the robotic surgical system

◦Very useful for treating wounded people in battlefields

Application of Medical Application of Medical RoboticsRoboticsLaparoscopic Surgery

◦Performed in the abdominal cavity using MIS

◦Abdomen cavity is expanded using CO2

◦Uses Laparoscopic instrument Fiber optic channels to illuminate the inside of

abdominal cavity Lens optics to transmit image CCD camera at the outer end Image displayed on high resolution TV

Design of Robotic Design of Robotic TelesurgeryTelesurgeryThe Concept

Telesurgical system concept

Existing surgical systemsExisting surgical systemsDa Vinci Surgical® system by

Intuitive Surgical, Inc.

Existing surgical systemsExisting surgical systemsDa Vinci Surgical® system by

Intuitive Surgical, Inc.

VR in Brain Computer VR in Brain Computer InterfaceInterface

VR in BCIVR in BCI

Monitoring Driver’s Monitoring Driver’s AlternessAlterness

VR, BCI, ImaginationVR, BCI, Imagination

My Office in FutureMy Office in Future

The End of the Beginning

- Surgery Planning and SimulationSurgery Planning and Simulation- Rehabilitation and EmpoweringRehabilitation and Empowering- Mind Engineering Mind Engineering - Neurofeedback and BiofeedbackNeurofeedback and Biofeedback- Mixed RealityMixed Reality- Real VirtualityReal Virtuality- Pervasive computingPervasive computing- Ubiquitous computingUbiquitous computing

© www.ozedweb.com

Invitation for ContributionInvitation for Contribution

"Mixed Reality and Human-Robot Interaction”

Springer

What we are doing in IAUWhat we are doing in IAUVirtual Upper limb and Lower Limb

Rehab.

Mind Engineering and Neurofeedback(CARTEME)

Collaborative Interfacing

Affective Computing