Collaborative Mixed Reality Visualization of an Archaeological Excavation Hrvoje Benko Department of...
-
Upload
sean-henderson -
Category
Documents
-
view
213 -
download
1
Transcript of Collaborative Mixed Reality Visualization of an Archaeological Excavation Hrvoje Benko Department of...
Collaborative Mixed RealityCollaborative Mixed Reality Visualization of an Archaeological Visualization of an Archaeological
ExcavationExcavation
Hrvoje BenkoHrvoje BenkoDepartment of Computer ScienceDepartment of Computer Science
Columbia UniversityColumbia University
MIT talk – January 12, 2005MIT talk – January 12, 2005
2
Multidisciplinary TeamMultidisciplinary Team
3
MotivationMotivation
►Excavation is destructive and Excavation is destructive and physically “unreconstructable” processphysically “unreconstructable” process
►Need to preserve as much data as Need to preserve as much data as possible for analysispossible for analysis
►Data interpretation happens off-siteData interpretation happens off-site►Current tools focus on 2D data and do Current tools focus on 2D data and do
not incorporate 3D informationnot incorporate 3D information►Many experts—collaboration is a must!Many experts—collaboration is a must!
4
Archaeological Excavation at Archaeological Excavation at Monte Polizzo, Sicily, Summer Monte Polizzo, Sicily, Summer
20032003Ian Morris, Director (Stanford University)Ian Morris, Director (Stanford University)
5
Working in the field!Working in the field!
6
Meshed 3D Model (13 scans)Meshed 3D Model (13 scans)
7
We have a 3D site model, We have a 3D site model, now what?now what?
RealReal VirtualVirtual
8
High Resolution Images
DrawingsField Notes
Panoramic Images
Videos
GIS Data
9
High Resolution Images
DrawingsField Notes
Panoramic Images3D Object Models
3D Site Model Videos
GIS Data
Database
10
Two ProblemsTwo Problems
►How to combine all this data in one How to combine all this data in one seamless environment?seamless environment?
►How to make it easy to interact with?How to make it easy to interact with?
11
VITA:VITA: Visual Visual InteractionInteraction Tool Tool forfor
ArchaeologyArchaeology► Multiple usersMultiple users► Multiple displaysMultiple displays
Projected tabletopProjected tabletop HandheldHandheld High-resolution High-resolution
monitormonitor See-through head-See-through head-
wornworn► Multiple interaction Multiple interaction
devicesdevices MERL DiamondTouch MERL DiamondTouch
tabletable EssentialReality P5 EssentialReality P5
glovesgloves Speech inputSpeech input 6DOF tracker6DOF tracker
12
Design ConsiderationsDesign Considerations
►Use the most appropriate display for Use the most appropriate display for the given datathe given data
►Facilitate both human-system and Facilitate both human-system and human-human interactionhuman-human interaction
13
Modular ArchitectureModular Architecture
AR Module
DT ModuleSCREEN Module
HANDHELDModule
AR Module …
DatabaseMessageFacilitator
2D Modules
3D Modules
14
AR Module ComponentsAR Module Components
Head TrackerHand Tracker
(Intersense IS900)
Sony Head-Worn Display(LDI-D100B)
Microphone
P5 Glove(Essential Reality)
DiamondTouch Table Connector
(MERL)
15
Life-size Immersive Life-size Immersive ExplorationExploration
16
3D Multimodal Interaction3D Multimodal Interaction
► Provide natural interaction Provide natural interaction mechanism for our 3D mechanism for our 3D environmentenvironment
► ModalitiesModalities Speech: IBM ViaVoice 10Speech: IBM ViaVoice 10 Gestures: Gestures:
EssentialReality P5 gloveEssentialReality P5 glove Selection statistics: Selection statistics:
SenseShapesSenseShapes
► Focus on selectionFocus on selection Based on collaboration with Phil Based on collaboration with Phil
Cohen et al. (Cohen et al. (ICMI 2003ICMI 2003) and ) and SenseShapes (SenseShapes (ISMAR 2003ISMAR 2003) )
17
VirtualTrayVirtualTray
18
Desktop ComponentsDesktop Components
Touch-sensitive Projective Display
(MERL DiamondTouch)
High-resolutionDisplay
Handheld Display
19
World-In-MiniatureWorld-In-Miniature
20
Harris MatrixHarris Matrix
21
Enhanced Harris Matrix Enhanced Harris Matrix
22
Enhanced Harris MatrixEnhanced Harris Matrix
23
Cross-Dimensional Hybrid Cross-Dimensional Hybrid GesturesGestures
►Synchronized 2D and 3D gesturesSynchronized 2D and 3D gestures►Facilitate seamless transition across Facilitate seamless transition across
dimensionsdimensions
To appear in IEEE VR 2005
24
Cross-Dimensional Hybrid Cross-Dimensional Hybrid GesturesGestures
PullPush
To appear in IEEE VR 2005
25
Cross-Dimensional Hybrid Cross-Dimensional Hybrid GesturesGestures
PullPinDragRotate Push
To appear in IEEE VR 2005
26
Cross-Dimensional Hybrid Cross-Dimensional Hybrid GesturesGestures
PullConnectScaleDisconnect Push
To appear in IEEE VR 2005
27
Handheld Focus-in-Context Handheld Focus-in-Context DisplayDisplay
►Movable high-Movable high-resolution insetresolution inset Tracked by Tracked by
DiamondTouchDiamondTouch Projection Projection
suppressed in its suppressed in its boundsbounds
Physical magic Physical magic lenslens
28
Tabletop InteractionTabletop Interaction
29
User FeedbackUser Feedback
►Overall very positive reactionOverall very positive reaction►Archaeologists benefited from:Archaeologists benefited from:
Temporal–Spatial connectionTemporal–Spatial connection Aggregated collection of all dataAggregated collection of all data Accurate 3D modelAccurate 3D model Simple touch-based interactions Simple touch-based interactions
►Potential for increased collaborationPotential for increased collaboration
30
Room for ImprovementRoom for Improvement► Reduce wiresReduce wires► Reduce weight Reduce weight ► Eye occlusion hinders communication Eye occlusion hinders communication ► Missing data:Missing data:
More objects, features, notes and picturesMore objects, features, notes and pictures More scans during excavation (time-lapse spatial More scans during excavation (time-lapse spatial
record) record) ► Missing features:Missing features:
Virtual scale measure (implemented since)Virtual scale measure (implemented since) Variable site model scalingVariable site model scaling Improved selection in world-in-miniatureImproved selection in world-in-miniature
31
Current and Future WorkCurrent and Future Work
► Larger Site:Larger Site: Summer 2004 - Thulamela, South Summer 2004 - Thulamela, South
AfricaAfrica
► Personalized user experience Personalized user experience based on expertisebased on expertise
► Environment managementEnvironment management
32
AcknowledgmentsAcknowledgments► Special thanks to: Special thanks to:
Peter Allen, Alejandro Troccoli, and Ben Smith (CU Robotics Peter Allen, Alejandro Troccoli, and Ben Smith (CU Robotics Lab)Lab)
Ian Morris and Trinity Jackman (Stanford Archaeology Ian Morris and Trinity Jackman (Stanford Archaeology Center) Center)
Lynn Meskell and James Conlon (CU Dept. of Archaeology) Lynn Meskell and James Conlon (CU Dept. of Archaeology) Sajid Sadi and Avinanindra Utukuri (P5 glove) Sajid Sadi and Avinanindra Utukuri (P5 glove) Shezan BaigShezan Baig Mitsubishi Electric Research Labs (DiamondTouch table)Mitsubishi Electric Research Labs (DiamondTouch table) Alias Systems Alias Systems Microsoft Research Microsoft Research
► Funded in part by NSF ITR Grants IIS-0121239 and Funded in part by NSF ITR Grants IIS-0121239 and IIS-00-82961, and Office of Naval Research IIS-00-82961, and Office of Naval Research Contracts N00014-99-1- 0394, N00014-99-1-0683, Contracts N00014-99-1- 0394, N00014-99-1-0683, N00014-99-1-0249, and N00014-04-1-0005.N00014-99-1-0249, and N00014-04-1-0005.
Questions?Questions?
34