Mobile AR and Electronic Secretary Wouter Pasman.
-
Upload
imogen-byrd -
Category
Documents
-
view
222 -
download
0
Transcript of Mobile AR and Electronic Secretary Wouter Pasman.
![Page 1: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/1.jpg)
Mobile AR and Electronic Secretary
Wouter Pasman
![Page 2: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/2.jpg)
Overview
1. Who am I
2. About UbiCommobile AR
3. About Cactuselectronic secretary
![Page 3: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/3.jpg)
Wouter Pasman
1987-1991: Student Computer Science, Masters completed with honours (Complexity theory, correctness proving, ..)
1991-1993: Student Musicology, completed first 2 years
1993-1997: Ph.D. Industrial Design Engineering, Delft University.
1997-2001: Post-doc work: UbiCom
2001-2002: Post-doc work: Cactus
![Page 4: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/4.jpg)
One of my hobbies…
![Page 5: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/5.jpg)
UbiCom project
Ubiquitous Communications
Mobile Augmented Reality (AR)
VIDEO: UbiCom short intro
![Page 6: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/6.jpg)
Overview
UbiCom structureOutdoor AR TrackingLow latency mobile AR renderingDynamic simplification Mathematical model for rendering ARC framework QoS, Accuracy curves VRML integration
![Page 7: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/7.jpg)
UbiCom structure
Project leader: R.L. Lagendijk (E. Deprettere)
Personal supervisorF.W. JansenComputer Graphics &CAD
P1'wireless' P2 'visual info'
me
P3 'protocols'
![Page 8: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/8.jpg)
P1. Adaptive wireless comm transceivers.
Subproject-leader Wouter Serdijn
5 Ph.Ds
OFDM codec hardwareOFDM on infrared5 and 17GHz radio linkSignal propagation measurements
![Page 9: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/9.jpg)
P2: Audio-visual information processing.
Subproject leader: Richard Heusdens
2 postdocs, 4 PhDs
TrackingLow-latency renderingVideo CodingVisual Presentation FilterRetinal Scanning DisplayGIS database handling
![Page 10: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/10.jpg)
QuickTime™ and aPhoto - JPEG decompressor
are needed to see this picture.
P3: System architecture and protocols.
Subproject leader: Henk Sips
3 postdocs, 2 PhDs
ARC QoS systemLART mobile sysVoltage ScalingP123 interfaceJava compiler
![Page 11: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/11.jpg)
VIDEO: Outdoor AR Tracking
VIDEO: Low latency mobile AR
rendering
![Page 12: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/12.jpg)
Dynamic Simplification
Dynamic LoD generation in backbone
Maximize perf/cost ratio in headset.
![Page 13: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/13.jpg)
Mathematical model per object
• Estimate link and CPU load, memory usage, lifetime of objects, etc • Est screenspace error and geometric distortions
D=0.001R=1m
![Page 14: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/14.jpg)
QoS: Scheduling of resources
Funkhouser,Séquin (1992), Mason,Blake (1997)
Goal: maximize benefit within a cost budget.
Benefit: pixel size on screen,accuracy heuristic
Cost: # triangles and pixel size on screen
![Page 15: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/15.jpg)
Mason showed this is NP complete..
- Iterative approx giving in worst case half
the maximum possible quality
- Quality only known after iteration
- Only feedback loop with application possible
![Page 16: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/16.jpg)
QoS Management
Phase I
At top level, acceptable options for operation of the server are specified or changed.
UbiCom-wide QoS mechanism: ARC
Modeserveroperation spaceModeclientclientoptions
![Page 17: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/17.jpg)
Phase IIa
Server searches for possibilities, considering • internal state• ARC services
A number of options within the requested range are returned.
clientoptionsserver optionsModeserveroperation spaceoperation space 3operation space 2
![Page 18: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/18.jpg)
Phase IIb
An optimizer filters the best options and returns them to the client
Modeclientclientoptionsserver optionsModeoptimizerfeasibleoptionsserveroperation space
![Page 19: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/19.jpg)
Phase III
At top level the operation point is chosen and a contract is established. The server probably needs to set its own contracts accordingly.
Operation pointModeserveroperation spaceModeclient
![Page 20: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/20.jpg)
Accuracy Curves
To fit graphics rendering to ARC, each node in scene graph is seen as a 'client' in ARC. Each is assigned an accuracy curve
RA• required resources as function of accuracy target
• monotonically increasing.
R->#polygons
![Page 21: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/21.jpg)
Measurement of geometric distortion d as function of number of polygons n
d~ C/n.
Accuracy a =1/d
Resource usage r = K a + R0
-> piecewise linear function
cowhousebunnyknotbuddha
100 1k 10k 100k
0.1m
1m
0.01
Number of polygons
![Page 22: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/22.jpg)
Propagating accuracy curves
Leaf nodes: accuracy curve from (1) mathematical model or (2) measurements
Other nodes: propagate curve upwards through scenegraphLODChild 1Child 2ARARAR
![Page 23: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/23.jpg)
Res
ourc
e U
sage
LoD node behaviour
AccuracyLOD curveCurve ofchild 2Curve of child 1
LODChild 1Child 2ARARAR
![Page 24: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/24.jpg)
Different end points
curve 1
curve 2
Accuracy
Reso
urc
e u
sag
e
incorrectextension
incorrect minimum
20 50
![Page 25: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/25.jpg)
Complexity
Root client/node in scene graph: Target Accuracy -> Resources requiredIdem R -> A (so not NP-complete)
RAmax.Rmax.Atarget A
![Page 26: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/26.jpg)
Optimizing curve updates
Upto now: screenspace error=visual accuracyRefresh required if user moves
Optimizations:
(1) Determine range where a curve is ‘accurate enough’ as long as viewer is within the range.
(2) Visual accuracy is derived from geometric distortion - which is viewpoint independent
Relative acc = geometric accuracy object radius
![Page 27: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/27.jpg)
Using relative accuracy curves
Slight changes in algorithms:
• grouping -> 'object' diameter changes.
• Conversion to visual accuracy needed. Group's bbox is much closer to viewer than the individual bboxes -> convert to visual accuracy at reasonable distance.
![Page 28: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/28.jpg)
VRML integration
Accuracy curves & simplified objects: valid only in part of space. Efficient checking with ConicRange.
d
![Page 29: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/29.jpg)
Imposter nodes
Replace children with image. Automatic refresh & calc of accuracy curve
SimpleImposter{ MFNode children [ ] SFVec2f size 2 2 SFBool autosize true SFVec3f center 0 0 0 SFBool autocenter true}
![Page 30: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/30.jpg)
LOD
LOD picks valid level requiring least resources
LODSimpleImposterrange 10 1000MeshedImposterCylinder {}
![Page 31: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/31.jpg)
Statue on the campus
-Prototype implementation of all previous
-Very complex, implementation was simplified at several places (caching, prediction, etc)
![Page 32: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/32.jpg)
Statue application
Tracker was not yet working -> tracker stubvideo position & orientation
renderingfrontendpositionlookupVGAmixer
finalvideo file
Statueapplication
Statue VRML filebackbonemobile systemrendering
compilerSimplification
machinestracker ‘stub’Offlineanalysis
![Page 33: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/33.jpg)
DEMO VIDEO
![Page 34: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/34.jpg)
![Page 35: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/35.jpg)
![Page 36: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/36.jpg)
Cactus project
Context Aware Communication, Terminal and User
![Page 37: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/37.jpg)
Overview
Scenario sketchCactus structure, manpowerFirst ideas for architecture
![Page 38: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/38.jpg)
Cactus Scenario: cultural outing
![Page 39: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/39.jpg)
Current electronic secretary (PCA)
Existing PCA's:• enhancing comm between customer and business representative• 'Unified Messaging': voice,email,fax• often also agenda and conference booking• Menu-like voice interface
KPN Eileen• Additional: news, weather, teletekst, tv programs, travel info, etc• Human operator
![Page 40: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/40.jpg)
Cactus electronic secretary
Adding context sensitivityTracking the user and estimating user plansPro-active secretary
![Page 41: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/41.jpg)
Project structure
1 of 13 projects of Freeband Kennisimpuls project (www . freeband.nl)(EZ and OCW)
Cactus consists of 2 phases
2002-2004: Cactus Impulse
2004-2006: Cactus
![Page 42: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/42.jpg)
Cactus Impulse structure
Project leader: R.L. Lagendijk
UseT TermiNet
me
Personal supervisorF.W. JansenComputer Graphics &CAD
![Page 43: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/43.jpg)
UseT (User & Terminal)
Three subprojects:
Trust, Consistency and Interaction (1 Ph.D.)
User-context Analysis, Modeling and Sensing
(1 Ph.D.)
i-DEA Proof-of-Concept (me + 1 Ph.D.)
![Page 44: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/44.jpg)
Questions UseT.1 "Trust"
Will the user see the system as consistent and trust it?
Which cognitive and user interface factors are most important wrt trust and consistency?
eg, when should system ask and not ask? how should information and questions be presented?
Wizard of Oz studies to test our systemand ideas
![Page 45: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/45.jpg)
Questions UseT.2 "Sensing"
1. Which context- and personalization parameters are relevant for Eileen?
2. How can these parameters be sensed?
3. How can this info best be stored?
![Page 46: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/46.jpg)
Questions T.3 "Proof of Concept"
1. What is proper architecture for UseT System?
2. What reasoning system is appropriate?
3. What filtering of information based on user preferences and context is possible?
![Page 47: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/47.jpg)
Knowledge required
• Knowledge of applications
• Knowledge of user
• Knowledge on inferring user plans, goals etc
• Action planning system
• Knowledge about relatives of user
Etc..
![Page 48: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/48.jpg)
Architecture sketchuser
others9292 info
shopsvoice mail
etc
inputselectparsersemanticssecretary
applic.historylogplans, themesapplicknowl.knowl. about user
i-DeaOCRemailapplic.
useremaili-Deaemail
agendaapplic.
useragendaText to Speechstandard PCOS messages
Primitives-hooks
TextgrabberPhoneinterfaceuser planinferenceactionplanner
voicerecorder
voiceplaybackrecorded audio
![Page 49: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/49.jpg)
(Minsky)
Easy
Few anyNUMBERS OF CAUSES
Using magnitudes helpes makingcomparisons by hiding
Find betterrepresentation!
Intractable
ConnectionistNeural Netw orkFuzzy Lo gic
Analogy-basedReasoning
Case-basedReasoning
Societyof Mind
ClassicalAl
TraditionalComputing
Or dinar yQualitativ eReasoning
SymbolicLo gicReasoning
Linear ,Statistical
Scaleof
Effect
Large
Small
Figur e 1. Causal-diversity matrix
Appropriate Reasoning system
In our case, subtile differences have large effects, but not so many causes -> classical AI
![Page 50: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/50.jpg)
Proposal: logical goal- and plan-inference
In style of Wilensky's Plan Application Mechanism
![Page 51: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/51.jpg)
Plan/goal graph required for this (figure)Task analysis is good idea anywayLearning opportunities for system
![Page 52: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/52.jpg)
Multimodality
Use speech semantic structures to represent other modalities as well (pen input, pointing, scrolling, etc).
Advantages: uniform representation, easy modality crossing, late modality binding, language independent application, …
Problem: stills & video images
![Page 53: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/53.jpg)
Open questions when I left Cactus..
• Which subset of requirements do we start with?
• How to get there without doing an impossible amount of programming?
• Video modality not even theoretically clear how to handle in plan-based approach
![Page 54: Mobile AR and Electronic Secretary Wouter Pasman.](https://reader034.fdocuments.in/reader034/viewer/2022051215/56649eab5503460f94bb0f75/html5/thumbnails/54.jpg)