Creating and Exploring a Large Photorealistic Virtual Space INRIA / CSAIL / Adobe First IEEE...
-
Upload
katrina-hunter -
Category
Documents
-
view
213 -
download
0
Transcript of Creating and Exploring a Large Photorealistic Virtual Space INRIA / CSAIL / Adobe First IEEE...
Creating and Exploring a Large Photorealistic Virtual Space
INRIA / CSAIL / Adobe
First IEEE Workshop on Internet Vision, associated with CVPR 2008.
Outline• Introduction• Constructing the image space• Navigating the virtual 3D space• Limitations and Conclusion
Introduction
• We present a system for exploring large collections of photos in a virtual 3D space.
• Let users navigate within each theme using intuitive 3D controls that include move left/right, zoom and rotate.
• In a similar fashion we can create infinite zoom effects that resemble the ”Droste effect”.
Constructing the image space
•The image database– We have collected ~6 million images from Flickr
based on keyword and group searches• typical image size is 500x375 pixels• 720GB of disk space (jpeg compressed)
Image representation
Color layout
GIST [Oliva and Torralba’01]
Original image
Obtaining semantically coherent themesWe further break-up the collection into themes of semantically coherent scenes:
Train SVM-based classifiers from 1-2k training images [Oliva and Torralba, 2001]
Basic camera motions
Forward motion Camera rotation Camera pan
Starting from a single image, find a sequence of images to simulate a camera motion:
3. Find a match to fill the missing pixels
Scene matching with camera view transformations: Translation
1. Move camera
2. View from the virtual camera
4. Locally align images
5. Find a seam
6. Blend in the gradient domain
4. Stitched rotation
Scene matching with camera view transformations: Camera rotation
1. Rotate camera
2. View from the virtual camera
3. Find a match to fill-in the missing pixels
5. Display on a cylinder
Steps
• Collect images• Classify images into topic themes• Calculate the descriptors:
– GIST– RGB
• Build the graph• Find the path for given query image(s)
– Dijkstra algorithm• Alignment
– Gradient decent alignment• Composition
– Poisson blending
More “infinite” images – camera translation
Forward
Rotate (left/right)
Pan (left/right)
• Nodes represent Images
• Edges represent particular motions:
• Edge cost is given by the cost of the image match under the particular transformation
Image graph
Navigating the virtual 3D space
• Virtual space as an image graph
•Virtual image space laid out in 3D
Limitations and Conclusion
• The larger the database, the better the results.• Compositing two distinct images is always a challenge
and at times, the seam is quite visible.• This system can be used to create photorealistic visual
content for large online virtual 3D worlds like Second Life.
• Create infinite panoramas or use the image taxi to generate tailor-made tours in the virtual 3D space. These applications can find use in games, movies and other media creation processes.
Thank you!