Raskar Emtech2010 Mar Final
-
Upload
cameraculture-mit-media-lab -
Category
Technology
-
view
1.632 -
download
0
Transcript of Raskar Emtech2010 Mar Final
Raskar, Camera Culture, MIT Media Lab
Camera Culture
Ramesh Raskar
Camera CultureMIT Media Lab http://raskar.info
http://cameraculture.info
Ramesh RaskarAssociate Professor
Future of Imaging
Film-like Digital Photography
Cameras Phones are EverywhereCameras Phones are Everywhere
Kush R. Varshney, “Jagannath Temple,” Puri, Orissa, India, Dec. 2007.
Wish List TodayWish List Today
• ConsumersConsumers– Super-human visionSuper-human vision– Microscope like resolutionMicroscope like resolution– High speed (burst, video, no blur)High speed (burst, video, no blur)– See inside the body (health)See inside the body (health)– Auto-triggerAuto-trigger– Battery lifeBattery life– Zero start or shutter delayZero start or shutter delay– Keep only ‘good’ picsKeep only ‘good’ pics– Find ‘relevant’ pics and better Find ‘relevant’ pics and better
archiving/accessarchiving/access
– Put photographer back in photo!Put photographer back in photo!
• CompaniesCompanies– CostCost– ResolutionResolution– Low-light sensitivity, HDRLow-light sensitivity, HDR– ((For cameraphoneFor cameraphone) Size and depth) Size and depth– Stereo and 3DStereo and 3D– Mecha-free zoom/focusMecha-free zoom/focus– Auto-tagging for sharingAuto-tagging for sharing– RecognitionRecognition
Can you look around a corner ?
Convert LCD into a big flat camera?
Where are the ‘camera’s?Where are the ‘camera’s?
Where are the ‘camera’s?Where are the ‘camera’s?
We focus on creating tools to better capture and share visual information via a
new class of imaging platforms
Ramesh RaskarRamesh Raskar http://raskar.info
What is the Media Lab?
A Graduate Program in the Media Arts & SciencesHouses ~150 students and 30 PIs
A Research Lab that spans across disciplines and academic/ industrial lines65 sponsor companiesSponsors get free, non-exclusive licenses for ML IP
Founded in 1985 by Nicholas Negroponte and Jerome Wiesner
Close Ties With Industry
05/03/23 R E S E A R C H S T R A T E G Y 18
Our 65 corporate sponsors include some of the most creative companies in the world
Smart Cities
What if cars could stack like shopping carts in cities?Ryan Chin and Bill Mitchell
ADAPTABILTIY
Hyper -
Adaptability
Bio-Mechatronics
Music/Mind/Health
People -Sense
Sociable Robots
Neuro-MediaHUMAN
ADAPTABILITY
Bio-Mechatronics
Neuro-Media
Sociable Robots
People -Sense
Music/Mind/Health
Raskar, Camera Culture, MIT Media Lab
Camera Culture
Ramesh Raskar
Ramesh RaskarAssociate Professor, MIT Media Lab http://raskar.info
New Emerging TechnologiesMedical Imaging
EntertainmentUser InterfacesIndustrial Vision
The impact of Next Billion CamerasMovie-making, news reporting
Social stability
Visual Social ComputingImage-based commerce
Future of Imaging
Ramesh Raskar, MIT Media Lab
Medical Imaging Today .. Medical Imaging Today ..
http://info.med.yale.edu/intmed/cardio/imaging/techniques/ct_imaging/http://info.med.yale.edu/intmed/cardio/imaging/techniques/ct_imaging/
Self Evaluation of Eye
What are the problems with ‘real’ photo in conveying information ?
Why do we hire artists to draw what can be photographed ?
Shadows
Clutter
Many Colors
Highlight Shape Edges
Mark moving parts
Basic colors
Cartoon CameraCartoon CameraUsing Depth EdgesUsing Depth Edges
Canny Intensity Edge Detection
Our Method
Photo Result
Convert LCD into a big flat camera?Beyond Multi-touch: 3D Gestures
Large Virtual Camera for 3D Interactive HCI and Video Conferencing
Matthew Hirsch, Henry HoltzmanDoug Lanman, Ramesh Raskar
Siggraph Asia 2009
BiDi Screen
Touch + Hover using Thin, Depth Sensing LCD Sensor
Sensing Depth from Array of Virtual Eyes in LCD
Raskar, Camera Culture, MIT Media Lab
Camera Culture
Ramesh Raskar
Understand the World•Identify/recognize Objects
•3D Awareness•Interact with embedded
information
Can you ‘see’ around a corner ?
Femto-Photography: Higher Dimensional Capture
FemtoFlash
UltraFast Detector
Computational OpticsSerious Sync
Application: Rescue and Planning
Bokode
Defocus blur of Bokode
Coding in Angle
Mohan, Woo, Smithwick, Hiura, Raskar [Siggraph 2009]
camera
sensor
circle of confusion circle of information
Bokode(angle)
Quote suggested by Kurt Akeley
Encoding in Angle, not space, time or wavelength
cell-phone cameraclose to the Bokode
(10,000+ bytes of data)
Raskar, Camera Culture, MIT Media Lab
Camera Culture
Ramesh Raskar
Understand the World•Identify/recognize Objects
•3D Awareness•Interact with embedded
information
Visual Computing
Forward (Synthesize)– Special Effects, Image Processing, Design
Inverse (Understand)– Human-like performance– Recognizing/finding objects, image search– Challenging !
Power ofthe
Processor
Power of
theNetwork
Powerof
People
1985 1995 2005 2015
Powershifts
CrowdsourcingCrowdsourcing
http://www.wired.com/wired/archive/14.06/crowds.html
Amazon Mechanical Turk: Steve Fossett search
ReCAPTCHA=OCR
Visual Social Computing
• Image-based commerceBy the people, for the people, of the people
• Next Trend in India ?Harnessing human power beyond language barriers
• Business Model ?Outsource → Image-based Crowdsource
Cameras in Developing CountriesCameras in Developing Countries
http://news.bbc.co.uk/2/hi/south_asia/7147796.stm
Community news program run by village women
Developing Countries: CAMForms• Paper forms with
barcodes• 83-bit 2D codes
(including seven bits of error correction)
Parikh (2005)
User Generated Content Visualization
• Google Map overlayed with geo-tagged photos
• Image-based Mashups
http://phototour.cs.washington.edu/
Photo Synth
Socio-Political Goals: B’TselemSocio-Political Goals: B’Tselem
From websiteof : Israeli Information Center for Human Rights in the Occupied Territories
Truth in ImagesTruth in Images
From Hany Farid
LA Times March’03
Raskar, Camera Culture, MIT Media Lab
Camera Culture
Ramesh Raskar
How will the next billion cameras change the social culture ?
How can we augment the camera to support best ‘image search’ ?
How will camera improve trust and social stability ?
How will movie-making, news reporting change ?
Next model for image-based commerce ?
Visual Social Computing
MIT Media Lab Workshops
Session before lunch (12:15 to 1pm):How to Invent?Health, Entertainment and User Interfaces
Session after lunch: (by Invitation Only)How to innovateBrainstorming
Raskar, Camera Culture, MIT Media Lab
Camera Culture
Ramesh Raskar
Ramesh RaskarAssociate Professor, MIT Media Lab http://raskar.info
New Emerging TechnologiesMedical Imaging
EntertainmentUser InterfacesIndustrial Vision
The impact of Next Billion CamerasMovie-making, news reporting
Social stability
Visual Social ComputingImage-based commerce
Future of Imaging
END Part 1
Start Part 2
Raskar, Camera Culture, MIT Media Lab
Camera Culture
Ramesh Raskar
Camera CultureMIT Media Lab http://raskar.info
http://cameraculture.info
Ramesh RaskarAssociate Professor
Ramesh Raskar, MIT Media Lab
After X, what is neXt
How to Invent?
Ramesh Raskar, MIT Media Lab
Ramesh Raskar, MIT Media Lab
Xd
X++
X X+Y
X
X
neXt
Ramesh Raskar, MIT Media Lab
Ramesh Raskar, MIT Media Lab
Simple Exercise .. Simple Exercise .. • Image CompressionImage Compression
– Save Bandwidth and storageSave Bandwidth and storage
What is neXt
Ramesh Raskar, MIT Media Lab
Strategy #1: XStrategy #1: Xdd
• Extend it to next dimension (or some other) dimensionExtend it to next dimension (or some other) dimension
– Flickr to YoutubeFlickr to Youtube
– Wikipedia to .. ?Wikipedia to .. ?
• Text, Audio (Speech), Image, Video .. Whats next ?Text, Audio (Speech), Image, Video .. Whats next ?– CD ..CD ..
• Images to infrared, sound, ultrasoundImages to infrared, sound, ultrasound• Macro scale to microscaleMacro scale to microscale
• Airbag for car to airbag for .. ?Airbag for car to airbag for .. ?
Ramesh Raskar, MIT Media Lab
Strategy #2: X+YStrategy #2: X+Y• Fusion of the dissimilarFusion of the dissimilar
– More dissimilar, more spectacular the outputMore dissimilar, more spectacular the output
• ExampleExample– Scientific imaging + PhotographyScientific imaging + Photography
• Coded apertureCoded aperture• TomographyTomography
Ramesh Raskar, MIT Media Lab
Prototype cameraPrototype camera
4000 4000 × 4000 pixels ÷ 292 × 292 lenses = 14 × 14 pixels per lens× 4000 pixels ÷ 292 × 292 lenses = 14 × 14 pixels per lens
Contax medium format camera Kodak 16-megapixel sensor
Adaptive Optics microlens array 125μ square-sided microlenses
Ramesh Raskar, MIT Media Lab
Example of digital refocusingExample of digital refocusing
Ramesh Raskar, MIT Media Lab
Imaging in Sciences: Imaging in Sciences: Computer TomographyComputer Tomography
• http://info.med.yale.edu/intmed/cardio/imaging/techniques/http://info.med.yale.edu/intmed/cardio/imaging/techniques/ct_imaging/ct_imaging/
Self Evaluation of Eye
Ramesh Raskar, MIT Media Lab
Strategy #3: X Strategy #3: X Do exactly the oppositeDo exactly the opposite
• Processing, Memory, BandwidthProcessing, Memory, Bandwidth– In Computing world, in any era, one of this is a bottleneckIn Computing world, in any era, one of this is a bottleneck– But overtime, they change. You can often take an older idea and do But overtime, they change. You can often take an older idea and do
exactly the opposite.exactly the opposite.– E.g. bandwidth is now considered virtually limitlessE.g. bandwidth is now considered virtually limitless
• Business Process Reengineering (BPR) Business Process Reengineering (BPR) – Michael Hammer, James Champy, 1990sMichael Hammer, James Champy, 1990s
• In imaging:In imaging:– SLR: Faster mirror flip or no mirror flipSLR: Faster mirror flip or no mirror flip
• Companies spent years improving mirror flip speedCompanies spent years improving mirror flip speed• Why not just remove it?Why not just remove it?
• More computationMore computation• Less lightLess light
Ramesh Raskar, MIT Media Lab
• Toll Free callsToll Free calls• Reverse AuctionReverse Auction
Power ofthe
Processor
Power of
theNetwork
Powerof
People
1985 1995 2005 2015
Powershifts
Larval Trematode WormLarval Trematode Worm
Ramesh Raskar, MIT Media Lab
Strategy #4: X Strategy #4: X • Given a Hammer ..Given a Hammer ..
– Find all the nailsFind all the nails– Sometimes even screws and boltsSometimes even screws and bolts
• Given a cool solution/technique/Opportunity Given a cool solution/technique/Opportunity – Find other problemsFind other problems– (Where to find them?)(Where to find them?)
• ExamplesExamples– Peltier effect: Peltier effect:
• Create a jacket that keeps you warm or coldCreate a jacket that keeps you warm or cold– Mobile phone opportunityMobile phone opportunity
Ramesh Raskar, MIT Media Lab
Strategy #5: X Strategy #5: X • Given a nail, Given a nail,
– Find all hammersFind all hammers– Sometimes even screwdrivers and pliers may Sometimes even screwdrivers and pliers may
workwork• Given a problem, Given a problem,
– Find other solutionsFind other solutions– (Where to find them?)(Where to find them?)
• ExamplesExamples– App store (Apple) .. Open platform for all devicesApp store (Apple) .. Open platform for all devices– ....
Ramesh Raskar, MIT Media Lab
Strategy #6: X++Strategy #6: X++• Pick your adjective ..Pick your adjective ..• Making it faster, better, cheaperMaking it faster, better, cheaper
neXt = adjective + XneXt = adjective + X
Ramesh Raskar, MIT Media Lab
X++ : Add your favorite adjectiveX++ : Add your favorite adjective• Context aware, Context aware, • AdaptiveAdaptive• (temporally) Coherent, (temporally) Coherent, • Hierarchical, Hierarchical, • ProgressiveProgressive• EfficientEfficient• ParallelizedParallelized• DistributedDistributed
• Good example: Image or video compression schemesGood example: Image or video compression schemes
Ramesh Raskar, MIT Media Lab
X++ : Add your favorite adjectiveX++ : Add your favorite adjective• Good example: Image or video compression schemesGood example: Image or video compression schemes• But X++ is a signBut X++ is a sign
– The field is maturing in terms of research but booming in business The field is maturing in terms of research but booming in business impactimpact
• KaizenKaizen– Small incremental changesSmall incremental changes– Japanese Management styles (6sigma, Kanban)Japanese Management styles (6sigma, Kanban)– Mainly to save money/time/resources. Not everyone can do it. GM, Mainly to save money/time/resources. Not everyone can do it. GM,
0.84 suggestions per employee vs Toyota 18. GM accepted 23%, 0.84 suggestions per employee vs Toyota 18. GM accepted 23%, Toyota 90%Toyota 90%
Ramesh Raskar, MIT Media Lab
Hexagon Corners for Different StagesHexagon Corners for Different Stages
[Bruce Tuckman, 1965][Bruce Tuckman, 1965]
• FormingForming• StormingStorming• NormingNorming• PerformingPerforming• AdjourningAdjourning
Ramesh Raskar, MIT Media Lab
PitfallsPitfalls• These six ways are only a start These six ways are only a start • They are a good mental exercise and will They are a good mental exercise and will
allow you to train as a researcherallow you to train as a researcher• Great for projectsGreat for projects• But But
– Maynot produce radically new ideasMaynot produce radically new ideas– Sometimes a danger of being labeled incrementalSometimes a danger of being labeled incremental– Could be into ‘public domain ideas’Could be into ‘public domain ideas’
Ramesh Raskar, MIT Media Lab
What are Bad ideas to pursueWhat are Bad ideas to pursue• X then Y (then Z)X then Y (then Z)
– X+Y is great with true fusion, fusion of dissimilar is bestX+Y is great with true fusion, fusion of dissimilar is best– But avoid a ‘pipeline’ systems, where the output of one is But avoid a ‘pipeline’ systems, where the output of one is
THEN channeled into the input of the next stage, and non THEN channeled into the input of the next stage, and non of the components are novelof the components are novel
• Follow the hype (too much competition) Follow the hype (too much competition) • Do because it can be done Do because it can be done
– (Why do we climb a mountain? because it is there! )(Why do we climb a mountain? because it is there! )– But only the first one gets a credit. But only the first one gets a credit. – May make you strong, and give you a sense of May make you strong, and give you a sense of
achievement but not a research project.achievement but not a research project.
Ramesh Raskar, MIT Media Lab
Xd
X++
X X+Y
X
X
neXt
Ramesh Raskar, MIT Media Lab
End Part 2
Raskar, Camera Culture, MIT Media Lab
Camera Culture
Ramesh Raskar
Camera CultureMIT Media Lab http://raskar.info
http://cameraculture.info
Raskar, Camera Culture, MIT Media Lab
Extra Slides