Time lapse and stabilising a sequence of images (GStreamer Conference 2011)
-
Upload
igalia -
Category
Technology
-
view
534 -
download
0
description
Transcript of Time lapse and stabilising a sequence of images (GStreamer Conference 2011)
![Page 1: Time lapse and stabilising a sequence of images (GStreamer Conference 2011)](https://reader034.fdocuments.in/reader034/viewer/2022052619/55653f74d8b42a313f8b52ee/html5/thumbnails/1.jpg)
Image Stabilisation
Guillaume EmontIgalia
![Page 2: Time lapse and stabilising a sequence of images (GStreamer Conference 2011)](https://reader034.fdocuments.in/reader034/viewer/2022052619/55653f74d8b42a313f8b52ee/html5/thumbnails/2.jpg)
Image StabilisationHoliday pictures
![Page 3: Time lapse and stabilising a sequence of images (GStreamer Conference 2011)](https://reader034.fdocuments.in/reader034/viewer/2022052619/55653f74d8b42a313f8b52ee/html5/thumbnails/3.jpg)
that picture is not by me(it is © 2011 Ugo Riboni)
![Page 4: Time lapse and stabilising a sequence of images (GStreamer Conference 2011)](https://reader034.fdocuments.in/reader034/viewer/2022052619/55653f74d8b42a313f8b52ee/html5/thumbnails/4.jpg)
...but I was there!
![Page 5: Time lapse and stabilising a sequence of images (GStreamer Conference 2011)](https://reader034.fdocuments.in/reader034/viewer/2022052619/55653f74d8b42a313f8b52ee/html5/thumbnails/5.jpg)
with a plan!
![Page 6: Time lapse and stabilising a sequence of images (GStreamer Conference 2011)](https://reader034.fdocuments.in/reader034/viewer/2022052619/55653f74d8b42a313f8b52ee/html5/thumbnails/6.jpg)
![Page 7: Time lapse and stabilising a sequence of images (GStreamer Conference 2011)](https://reader034.fdocuments.in/reader034/viewer/2022052619/55653f74d8b42a313f8b52ee/html5/thumbnails/7.jpg)
What the balloon sees
![Page 8: Time lapse and stabilising a sequence of images (GStreamer Conference 2011)](https://reader034.fdocuments.in/reader034/viewer/2022052619/55653f74d8b42a313f8b52ee/html5/thumbnails/8.jpg)
Let's go higher
![Page 9: Time lapse and stabilising a sequence of images (GStreamer Conference 2011)](https://reader034.fdocuments.in/reader034/viewer/2022052619/55653f74d8b42a313f8b52ee/html5/thumbnails/9.jpg)
Higher!
![Page 10: Time lapse and stabilising a sequence of images (GStreamer Conference 2011)](https://reader034.fdocuments.in/reader034/viewer/2022052619/55653f74d8b42a313f8b52ee/html5/thumbnails/10.jpg)
HIGHER!
![Page 11: Time lapse and stabilising a sequence of images (GStreamer Conference 2011)](https://reader034.fdocuments.in/reader034/viewer/2022052619/55653f74d8b42a313f8b52ee/html5/thumbnails/11.jpg)
I said: HIIIGHEEEER!
![Page 12: Time lapse and stabilising a sequence of images (GStreamer Conference 2011)](https://reader034.fdocuments.in/reader034/viewer/2022052619/55653f74d8b42a313f8b52ee/html5/thumbnails/12.jpg)
Ok, that will do
![Page 13: Time lapse and stabilising a sequence of images (GStreamer Conference 2011)](https://reader034.fdocuments.in/reader034/viewer/2022052619/55653f74d8b42a313f8b52ee/html5/thumbnails/13.jpg)
We wanted to take pictures for 7 days
![Page 14: Time lapse and stabilising a sequence of images (GStreamer Conference 2011)](https://reader034.fdocuments.in/reader034/viewer/2022052619/55653f74d8b42a313f8b52ee/html5/thumbnails/14.jpg)
The balloon agreed to 4 hours and a half
![Page 15: Time lapse and stabilising a sequence of images (GStreamer Conference 2011)](https://reader034.fdocuments.in/reader034/viewer/2022052619/55653f74d8b42a313f8b52ee/html5/thumbnails/15.jpg)
There was an intimate ceremonywith family and friends
![Page 16: Time lapse and stabilising a sequence of images (GStreamer Conference 2011)](https://reader034.fdocuments.in/reader034/viewer/2022052619/55653f74d8b42a313f8b52ee/html5/thumbnails/16.jpg)
Nice pics! ...now what?
![Page 17: Time lapse and stabilising a sequence of images (GStreamer Conference 2011)](https://reader034.fdocuments.in/reader034/viewer/2022052619/55653f74d8b42a313f8b52ee/html5/thumbnails/17.jpg)
Time lapse!
![Page 18: Time lapse and stabilising a sequence of images (GStreamer Conference 2011)](https://reader034.fdocuments.in/reader034/viewer/2022052619/55653f74d8b42a313f8b52ee/html5/thumbnails/18.jpg)
-- [timelapse-theory.png] [fit] [text-align=center] <span font="20">(image from wikipedia under GFDL)</span> -- [timelapse-theory.png] [fit] [text-align=center] Crazier: one frame per minute!
![Page 19: Time lapse and stabilising a sequence of images (GStreamer Conference 2011)](https://reader034.fdocuments.in/reader034/viewer/2022052619/55653f74d8b42a313f8b52ee/html5/thumbnails/19.jpg)
Removing the training wheels:No tripod.
(luckily, we have OpenCV)
![Page 20: Time lapse and stabilising a sequence of images (GStreamer Conference 2011)](https://reader034.fdocuments.in/reader034/viewer/2022052619/55653f74d8b42a313f8b52ee/html5/thumbnails/20.jpg)
Stabilisation in post-processing:
1. detect the movement (optical flow)
![Page 21: Time lapse and stabilising a sequence of images (GStreamer Conference 2011)](https://reader034.fdocuments.in/reader034/viewer/2022052619/55653f74d8b42a313f8b52ee/html5/thumbnails/21.jpg)
Optical flow: position changes of features
![Page 22: Time lapse and stabilising a sequence of images (GStreamer Conference 2011)](https://reader034.fdocuments.in/reader034/viewer/2022052619/55653f74d8b42a313f8b52ee/html5/thumbnails/22.jpg)
Stabilisation in post-processing:
2. compensate
![Page 23: Time lapse and stabilising a sequence of images (GStreamer Conference 2011)](https://reader034.fdocuments.in/reader034/viewer/2022052619/55653f74d8b42a313f8b52ee/html5/thumbnails/23.jpg)
Compensation:
OpenCV provides nice tools(but we won't talk about them here)
![Page 24: Time lapse and stabilising a sequence of images (GStreamer Conference 2011)](https://reader034.fdocuments.in/reader034/viewer/2022052619/55653f74d8b42a313f8b52ee/html5/thumbnails/24.jpg)
Optical flow: a few algorithms
![Page 25: Time lapse and stabilising a sequence of images (GStreamer Conference 2011)](https://reader034.fdocuments.in/reader034/viewer/2022052619/55653f74d8b42a313f8b52ee/html5/thumbnails/25.jpg)
Horn-Schunck
![Page 26: Time lapse and stabilising a sequence of images (GStreamer Conference 2011)](https://reader034.fdocuments.in/reader034/viewer/2022052619/55653f74d8b42a313f8b52ee/html5/thumbnails/26.jpg)
Lucas-Kanade
gst-launch-0.10 filesrc location=shaky-hippo.ogg ! decodebin ! ffmpegcolorspace ! opticalflowcorrector algorithm=1 ! ffmpegcolorspace ! xvimagesink
![Page 27: Time lapse and stabilising a sequence of images (GStreamer Conference 2011)](https://reader034.fdocuments.in/reader034/viewer/2022052619/55653f74d8b42a313f8b52ee/html5/thumbnails/27.jpg)
Err, yeah but...What about the balloon pictures?
![Page 28: Time lapse and stabilising a sequence of images (GStreamer Conference 2011)](https://reader034.fdocuments.in/reader034/viewer/2022052619/55653f74d8b42a313f8b52ee/html5/thumbnails/28.jpg)
Horn-Schunck
![Page 29: Time lapse and stabilising a sequence of images (GStreamer Conference 2011)](https://reader034.fdocuments.in/reader034/viewer/2022052619/55653f74d8b42a313f8b52ee/html5/thumbnails/29.jpg)
Lucas-Kanade
![Page 30: Time lapse and stabilising a sequence of images (GStreamer Conference 2011)](https://reader034.fdocuments.in/reader034/viewer/2022052619/55653f74d8b42a313f8b52ee/html5/thumbnails/30.jpg)
SpeededUpRobustFeature
![Page 31: Time lapse and stabilising a sequence of images (GStreamer Conference 2011)](https://reader034.fdocuments.in/reader034/viewer/2022052619/55653f74d8b42a313f8b52ee/html5/thumbnails/31.jpg)
SURF
![Page 32: Time lapse and stabilising a sequence of images (GStreamer Conference 2011)](https://reader034.fdocuments.in/reader034/viewer/2022052619/55653f74d8b42a313f8b52ee/html5/thumbnails/32.jpg)
Woohoo!
![Page 33: Time lapse and stabilising a sequence of images (GStreamer Conference 2011)](https://reader034.fdocuments.in/reader034/viewer/2022052619/55653f74d8b42a313f8b52ee/html5/thumbnails/33.jpg)
Bonus track
![Page 34: Time lapse and stabilising a sequence of images (GStreamer Conference 2011)](https://reader034.fdocuments.in/reader034/viewer/2022052619/55653f74d8b42a313f8b52ee/html5/thumbnails/34.jpg)
TODO:Check out the optical flow work by David Jordan (used for slow motion)and try to apply it to stabilisation
![Page 35: Time lapse and stabilising a sequence of images (GStreamer Conference 2011)](https://reader034.fdocuments.in/reader034/viewer/2022052619/55653f74d8b42a313f8b52ee/html5/thumbnails/35.jpg)
TODO:better compensation (tesselation?)
![Page 36: Time lapse and stabilising a sequence of images (GStreamer Conference 2011)](https://reader034.fdocuments.in/reader034/viewer/2022052619/55653f74d8b42a313f8b52ee/html5/thumbnails/36.jpg)
TODO:try more algorithms (BRIEF and ORB)
![Page 37: Time lapse and stabilising a sequence of images (GStreamer Conference 2011)](https://reader034.fdocuments.in/reader034/viewer/2022052619/55653f74d8b42a313f8b52ee/html5/thumbnails/37.jpg)
TODO:automatic optimal choice of algorithm and parameters
![Page 38: Time lapse and stabilising a sequence of images (GStreamer Conference 2011)](https://reader034.fdocuments.in/reader034/viewer/2022052619/55653f74d8b42a313f8b52ee/html5/thumbnails/38.jpg)
TODO:reimplement in C and put in -bad?
![Page 39: Time lapse and stabilising a sequence of images (GStreamer Conference 2011)](https://reader034.fdocuments.in/reader034/viewer/2022052619/55653f74d8b42a313f8b52ee/html5/thumbnails/39.jpg)
TODO:for videos: movement prediction/averaging(follow the intended movement of the cameraman)
![Page 40: Time lapse and stabilising a sequence of images (GStreamer Conference 2011)](https://reader034.fdocuments.in/reader034/viewer/2022052619/55653f74d8b42a313f8b52ee/html5/thumbnails/40.jpg)
questions?
Image credits:training wheels: CC BY-NC-SA 2.0 Dale Basler http://www.flickr.com/photos/baslercast/surf: CC BY-NC-SA 2.0 Guilherme Meneghelli http://www.flickr.com/people/iberere/last one: CC BY-NC-SA 2.0 Martin Ibert http://www.flickr.com/photos/mar_ibert/
![Page 41: Time lapse and stabilising a sequence of images (GStreamer Conference 2011)](https://reader034.fdocuments.in/reader034/viewer/2022052619/55653f74d8b42a313f8b52ee/html5/thumbnails/41.jpg)
Thanks! http://www.gitorious.org/gststabilizerhttp://balloonfreaks.mooo.com/http://emont.org/blog/http://www.igalia.com/[email protected]