Krzysztof Templin 1,2 Piotr Didyk 2 Tobias Ritschel 3 Elmar Eisemann 3 Karol Myszkowski 2 Hans-Peter...
-
Upload
gyles-shields -
Category
Documents
-
view
214 -
download
0
Transcript of Krzysztof Templin 1,2 Piotr Didyk 2 Tobias Ritschel 3 Elmar Eisemann 3 Karol Myszkowski 2 Hans-Peter...
![Page 1: Krzysztof Templin 1,2 Piotr Didyk 2 Tobias Ritschel 3 Elmar Eisemann 3 Karol Myszkowski 2 Hans-Peter Seidel 2 Apparent Resolution Enhancement for Animations.](https://reader036.fdocuments.in/reader036/viewer/2022062717/56649e4b5503460f94b3f150/html5/thumbnails/1.jpg)
Krzysztof Templin1,2 Piotr Didyk2 Tobias Ritschel3
Elmar Eisemann3 Karol Myszkowski2 Hans-Peter Seidel2
Apparent Resolution Enhancement for Animations
1 University of Wrocław, Poland 2 MPI Informatik, Germany 3 Télécom ParisTech, France
![Page 2: Krzysztof Templin 1,2 Piotr Didyk 2 Tobias Ritschel 3 Elmar Eisemann 3 Karol Myszkowski 2 Hans-Peter Seidel 2 Apparent Resolution Enhancement for Animations.](https://reader036.fdocuments.in/reader036/viewer/2022062717/56649e4b5503460f94b3f150/html5/thumbnails/2.jpg)
Motivation
easily ~50 MPix ~2-8 MPix 1px → ~9 receptors
![Page 3: Krzysztof Templin 1,2 Piotr Didyk 2 Tobias Ritschel 3 Elmar Eisemann 3 Karol Myszkowski 2 Hans-Peter Seidel 2 Apparent Resolution Enhancement for Animations.](https://reader036.fdocuments.in/reader036/viewer/2022062717/56649e4b5503460f94b3f150/html5/thumbnails/3.jpg)
Standard methods• Cropping
• Downsampling
![Page 4: Krzysztof Templin 1,2 Piotr Didyk 2 Tobias Ritschel 3 Elmar Eisemann 3 Karol Myszkowski 2 Hans-Peter Seidel 2 Apparent Resolution Enhancement for Animations.](https://reader036.fdocuments.in/reader036/viewer/2022062717/56649e4b5503460f94b3f150/html5/thumbnails/4.jpg)
Decomposition into subframes
high-resolutionimage
low-resolutionsubframes
decompose
perceived high-resolutionimage
integrate
![Page 5: Krzysztof Templin 1,2 Piotr Didyk 2 Tobias Ritschel 3 Elmar Eisemann 3 Karol Myszkowski 2 Hans-Peter Seidel 2 Apparent Resolution Enhancement for Animations.](https://reader036.fdocuments.in/reader036/viewer/2022062717/56649e4b5503460f94b3f150/html5/thumbnails/5.jpg)
Related work“Display Supersampling”[Damera-Venkata and Chang 2009]
multiple projectors, one subframe each
“Wobulation: Doubling the Addressed Resolution of Projection Displays”[Allen and Ulichney 2005]
single projector, two subframes, subpixel shift
“Apparent Display Resolution Enhancement for Moving Images”[Didyk et al. 2010]
multiple subframes moving over 120Hz LCD display
![Page 6: Krzysztof Templin 1,2 Piotr Didyk 2 Tobias Ritschel 3 Elmar Eisemann 3 Karol Myszkowski 2 Hans-Peter Seidel 2 Apparent Resolution Enhancement for Animations.](https://reader036.fdocuments.in/reader036/viewer/2022062717/56649e4b5503460f94b3f150/html5/thumbnails/6.jpg)
Didyk et al.
time
![Page 7: Krzysztof Templin 1,2 Piotr Didyk 2 Tobias Ritschel 3 Elmar Eisemann 3 Karol Myszkowski 2 Hans-Peter Seidel 2 Apparent Resolution Enhancement for Animations.](https://reader036.fdocuments.in/reader036/viewer/2022062717/56649e4b5503460f94b3f150/html5/thumbnails/7.jpg)
Didyk et. al
![Page 8: Krzysztof Templin 1,2 Piotr Didyk 2 Tobias Ritschel 3 Elmar Eisemann 3 Karol Myszkowski 2 Hans-Peter Seidel 2 Apparent Resolution Enhancement for Animations.](https://reader036.fdocuments.in/reader036/viewer/2022062717/56649e4b5503460f94b3f150/html5/thumbnails/8.jpg)
Didyk et. al
![Page 9: Krzysztof Templin 1,2 Piotr Didyk 2 Tobias Ritschel 3 Elmar Eisemann 3 Karol Myszkowski 2 Hans-Peter Seidel 2 Apparent Resolution Enhancement for Animations.](https://reader036.fdocuments.in/reader036/viewer/2022062717/56649e4b5503460f94b3f150/html5/thumbnails/9.jpg)
pixel 1
pixel 2
frame 1 frame 3frame 2
pixel 1
pixel 2
Temporal domain
![Page 10: Krzysztof Templin 1,2 Piotr Didyk 2 Tobias Ritschel 3 Elmar Eisemann 3 Karol Myszkowski 2 Hans-Peter Seidel 2 Apparent Resolution Enhancement for Animations.](https://reader036.fdocuments.in/reader036/viewer/2022062717/56649e4b5503460f94b3f150/html5/thumbnails/10.jpg)
BC
A
A
B
C
pixel 1 pixel 2
receptor
frame 1 frame 3frame 2 temporal integration
Temporal domain – static case
![Page 11: Krzysztof Templin 1,2 Piotr Didyk 2 Tobias Ritschel 3 Elmar Eisemann 3 Karol Myszkowski 2 Hans-Peter Seidel 2 Apparent Resolution Enhancement for Animations.](https://reader036.fdocuments.in/reader036/viewer/2022062717/56649e4b5503460f94b3f150/html5/thumbnails/11.jpg)
A
B
C
pixel 1 pixel 2
frame 1 frame 3frame 2 temporal integration
Temporal domain – dynamic case
receptor
B
A
C
![Page 12: Krzysztof Templin 1,2 Piotr Didyk 2 Tobias Ritschel 3 Elmar Eisemann 3 Karol Myszkowski 2 Hans-Peter Seidel 2 Apparent Resolution Enhancement for Animations.](https://reader036.fdocuments.in/reader036/viewer/2022062717/56649e4b5503460f94b3f150/html5/thumbnails/12.jpg)
![Page 13: Krzysztof Templin 1,2 Piotr Didyk 2 Tobias Ritschel 3 Elmar Eisemann 3 Karol Myszkowski 2 Hans-Peter Seidel 2 Apparent Resolution Enhancement for Animations.](https://reader036.fdocuments.in/reader036/viewer/2022062717/56649e4b5503460f94b3f150/html5/thumbnails/13.jpg)
receptorReceptor signal:
– segment– pixel in segment i– intensity of pixel x in segment i
– weights proportional to the length of the segment
Temporal integration model
![Page 14: Krzysztof Templin 1,2 Piotr Didyk 2 Tobias Ritschel 3 Elmar Eisemann 3 Karol Myszkowski 2 Hans-Peter Seidel 2 Apparent Resolution Enhancement for Animations.](https://reader036.fdocuments.in/reader036/viewer/2022062717/56649e4b5503460f94b3f150/html5/thumbnails/14.jpg)
• Receptors at grid points.• Perfect tracking.
Receptor layout
![Page 15: Krzysztof Templin 1,2 Piotr Didyk 2 Tobias Ritschel 3 Elmar Eisemann 3 Karol Myszkowski 2 Hans-Peter Seidel 2 Apparent Resolution Enhancement for Animations.](https://reader036.fdocuments.in/reader036/viewer/2022062717/56649e4b5503460f94b3f150/html5/thumbnails/15.jpg)
prediction forone receptor
Prediction in equations
subframes retina image
integrationmodel
![Page 16: Krzysztof Templin 1,2 Piotr Didyk 2 Tobias Ritschel 3 Elmar Eisemann 3 Karol Myszkowski 2 Hans-Peter Seidel 2 Apparent Resolution Enhancement for Animations.](https://reader036.fdocuments.in/reader036/viewer/2022062717/56649e4b5503460f94b3f150/html5/thumbnails/16.jpg)
![Page 17: Krzysztof Templin 1,2 Piotr Didyk 2 Tobias Ritschel 3 Elmar Eisemann 3 Karol Myszkowski 2 Hans-Peter Seidel 2 Apparent Resolution Enhancement for Animations.](https://reader036.fdocuments.in/reader036/viewer/2022062717/56649e4b5503460f94b3f150/html5/thumbnails/17.jpg)
integrationmodel
Optimization problem
subframes high-resolution image
![Page 18: Krzysztof Templin 1,2 Piotr Didyk 2 Tobias Ritschel 3 Elmar Eisemann 3 Karol Myszkowski 2 Hans-Peter Seidel 2 Apparent Resolution Enhancement for Animations.](https://reader036.fdocuments.in/reader036/viewer/2022062717/56649e4b5503460f94b3f150/html5/thumbnails/18.jpg)
![Page 19: Krzysztof Templin 1,2 Piotr Didyk 2 Tobias Ritschel 3 Elmar Eisemann 3 Karol Myszkowski 2 Hans-Peter Seidel 2 Apparent Resolution Enhancement for Animations.](https://reader036.fdocuments.in/reader036/viewer/2022062717/56649e4b5503460f94b3f150/html5/thumbnails/19.jpg)
Panning (integer motion)
1 2 3 1’ 2’ 3’
![Page 20: Krzysztof Templin 1,2 Piotr Didyk 2 Tobias Ritschel 3 Elmar Eisemann 3 Karol Myszkowski 2 Hans-Peter Seidel 2 Apparent Resolution Enhancement for Animations.](https://reader036.fdocuments.in/reader036/viewer/2022062717/56649e4b5503460f94b3f150/html5/thumbnails/20.jpg)
Critical Flicker Frequency
Critical Flicker Frequency – Hecht and Smith’s data fromBrown J.L. Flicker and Intermittent Simulation
10 Hz
20 Hz
30 Hz
40 Hz
50 Hz
60 Hz
-3 -1 1 3 -3
Temporal contrastFr
eque
ncy
Three-frame cycleon 120 Hz display
40 Hz signal
Fusion frequency depends on:• Temporal contrast• Spatial extent
19 deg
1 deg
0.3 deg
![Page 21: Krzysztof Templin 1,2 Piotr Didyk 2 Tobias Ritschel 3 Elmar Eisemann 3 Karol Myszkowski 2 Hans-Peter Seidel 2 Apparent Resolution Enhancement for Animations.](https://reader036.fdocuments.in/reader036/viewer/2022062717/56649e4b5503460f94b3f150/html5/thumbnails/21.jpg)
Non-integer motion
1 2 3 4 5 6
![Page 22: Krzysztof Templin 1,2 Piotr Didyk 2 Tobias Ritschel 3 Elmar Eisemann 3 Karol Myszkowski 2 Hans-Peter Seidel 2 Apparent Resolution Enhancement for Animations.](https://reader036.fdocuments.in/reader036/viewer/2022062717/56649e4b5503460f94b3f150/html5/thumbnails/22.jpg)
1 2 3 4 5 6
Non-integer motion
![Page 23: Krzysztof Templin 1,2 Piotr Didyk 2 Tobias Ritschel 3 Elmar Eisemann 3 Karol Myszkowski 2 Hans-Peter Seidel 2 Apparent Resolution Enhancement for Animations.](https://reader036.fdocuments.in/reader036/viewer/2022062717/56649e4b5503460f94b3f150/html5/thumbnails/23.jpg)
Non-integer motion
?
≈
1 2 3 4 5 6
![Page 24: Krzysztof Templin 1,2 Piotr Didyk 2 Tobias Ritschel 3 Elmar Eisemann 3 Karol Myszkowski 2 Hans-Peter Seidel 2 Apparent Resolution Enhancement for Animations.](https://reader036.fdocuments.in/reader036/viewer/2022062717/56649e4b5503460f94b3f150/html5/thumbnails/24.jpg)
General animations• Motion already present
– no need to move.• Eye follows the motion
of the corresponding detail.• Local optimization, in space
and time.
![Page 25: Krzysztof Templin 1,2 Piotr Didyk 2 Tobias Ritschel 3 Elmar Eisemann 3 Karol Myszkowski 2 Hans-Peter Seidel 2 Apparent Resolution Enhancement for Animations.](https://reader036.fdocuments.in/reader036/viewer/2022062717/56649e4b5503460f94b3f150/html5/thumbnails/25.jpg)
Receptors pathsProblem:
too sparse
non-uniform distribution
Solution: we reintroduce receptors
![Page 26: Krzysztof Templin 1,2 Piotr Didyk 2 Tobias Ritschel 3 Elmar Eisemann 3 Karol Myszkowski 2 Hans-Peter Seidel 2 Apparent Resolution Enhancement for Animations.](https://reader036.fdocuments.in/reader036/viewer/2022062717/56649e4b5503460f94b3f150/html5/thumbnails/26.jpg)
Receptors paths
Solution: we reintroduce receptors
![Page 27: Krzysztof Templin 1,2 Piotr Didyk 2 Tobias Ritschel 3 Elmar Eisemann 3 Karol Myszkowski 2 Hans-Peter Seidel 2 Apparent Resolution Enhancement for Animations.](https://reader036.fdocuments.in/reader036/viewer/2022062717/56649e4b5503460f94b3f150/html5/thumbnails/27.jpg)
Optimization
retina image
current solution
original
subtract
error
integrate
project backimproved solution
iterate
rows / s = 120 × resolution × lifetime
optimal subframes
![Page 28: Krzysztof Templin 1,2 Piotr Didyk 2 Tobias Ritschel 3 Elmar Eisemann 3 Karol Myszkowski 2 Hans-Peter Seidel 2 Apparent Resolution Enhancement for Animations.](https://reader036.fdocuments.in/reader036/viewer/2022062717/56649e4b5503460f94b3f150/html5/thumbnails/28.jpg)
GPU implementationsimple fragment
shader
line drawing with alpha blending
![Page 29: Krzysztof Templin 1,2 Piotr Didyk 2 Tobias Ritschel 3 Elmar Eisemann 3 Karol Myszkowski 2 Hans-Peter Seidel 2 Apparent Resolution Enhancement for Animations.](https://reader036.fdocuments.in/reader036/viewer/2022062717/56649e4b5503460f94b3f150/html5/thumbnails/29.jpg)
Lanczos filtering• Standard approach: radius 6.• Smaller kernels leave aliasing in frames.• Can integrate, similarly to optimized solution.• We compare to radius 3, 4, 5 and 6.• Similar to [Basu and Baudisch 2009].• No perfect solution [Mitchell and Netravali 1988].
![Page 30: Krzysztof Templin 1,2 Piotr Didyk 2 Tobias Ritschel 3 Elmar Eisemann 3 Karol Myszkowski 2 Hans-Peter Seidel 2 Apparent Resolution Enhancement for Animations.](https://reader036.fdocuments.in/reader036/viewer/2022062717/56649e4b5503460f94b3f150/html5/thumbnails/30.jpg)
Results (general animations)• More detailed than Lanczos 6.• Details similar to Lanczos 4, but less aliasing.
![Page 31: Krzysztof Templin 1,2 Piotr Didyk 2 Tobias Ritschel 3 Elmar Eisemann 3 Karol Myszkowski 2 Hans-Peter Seidel 2 Apparent Resolution Enhancement for Animations.](https://reader036.fdocuments.in/reader036/viewer/2022062717/56649e4b5503460f94b3f150/html5/thumbnails/31.jpg)
Perceptual study• Number of participants: 14.• Two-step procedure:
1. Lanczos kernel adjustment. 2. Lanczos vs. ours comparison.
• Question asked: which reproduces the original better.• Study showed, that our method gives the best results:
Method Preference
Lanczos 3 1%
Lanczos 4 3%
Lanczos 5 17%
Lanczos 6 19%
Our 60%
![Page 32: Krzysztof Templin 1,2 Piotr Didyk 2 Tobias Ritschel 3 Elmar Eisemann 3 Karol Myszkowski 2 Hans-Peter Seidel 2 Apparent Resolution Enhancement for Animations.](https://reader036.fdocuments.in/reader036/viewer/2022062717/56649e4b5503460f94b3f150/html5/thumbnails/32.jpg)
Velocity vs. Quality• Subframes integrate giving impression of increased resolution.• Often fusion is not complete – some artifacts visible.• But not always.
![Page 33: Krzysztof Templin 1,2 Piotr Didyk 2 Tobias Ritschel 3 Elmar Eisemann 3 Karol Myszkowski 2 Hans-Peter Seidel 2 Apparent Resolution Enhancement for Animations.](https://reader036.fdocuments.in/reader036/viewer/2022062717/56649e4b5503460f94b3f150/html5/thumbnails/33.jpg)
Conclusion• We generalized previous results, and showed how to enhance
depiction of details in arbitrary animations.• Compared our algorithm to other filtering methods in a
perceptual study.• Designed an efficent GPU implementation.
![Page 34: Krzysztof Templin 1,2 Piotr Didyk 2 Tobias Ritschel 3 Elmar Eisemann 3 Karol Myszkowski 2 Hans-Peter Seidel 2 Apparent Resolution Enhancement for Animations.](https://reader036.fdocuments.in/reader036/viewer/2022062717/56649e4b5503460f94b3f150/html5/thumbnails/34.jpg)
Future work• Higher refresh rates.• Flicker reduction methods.• Faster implementation.• Eye-tracking.• Non-uniform sampling.• Other media.
![Page 35: Krzysztof Templin 1,2 Piotr Didyk 2 Tobias Ritschel 3 Elmar Eisemann 3 Karol Myszkowski 2 Hans-Peter Seidel 2 Apparent Resolution Enhancement for Animations.](https://reader036.fdocuments.in/reader036/viewer/2022062717/56649e4b5503460f94b3f150/html5/thumbnails/35.jpg)
Thank you!
Apparent Resolution Enhancement for AnimationsKrzysztof Templin Piotr Didyk Tobias Ritschel Elmar Eisemann Karol Myszkowski Hans-Peter Seidel
http://www.mpi-inf.mpg.de/resources/ResolutionEnhancement/Animations/