Post on 06-Feb-2016
description
1
Window-based Approach For Fast Stereo Correspondence
Raj Kumar Gupta, Siu-Yeung ChoIET Computer Vision, 2013
2
Outline•Introduction•Related Work•Proposed Method•Experimental Results•Conclusion
3
Introduction•Using two correlation windows to improve the performance of the algorithm •3*3 and 9*9•Real-time suitability •more than 10 frame/s on CPU in case of 320 × 240-sized image pair with disparity value 16
4
Related Work•Local methods are usually base on correlation.•Area-based (NCC, SAD, SSD)•Feature-based: rely on feature extraction and match local cues (BF, GF)
•Bigger window size, more information, more blurred.
5
Outline•Introduction•Related Work•Proposed Method•Experimental Results•Conclusion
6
Flow Chart
7
Flow Chart
8
Initial Matching•Matching cost computation: SAD
Left Right
d
9
Problem in disparity selection
a. Determine disparity easily for unique minimum valueb. Ambiguous disparity in case of multiple minimac. Matching cost calculated at point (205, 230) of Tsukuba image
10
Initial Matching: large correlation window
•Matching cost computation: SAD + penalty
•Penalty term
• Disparity computation
11
Problem in disparity selection
12
Initial Matching: small correlation window
•Only those disparity values that are carried by neighbouring pixels.•Matching cost computation without penalty
•N: the disparity values of the neighbouring pixels.•Avoid local minima and speed up
13
Flow Chart
14
Unreliable pixel detection• left–right cross-checking
15
Disparity Interpolation•Search for pixels with reliable disparity value in its eight neighbouring pixels.•Compute similarity of unreliable pixel and its reliable neighbor.
16
Flow Chart
17
Disparity Refinement
18
Outline•Introduction•Related Work•Proposed Method•Experimental Results•Conclusion
19
Experimental Results•Computation time of the proposed algorithm for different window sizes on Tsukuba image.(image size 384 × 288 with 16 disparity labels)
20
• Percentage error in non-occluded (nocc), whole image (all) and near depth discontinuities (disc) for different window sizes for all four images (Tsukuba, Venus, Teddy and Cones)
21
Experimental Results
• a. Without using small correlation window Ws and the disparity refinement step• b. Without using the disparity refinement step• c. Without using small correlation window Ws• d. With all four steps on Tsukuba image
22
Experimental Results
23
24
25
Experimental Results•Comparison the performance of the proposed algorithm with other correlation-based algorithms.
26
Experimental Results
27
Reference• [24] Gupta, R., Cho, S.-Y.: ‘Real-time stereo matching using adaptive binary window’
(3D Data Processing, Visualization and Transmission, 2010)• [25] Zhang, K., Lu, J., Lafruit, G., Lauwereins, R., Gool, L.V.: ‘Real-time accurate
stereo with bitwise fast voting on Cuda’. Int. Conf. Computer Vision Workshops, 2009, pp. 540–547• [26] Humenberger, M., Zinner, C., Weber, M., Kubinger, W., Vincze, M.:‘A fast
stereo matching algorithm suitable for embedded real-time systems’, Comput. Vis. Image Underst., 2010, 114, (11),pp. 1180–1202• [27] Gong, M., Yang, Y.: ‘Near real-time reliable stereo matching using
programmable graphics hardware’. IEEE Conf. Computer Vision and Pattern Recognition, 2005, pp. 924–931• [28] Richardt, C., Orr, D., Davies, I., Criminisi, A., Dodgson, N.: ‘Real-time
spatiotemporal stereo matching using the dual-cross-bilateral grid’. European Conf. Computer Vision, 2010, vol. 6313, pp. 510–523• [29] Ambrosch, K., Kubinger, W.: ‘Accurate hardware-based stereo vision’,Comput.
Vis. Image Underst., 2010, 114, (11), pp. 1303–1316
28
29
30
31
32
33
34
Conclusion•A new correlation-based stereo-matching approach.• Large window improves at non-textured image regions• Small window improves at depth discontinuities
•The CPU implementation computes at a speed of more than 10 frame/s.•Easily implemented on GPU. •The proposed method can be used in real-time applications to reconstruct the 3D structures with great accuracy at object boundaries.
35
Codebook based Stereo Matching for Natural User
Interface Sung-il Kang and Hyunki Hong
2013 IEEE International Conference on Consumer Electronics (ICCE)
36
Outline•Introduction•Proposed Method•Experimental Results•Conclusion
37
Introduction•Interactive user interface has been one of the major topics in consumer electronics.•Gesture based user interface• Interactive smart TV, Nintendo Wii, Sony PlayStation3 Move, and Microsoft Kinect.
•Propose a stereo system implemented on GPGPU for real-time performance.•Employ codebook to solve occlusion.
38
Flow chart
39
Proposed Method•Pre-processing• Laplace od Gaussian (LoG) filter for alleviating the lighting effects.
•Cost initialization• AD+Census[6]
[6] X. Mei, X. Sun, M. Zhou, H. Wang, and X. Zhang, “On building an accurate stereo matchng system on graphics hardware,” Proc. of GPUCV, pp. 467-474, 2011. http://www.camdemy.com/media/4724
40
Proposed Method•Cost aggregation[6]
• Cross-based aggregation• Color similarity and the length constraint
•Refinement[6,7]
• Left-right consistency check• Iterative region voting• Sub-pixel enhancement
d
[7] Q. Yang, C. Engels, R. Yang, H. Stewenius, and D. Nister, “Stereo matching with color-weighted correlation, hierarchical belief propagation and occlusion handling,” IEEE Transactions on PAMI, 2009.
41
Proposed Method
Occlusion? Find codeword
UpdatecodewordCodeword?
Yes
No
Yes
No
Add a newcodeword
42
Experimental Results• Device: Intel Quad 2.66GHz with Nvidia GTX460. • Stereo images are captured by a Bumblebee 3 from Point Grey Inc. • Time: 80~110ms/frame• Stereo matching is implemented on GPU.• The codebook generation and its evaluation is on CPU.
43
Experimental Results
44
Experimental Results
[8] K. J. Yoon and I. S. Kweon, “Adaptive support-weight approach for correspondence search,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, pp. 650-656, 2005.[9] C. Richardt, D Orr, I Davies, and A Criminisi, “Real-time spatiotemporal stereo matching using the dual-cross-bilateral grid,” Proc. of ECCV, 2010.
45
Conclusion•Propose a stereo system implemented on GPGPU for real-time performance.•Good performance at static background Only.