Post on 24-Feb-2016
description
Stereo analysis of low textured regions with application towards sea-
ice reconstruction
Rohith MV, Gowri Somanath, Chandra KambhamettuVideo/Image Modeling and Synthesis(VIMS) Lab, Dept. of Computer and Information Sciences
Cathleen GeigerCenter for Climatic Research, Department. of Geography
University of Delaware, USA
Sea ice
Need for reconstruction“The feasibility of using snow
surface roughness to infer ice thickness and ice bottom roughness is promising….”
“…the goal of a circumpolar high resolution data set of Antarctic sea ice and snow thickness distributions has not yet been achieved …”
“…crucial for future validation of satellite observations, climate models, and for assimilation into forecast models…”
Ref: Workshop on Antarctic Sea Ice Thickness, 2006; Annals of Glaciology
OutlineStereo in presence of large texture-less areasEntropy based SegmentationOur Approach
Two stage estimationMRF Formulation Occlusion Model
Comparison of resultsConclusion
Sample Images
Some characteristics in images
Smoothly changing disparityNo edge Low color variation
Stereo
Left Image Hierarchical BP
Graph Cuts Our Algorithm
Previous approachesMethod Matching measure Segmentation Multiple Segmentation Hierarchical Occlusion Model
Klaus et al. ICPR 2006
Matching Pixels +Voting for plane + BP
Adaptive measure with SAD and gradient Mean Shift No
(Only one image) No No
Hong et al. CVPR 2004 Matching Pixels + Graph cut SAD, Fixed Mean Shift No
(Only one image) No Yes
Wang et al. CVPR 2008
Adaptive correlation window + Voting for plane + BP Adaptive correlation Mean Shift No
(Only one image) No Yes
Nister et al. CVPR 2006
Matching Pixels + Hierarchical BP Color weighted correlation Rectangular Grid Yes
(Both images) Yes Yes
Trinh BMVC 2008 Matching Segments + BP SSD + gradient Mean shift Yes
(Only one image) Yes No
Felzenswalb et al. CVPR 2004 Matching Pixels + BP None - Yes Yes
Our Method Matching Segments + 2 level BP SAD Entropy filtering + graph based segmentation No 2 levels Yes
Entropy based segmentation
argmax
Entropy based segmentation1. Convert the image to grayscale and
calculate the histogram.2. Estimate the brightness threshold as the
gray value that maximizes the entropy of the segmented image.
3. Partition the histogram based on that threshold into two parts. Equalize the two histograms. For each histogram repeat steps 2 and 3.
Comparison with mean shift
Left Image
Entropy based segments
Entropy based segmentation
Mean Shift segments
Our approachTwo stage solution
S2 S3
S1
S2 S3
S1
S2 S3
S1
Segment disparity• Single disparity per segment• Fewer disparity levels• Segment neighborhood
Pixel disparity• Disparity per pixel• Full range of disparities• Pixel neighborhood• Occlusion Detection
Example
MRF FormulationSegment Level DisparityΕሺ𝑓ሻ= 𝐷𝑝𝑝∈𝑆 ൫𝑓𝑝൯+ 𝑉൫𝑓𝑝,𝑓𝑞൯
ሺ𝑝,𝑞ሻ∈𝑁
Where
𝑓 is the disparity assignment
𝐷𝑝൫𝑓𝑝൯ is the SAD error of the segment 𝑝 under the disparity 𝑓𝑝
𝑉൫𝑓𝑝,𝑓𝑞൯ is the penalty for assigning different disparities to adjacent segments.
𝑉൫𝑓𝑝,𝑓𝑞൯= 𝜂 ห𝑓𝑝 − 𝑓𝑞ห.
MRF ModelPixel Level Disparity𝑬′ሺ𝒇′ሻ= σ 𝑫𝒖′𝒖∈𝑰 ሺ𝒇𝒖′ ሻ+ σ 𝑽′ሺ𝒇𝒖′ ,𝒇𝒗′ ሻሺ𝒖,𝒗ሻ∈𝑵′ (2)
𝑫𝒖′ ሺ𝒇𝒖′ ሻ= ൜𝜔∗𝑒−𝜇 if 𝒇𝒖′ is OCCLUDED𝛿∗𝑆𝐴𝐷+ 𝛽∗ȁ#𝒇𝒖′ − 𝑓𝑠ȁ# otherwise
𝑉′ሺ𝒇𝒖′ ,𝒇𝒗′ ሻ= ൜0 if either 𝒇𝒖′ or 𝒇𝒗′ is OCCLUDED 𝜂′ȁ#𝑓′𝑢 − 𝑓′𝑣ȁ# otherwise
where 𝑠 is the segment to which pixel 𝑢 belongs 𝜇 is the minimum SAD error for pixel 𝑢 𝑓𝑠 is the labeling from the segment disparity for segment s. 𝜔∗𝑒−𝜇 penalizes occlusion
Occlusion Model
Rohith MV, Gowri Somanath, Chandra Kambhamettu, Cathleen Geiger Towards estimation of dense disparities from stereo images containing large textureless regions. 19th International Conference on Pattern Recognition(ICPR), 2008
Results
ResultsResults
ResultsResults
Middlebury dataset
Tsukuba Venus Teddy Cones nonocc all disc nonocc all disc nonocc all disc nonocc all disc
Our Method 1.49 3.4 7.87 0.33 0.6 3.57 3.73 7.13 9.75 3.25 9.13 8.80 Hierarchical
BP [4] 2.13 4.29 11.4 1.4 2.38 16.5 17.3 25.2 31.0 12.5 20.6 22.0
Multiscale BP [10]
4.85 7.03 19.0 7.4 8.94 28.1 14.2 22.8 30.9 10.6 20.5 22.9
ConclusionsEntropy based segmentation to handle large
texture-less regionsTwo step MRF formulation Solution using belief propagationCan handle large disparity ranges
Future workExplore combination of segmentations based
on region characteristicsUse priors over segmentation and disparity
calculation in sequence of images
AcknowledgementsThis work was made possible by National
Science Foundation (NSF) Office of Polar Program grants, ANT0636726 and ARC0612105.