Complaint: Reza Saleh, Defendant, and Amir Saleh, Relief Defendant
Shadow Detection In Video Submitted by: Hisham Abu saleh.
-
date post
22-Dec-2015 -
Category
Documents
-
view
220 -
download
1
Transcript of Shadow Detection In Video Submitted by: Hisham Abu saleh.
![Page 1: Shadow Detection In Video Submitted by: Hisham Abu saleh.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649d785503460f94a5ba30/html5/thumbnails/1.jpg)
Shadow Detection In Video
Submitted by: Hisham Abu saleh
![Page 2: Shadow Detection In Video Submitted by: Hisham Abu saleh.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649d785503460f94a5ba30/html5/thumbnails/2.jpg)
Why detecting shadows?
Moving objects detection:Shadows can cause object merging, object
shape distortion or object loses. Shadow points are detected as
foregroundThey differ from background.They move the same as the object .
![Page 3: Shadow Detection In Video Submitted by: Hisham Abu saleh.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649d785503460f94a5ba30/html5/thumbnails/3.jpg)
Why detecting shadows? cont.
![Page 4: Shadow Detection In Video Submitted by: Hisham Abu saleh.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649d785503460f94a5ba30/html5/thumbnails/4.jpg)
Shadow Detection Algorithms
Algorithms are divided into: Deterministic approaches
Model based Best results. Too Complex.
Non-Model based
Statistical approaches (use probabilistic functions). Parametric methods. Nonparametric methods. Selecting parameters is critical issue.
![Page 5: Shadow Detection In Video Submitted by: Hisham Abu saleh.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649d785503460f94a5ba30/html5/thumbnails/5.jpg)
Features presented by algorithms
Spectral Gray level Color (next lecture)
Spatial Pixel level Region level Frame level
Temporal Static Dynamic
![Page 6: Shadow Detection In Video Submitted by: Hisham Abu saleh.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649d785503460f94a5ba30/html5/thumbnails/6.jpg)
Algorithm Taxonomy
![Page 7: Shadow Detection In Video Submitted by: Hisham Abu saleh.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649d785503460f94a5ba30/html5/thumbnails/7.jpg)
Algorithm Comparison
We will compare between four algorithms: Statistical Nonparametric Statistical Parametric Deterministic Nonmodel-based (DNM1) Deterministic Nonmodel-based (DNM2)
The deterministic model-based class has not been considered due to its complexity and due to its reliance on very specific task domain assumptions.
![Page 8: Shadow Detection In Video Submitted by: Hisham Abu saleh.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649d785503460f94a5ba30/html5/thumbnails/8.jpg)
Statistical Nonparametric Approach
HorprasertColor, Pixel level, Static.Shadows have similar chromaticity, but
lower brightness than the background model.
A statistical learning procedure is used to automatically determine the appropriate thresholds.
![Page 9: Shadow Detection In Video Submitted by: Hisham Abu saleh.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649d785503460f94a5ba30/html5/thumbnails/9.jpg)
Statistical Parametric Approach
Mikie’ Color, Region level, Dynamic. The probability of a pixel belonging to shadow is
computed by assuming the color of the pixel not shadowed, and by using an approximated linear transformation to estimate the color of the point covered by a shadow.
Manual segmentation of a certain number of frames has to be done to collect statistics and to compute the values of matrix of the transformation.
![Page 10: Shadow Detection In Video Submitted by: Hisham Abu saleh.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649d785503460f94a5ba30/html5/thumbnails/10.jpg)
Deterministic Nonmodel-based Approach (DNM1)
Cucchiara, Color, Pixel level, Static.This algorithm works in the HSV color space
(closer to the human perception of color and it has more accuracy in distinguishing shadows).
A shadow cast on a background does not change its hue significantly.
![Page 11: Shadow Detection In Video Submitted by: Hisham Abu saleh.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649d785503460f94a5ba30/html5/thumbnails/11.jpg)
Deterministic Nonmodel-based Approach (DNM2)
StauderGray level, Frame level, Dynamic.The only algorithm that deals with
penumbra.Reference frame
Previous frame Has limitations: Object speed, Noise sensetive.
Background frame
![Page 12: Shadow Detection In Video Submitted by: Hisham Abu saleh.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649d785503460f94a5ba30/html5/thumbnails/12.jpg)
Deterministic Nonmodel-based Approach (DNM2) cont.
),(),(),(sk yxpyxEyx kk The image luminance:
A
pA
pA
c
LyxNcyxkc
LyxNcc
yx )),(cos(),(
)),(cos(
),(Ek
If illuminated
If penumbra
If umbra1),(0 yxk Describes the transition inside the penumbra
p(x,y) – The reflectance of the object surface.
E(x,y) – irradiance.
Cp – Intensity of light source.
CA – ambient light.
![Page 13: Shadow Detection In Video Submitted by: Hisham Abu saleh.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649d785503460f94a5ba30/html5/thumbnails/13.jpg)
Deterministic Nonmodel-based Approach (DNM2) cont.
Assumptions: 1. light source intensity cp is high.
The frame difference at a pixel changed by a moving cast shadow will be large.
If pixel is inside the umbra of a cast shadow at time instant k and outside the shadow at time instant k + 1. The reflectance of a static background does not change with time, thus the frame difference will be then:
0)),(cos(),(),(s-y)(x,s k1k LyxNcyxpyx pk
![Page 14: Shadow Detection In Video Submitted by: Hisham Abu saleh.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649d785503460f94a5ba30/html5/thumbnails/14.jpg)
Deterministic Nonmodel-based Approach (DNM2) cont.
2. Camera and background are static.3. Background is plane, light source
position is distant from background.4. Light source size and distance
between moving object and background are not negligible compared to the distance between light source and object.
![Page 15: Shadow Detection In Video Submitted by: Hisham Abu saleh.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649d785503460f94a5ba30/html5/thumbnails/15.jpg)
Deterministic Nonmodel-based Approach (DNM2) cont.
A. Detection of Static Background Edgesedges in the current
and in the previous image are detected and classified into moving and static edges.
![Page 16: Shadow Detection In Video Submitted by: Hisham Abu saleh.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649d785503460f94a5ba30/html5/thumbnails/16.jpg)
Deterministic Nonmodel-based Approach (DNM2) cont.
B. Detection of Uniform Changes of Shading Frame Ratio:
If the frame ratio is locally spatially constant, a moving cast shadow is assumed at position x,y.
),(
),(),( 1
yxE
yxEyxFR
k
k
![Page 17: Shadow Detection In Video Submitted by: Hisham Abu saleh.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649d785503460f94a5ba30/html5/thumbnails/17.jpg)
Deterministic Nonmodel-based Approach (DNM2) cont.
A shadow is decided to be in a region if it there is “darker” uniform region.
Static edges hint at a static background and can be exploited to detect nonmoving regions inside the frame difference.
![Page 18: Shadow Detection In Video Submitted by: Hisham Abu saleh.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649d785503460f94a5ba30/html5/thumbnails/18.jpg)
Deterministic Nonmodel-based Approach (DNM2) cont.
C. Penumbra DetectionAccording to assumption 4 the
cast shadow has penumbra.The penumbra causes a soft
luminance step at the contour of a shadow that is characterized by its step height, step width and its gradient.
![Page 19: Shadow Detection In Video Submitted by: Hisham Abu saleh.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649d785503460f94a5ba30/html5/thumbnails/19.jpg)
Deterministic Nonmodel-based Approach (DNM2) cont.
According to a table we put a candidate as a penumbra or not.
![Page 20: Shadow Detection In Video Submitted by: Hisham Abu saleh.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649d785503460f94a5ba30/html5/thumbnails/20.jpg)
Algorithm comparison
Two important quality measures: good detection
Low probability of misclassifying a shadow point.Minimizing the shadow points classified as
background/foreground.good discrimination
the probability of classifying nonshadow points as shadow should be low.
![Page 21: Shadow Detection In Video Submitted by: Hisham Abu saleh.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649d785503460f94a5ba30/html5/thumbnails/21.jpg)
Algorithm comparison
We define two metrics:
ss
s
FNTP
TP
FF
F
FNTP
TP
TP – True Positive
FN – False Negative
S – Shadow
F – Foreground (object)
![Page 22: Shadow Detection In Video Submitted by: Hisham Abu saleh.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649d785503460f94a5ba30/html5/thumbnails/22.jpg)
Algorithm comparison
The Results:
SNP – Very effective, sensitive to dark objects and noise.
SP – Not very effective since the assumption that the transformation matrix is constant on the entire image.
DNM1 – Good accuracy, stable performance.
DNM2 – fails when the background is not planar.
![Page 23: Shadow Detection In Video Submitted by: Hisham Abu saleh.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649d785503460f94a5ba30/html5/thumbnails/23.jpg)
Example
![Page 24: Shadow Detection In Video Submitted by: Hisham Abu saleh.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649d785503460f94a5ba30/html5/thumbnails/24.jpg)
Physics-Based Shadow Detection
This step detects moving pixels (object, shadow, and some erroneous pixels).
![Page 25: Shadow Detection In Video Submitted by: Hisham Abu saleh.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649d785503460f94a5ba30/html5/thumbnails/25.jpg)
Physics-Based Shadow Detection cont.
pixels on a detected surface cannot be shadow if they have higher intensity than the actual background.
This test does not reduce the shadow areas, but may successfully reduce object areas.
![Page 26: Shadow Detection In Video Submitted by: Hisham Abu saleh.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649d785503460f94a5ba30/html5/thumbnails/26.jpg)
Physics-Based Shadow Detection cont.
![Page 27: Shadow Detection In Video Submitted by: Hisham Abu saleh.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649d785503460f94a5ba30/html5/thumbnails/27.jpg)
Physics-Based Shadow Detection cont.
Shadow pixels falling on neutral surfaces, such as asphalt roads, tend to be more blue.
This test is not applied to all the pixels, but to neutral or gray surfaces that have low saturation (< 0.3)
![Page 28: Shadow Detection In Video Submitted by: Hisham Abu saleh.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649d785503460f94a5ba30/html5/thumbnails/28.jpg)
Physics-Based Shadow Detection cont.
![Page 29: Shadow Detection In Video Submitted by: Hisham Abu saleh.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649d785503460f94a5ba30/html5/thumbnails/29.jpg)
Physics-Based Shadow Detection cont.
This step performs surface segmentation. The connectivity C of two neighboring pixels p1 and p2 with
intensities u and v:
Otherwise
TvuHifppC
0
|),(|1)2,1(
![Page 30: Shadow Detection In Video Submitted by: Hisham Abu saleh.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649d785503460f94a5ba30/html5/thumbnails/30.jpg)
Physics-Based Shadow Detection cont.
H(u,v) is defined as follows:
21
21),(,2,1
1
1
1
1
RR
RRvuH
vv
vvR
uu
uuR
tt
tt
tt
tt
If two neighboring pixels belong to the same surface they will have temporal ratios that are close together.
![Page 31: Shadow Detection In Video Submitted by: Hisham Abu saleh.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649d785503460f94a5ba30/html5/thumbnails/31.jpg)
Physics-Based Shadow Detection cont.
![Page 32: Shadow Detection In Video Submitted by: Hisham Abu saleh.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649d785503460f94a5ba30/html5/thumbnails/32.jpg)
Physics-Based Shadow Detection cont.
This step removes the effect of sky illumination.
We subtract the foreground pixel values from the background over the masked area.
![Page 33: Shadow Detection In Video Submitted by: Hisham Abu saleh.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649d785503460f94a5ba30/html5/thumbnails/33.jpg)
Physics-Based Shadow Detection cont.
The user selects a patch of shadow region from a frame during the training.
The materials colors are stored to the verification step.
![Page 34: Shadow Detection In Video Submitted by: Hisham Abu saleh.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649d785503460f94a5ba30/html5/thumbnails/34.jpg)
Physics-Based Shadow Detection cont.
This step performs matching of body color of various surfaces with the stored body color of materials that we expect to see in the scene.
The algorithm has training and testing phases.
![Page 35: Shadow Detection In Video Submitted by: Hisham Abu saleh.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649d785503460f94a5ba30/html5/thumbnails/35.jpg)
Physics-Based Shadow Detection cont.
![Page 36: Shadow Detection In Video Submitted by: Hisham Abu saleh.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649d785503460f94a5ba30/html5/thumbnails/36.jpg)
References
A. Prati, I. Mikic, M. Trivedi, R. Cucchiara, “Detecting Moving Shadows: Algorithms and Evaluation”, IEEE Transactions on pattern analysis and machine intelligence, vol. 25, no. 7, July 2003.
S. Nadimi, B. Bhanu, “Physical Models for Moving Shadow and Object Detection in Video”, IEEE Transactions on pattern analysis and machine intelligence, vol. 26, no. 8, August 2004.
J. Stauder, R. Mech, J. Ostermann, “Detection of Moving Cast Shadows for Object Segmentation”, IEEE Transactions on multi media, vol. 1, no. 1, March 1999.
![Page 37: Shadow Detection In Video Submitted by: Hisham Abu saleh.](https://reader035.fdocuments.in/reader035/viewer/2022062314/56649d785503460f94a5ba30/html5/thumbnails/37.jpg)
References cont.
A. Prati, I. Mikic, C. Grana, M. Trivedi, “Shadow Detection Algorithms for Traffic Flow Analysis: a Comparative Study”.