Real-time shadow prediction using solar position and camera calibration

for ambient video surveillance

Computer Vision Laboratory

Inha University, South Korea

2015

1http://vision.inha.ac.kr/

Eunsoo Park, Xin Cui, Shengzhe Li, Hakil Kim

Background

• Problem• Shadows of the objects usually interfere with an automated

recognition system in detecting and tracking them

• Research Purpose• Predict the orientation and the length of the shadow of an object

based on solar position and the weather conditions at the current time

2

• A case where the correct tracking trajectory can only be obtained when shadows are removed.

• A.Sanin et al. “Shadow detection: A survey and comparative evaluation of recent method” Pattern Recognition (2012)

Proposed Method

Sun-Position

Calculation

Camera

Calibration

( Object Height )

( Azimuth ) ( Altitude )

Shadow

Estimation

Shadow

Detection

Time ,

Longitude,

Latitude

< Algorithm flow chart >

3

• Solar position can be described by the azimuth and the altitude

• θs = 90 − 𝑒0 −𝑃

1010×

283

273+𝑇×

1.02

60 tan(𝑒0+10.3

𝑒0+5.11)

• 𝑒0 = 𝑎𝑟𝑐 sin(sin𝜙0 sin 𝛿′ + cos𝜙0 cos 𝛿

′ cos𝐻′)

• 𝜙𝑠 = 𝑎𝑟𝑐 tansin 𝐻′

cos 𝐻′ sin 𝜙0−tan 𝛿′ cos 𝜙0+ 180

𝜃𝑠 : Sun zenith angle , 𝑃 is the local

pressure , 𝑇 is time

𝒆𝟎 : Sun’s topocentric elevation angle

𝝓𝒔 : Sun’s topocentric azimuth angle

𝜙0 : observer geometric latitude

calculated using the local latitude

𝛿′ : the sun declination calculated using

the geocentric sun declination from the

local longitude and current time

𝐻′ : the topocentric local hour angle

from the current time

• I.Reda et al. “Solar position algorithmfor solar radiation applications.” Technical report NREL/TP-

560-34302, National Renewable Energy Laboratory, USA, (2008)

• J.Wang et al. “Shadow extraction and application in pedestrian detection.” EURASIP Journal on

Image and Video Processing (2014)

From GPS

Proposed Method

Sun-Position

Calculation

Camera

Calibration

( Object Height )

( Azimuth ) ( Altitude )

Shadow

Estimation

Shadow

Detection

Time ,

Longitude,

Latitude

< Algorithm flow chart >

4

𝑺 = 𝑯 ∙ 𝒄𝒐𝒕 𝒆𝟎

• Estimation of the shadow’s length

• Relation between shadow’s direction and azimuth

• 𝐍 : True north

• 𝐙 : forward direction of the camera

• 𝝋 : Angle between 𝐍 and 𝐙

• 𝝓𝒔 : Azimuth angle of the Sun

• 𝜰 : Angle between 𝑵 and shadow

• (𝑿𝒇, 𝒁𝒇) : Object orientation

• (𝑿𝐬, 𝒁𝐬) : End coordinate of a shadow

𝜰 = 𝝓𝒔 − 𝝅 +𝝋

Shadow’s Length

Shadow’s direction

From GPS

Proposed Method

Sun-Position

Calculation

Camera

Calibration

( Object Height )

( Azimuth ) ( Altitude )

Shadow

Estimation

Shadow

Detection

Time ,

Longitude,

Latitude

< Algorithm flow chart >

5

• Simplified camera calibration method

• Calibration using Head and Foot points of pedestrians

Meaningful

parameters

• Only consider

• 𝒇 : focal length

• 𝜽 : camera angle

• 𝒄 : height of camera

• S. Li et. al., “Simplified Camera Calibration for Human Height Estimation in Video Surveillance”, EURASIP Journal on Image and

Video Processing, under reviewing.

From GPS

Proposed Method

Sun-Position

Calculation

Camera

Calibration

( Object Height )

( Azimuth ) ( Altitude )

Shadow

Estimation

Shadow

Detection

Time ,

Longitude,

Latitude

< Algorithm flow chart >

6

Y

Z

𝐗

(Xh, Yh, Zh)

(Xf, Yf, Zf)

(Xs, Zs)

𝑯

(𝑥h, 𝑦h)

(𝑥f, 𝑦f)

(𝑥𝑠, 𝑦𝑠)

N

𝝋

Principal axis

World Coordinates

𝚼

𝝓𝒔

𝐞𝐨

• Prediction of the object’s height derived from camera calibration and shadow’s length

From GPS

Experimental Results

• Static camera and test images for experiments

• Results

• Camera : SNC-VB600B

• Test image’s size: 1280x720

7

• Comparing real length

and predicted length of

shadows in various time

Time Azimuth Elevation Meas(cm) Est(cm) Error(cm) Rate10:00AM 102.7° 54.2° 71.5 73.74 -2.24 3.13%

10:30AM 141.691° 40.82° 113 116.92 -3.92 3.47%

11:00AM 155.1° 38.3° 126 126.62 -0.62 0.49%

11:00AM 150.80° 44.13° 102 104.11 -2.11 2.07%

11:30AM 160.87° 46.56° 95.6 95.64 -0.04 0.04%

11:30AM 164.2° 40.3° 119 117.9 1.1 0.92%

12:00PM 151.4° 73.9° 29 31.03 -2.03 7.00%

12:20PM 177.90° 51.75° 78 79.62 -1.62 2.08%

.

.

.01:30PM 204.45° 45.38° 102.5 99.67 2.83 2.76%

02:00PM 238.6° 66.5° 50 45.19 4.81 9.62%

02:00PM 214.02° 42.46° 114 110.38 3.62 3.18%

02:30PM 222.59° 38.76° 131 125.8 5.2 3.97%

03:00PM 255.1° 56.2° 70 68.46 1.54 2.20%

04:00PM 266.5° 44.5° 103 103.23 -0.23 0.22%

04:00PM 243.05° 24.47° 223 221.93 1.07 0.48%

04:10PM 248.44° 27.27° 210 195.94 14.06 6.70%

05:00PM 275.5° 32.6° 159 156.99 2.01 1.26%

06:00PM 283.6° 20.8° 270 265.8 4.2 1.56%

Error Rate =

𝑴𝒆𝒂𝒔−𝑬𝒔𝒕

𝑴𝒆𝒂𝒔× 100%

• Max. Error Rate: 9.62%

• Min. Error Rate: 0.04%

• Ave. Error Rate: 3.41%

Meas : Measured shadow distance, Est : Estimated shadow distance

Conclusions and Future Works

• Conclusions

8

• The proposed method is able to predict the direction and

the length of object’s shadow in an acceptable error rate

• The proposed method can operate in real time

• Future works

• The relational equation between cameras and the Sun

position can be derived from the proposed method

• The proposed method can be easily utilized to outdoor

video surveillance systems

• To develop shadow removal and video quality

enhancement method combined with weather conditions