When Does a Camera See Rain?

Department of Computer Science

Columbia University

Kshitiz GargShree K. Nayar

ICCV ConferenceOctober 2005, Beijing, China

Sponsors:NSF, DARPA

When is Rain Visible?

Rain

[Garg and Nayar, ICCV 05]

Visibility of Rain Depends Strongly on Camera Parameters ( Exposure time, Aperture size, Focus distance )

Defining the Visibility of RainIn

tens

ity

Inte

nsity

Time (Frames) Time (Frames)

Visibility: Variance due to a Volume of Rain

Intensity of a Rain Streak

Back

gro

un

dzm Rzm

Fog like

0I

Rzmzm

NoiseLevel

Distance

Change in Intensity

)(00 br LLT

kI

mzz

z

zI m

00Ibw

wI

cd

Dependence on Defocus

Dependence on Exposure Time

mzz

Rain Region[Garg and Nayar, ICCV 05]

Streak’s width

Radius of blur circle

Dependence on Distance

Visibility and Volume of Rain

z

)()()(),( 22 zIzwznzI ddr

Back

gro

un

dz

mRz

Numberof drops

Intensity ChangeStreakWidth

dzzII rr ),()( 22 T

zNfGLL

v

ak br

),,()( 0

2

0

Variance

Layer of Rain:

Volume of Rain:

Rain Properties Camera Properties

Scene Properties

[Garg and Nayar, ICCV 05]

Experimental Verification of Rain Visibility

0 2 4 6 8 10

1

2

3

F-number (N)Exposure time

Visibility

0 10 20 30 40 50

1

2

Distance of Focus Plane z0 (m) (ms)0 40 80 120

1

4

8

Visibility Visibility

T

zNfGLL

v

akI brr

),,()()( 0

2

02

Theoretical Visibility

Measured Visibility(with error bars)

Camera Parametes

F-num (Aperture)

Exposure Time

Focus Distance

zoom

Scene Near Depth Exposure F-numberMotion Distance Range (m) Time (ms) slow close large 66 14

small 33 4.4

far large 33 6

small 33 2 fast close large X X

small X X

far large 8 6

small 8 2.4

Optimal Settings for Rain Removal

Slow: Motion< 15 pixels/secClose: z < Rain Visible Region Small: z < Rain Visible Region

Scene with Fast Motion

Default camera settings

F-number =12 Exposure time = 8 ms Focus plane = 100 m

Scene with Fast Motion

F-number decreased from 12 2.4

F-number = 2.4 Exposure time = 8 ms Focus plane = 100 m

Comparison: Scene with Fast Motion

Default camera settings

F-number : 12 2.4

Scene with Large Depth

Default camera settings

F-number =14 Exposure time = 16 ms Focus plane=10 m

Scene with Large Depth

Exposure time Increased from 16 66 ms

F-number =14 Exposure time = 66 ms Focus plane=10 m

Comparison: Scene with Large Depth

Default camera settings

Exposure time: 16 66ms

Panning Video in Rain

Default camera settingsExposure time =16 ms F-

num=14

Optimal camera settings Exposure time =33 ms F-

num=6

Implications for Vision Algorithms

Default camera settingsExposure time =16 ms F-

num=14

Optimal camera settings Exposure time =33 ms F-

num=6

[KLT feature tracker code from Stan Birchfield]

Enhancing the Visibility of Rain: Rain Gauge

Moderate Rain Video Heavy Rain VideoBackgrou

nd

Sampled Rain Volume

Backgrou

nd

Sampled Rain Volume

Type of Rainfall Camera parameters (f, N, z0, T)

Measured rain rate mm/hr

Reported rain rate* mm/hr

(a) Light (7095, 3.8, 3, 16) 1.34 0.763

(b) Moderate (5148, 4.4, 3, 4) 3.215 2.28

(c) Heavy (3300, 1.8, 3, 4) 14.28 11.17

Summary

Contributions

Future Work

Camera Based Rain Gauge

Analysis of Visibility of Rain

Camera Parameters for Removal of Rain

Limitations

A Camera-Lens system that automatically detects rain in videos and sets optimal camera parameters for reducing rain during image acquisition

Not as effective in very heavy rain and for scenes with close by and fast-moving objects