When Does a Camera See Rain?

Kshitiz Garg, Shree K. Nayar

ICCV’ 05

Outline

Abstract Dynamic Weather And Vision Visibility of Rain Camera Parameters for Rain Removal Camera Based Rain Gauge Conclusion

Abstract

The Intensity fluctuations depend on the camera parameters, the properties of rain, and the brightness of the scene.

We use the camera parameters to remove rain in images and videos.

Dynamic Weather And Vision

Dynamic weather (rain and snow) introduces sharp intensity fluctuation in images and videos.

Dynamic Weather And Vision (Cont’d) Analysis of Visibility of Rain

The square of the raindrop size The brightness of the background scene Exposure time The depth of field

Dynamic Weather And Vision (Cont’d) Camera Parameters for Removal of Rain

Exposure time F-number Focus setting

Dynamic Weather And Vision (Cont’d) Camera Based Rain Gauge

Camera parameters can be set to enhance the visual effects of rain.

We can build a camera-based rain gauge to measure rain rate.

Visibility of Rain

Variance at a pixel over time can be used to measure the visibility of rain and can be used as a quantitative measure of it.

Visibility of Rain (Cont’d)

Camera and Intensity of a Raindrop

LkIN

Tkk

N

kTLTL

NkI

0

20

22

4

1

4

Ref:

B.K.P. Horn. Robot Vision. The MIT Press, 1986.

radiance the

time exposure the

number-F the

gain camera the

intensity the

L:

T:

N:

k:

I:

Visibility of Rain (Cont’d)

Raindrops and Exposure Time

fazm 2

Rain visible region: mRzz 0

Rain visible region drop the ofradius the

pixels in length focal the

a:

f:

Visibility of Rain (Cont’d)

Raindrops and Exposure Time When :

When :

mzz

mzz

T

LLT

kIII brbr

10

zT

LLTzv

fakI br

114 2

0

Ref: K. Garg and S.K. Nayar. Detection and removal of rain in videos. In CVPR04,2004.

view. of field the within

stays drop a that time the

intensity. background the

intensity. blurred-motion the

τ:

: I

: I

b

r

Visibility of Rain (Cont’d)

Rain and Depth of Field

Since raindrops fall at high velocity, we assume

I

bTvb

TvI

A

AI

cic

idd

ci bTv

Ib

Ic

d

Ref: K. Garg and S.K. Nayar. Detection and removal of rain in videos. In CVPR04,2004.

kernel.defocus the of diameter the

drop. focused the of width the

streak. rain defocused the ofareas the

streak. rain focused the ofareas the

: b

ω:

: A

A:

c

d

Visibility of Rain (Cont’d)

Rain and Depth of Field When :

When :

mzz

mzz

brc

d LLTb

I

brc

d LLTzv

fa

bI

1

1

1 2

Ref: K. Garg and S.K. Nayar. Detection and removal of rain in videos. In CVPR04,2004.

Visibility of Rain (Cont’d)

Camera Parameters and Volume of Rain We partition the volume into thin layers of rain of

thickness .

The variance due to a variance of rain is then the sum of the variances due to the different layers.

z

Visibility of Rain (Cont’d)

Camera Parameters and Volume of Rain The variance due to a single layer of rain:

zIzznzI dd

r22 ,

z. depth at raindrop a to due streak defocused the of width the

view of fields pixel' a throughpass that layer the indrops of number mean the

(z): ω

.(z): nd

zbzz

fdz

vT

LLakdzzI

c

brr

2420

2 4,

,1)max(

density. size drop the

drop. the of size the

a

fzω(z):

ρ:

a:

Visibility of Rain (Cont’d)

Camera Parameters and Volume of Rain The standard deviation due to a volume of rain:

02

0 f,N,zGLLav

ρa

T

kIσ brr

.): G(f,N,z

(z)bω(z)z

fdz)G(f,N,z

c

planefocus the ofz0 distance the and N, number-F length, focal of function a0

0

Visibility of Rain (Cont’d)

Camera Parameters and Volume of Rain The visibility of rain decreases as with

exposure time of the camera.T

1

Visibility of Rain (Cont’d)

Camera Parameters and Volume of Rain initially increases rapidly with F-number and

then reaches saturation for higher F-number. r N

Visibility of Rain (Cont’d)

Camera Parameters and Volume of Rain dependence with respect to distance of the f

ocal plane.r 0z

Camera Parameters for Rain Removal Reducing Rain using Depth of Field

As decreases, decreases.N I

Camera Parameters for Rain Removal (Cont’d) Reducing Rain using Exposure Time

As increases, decreases. IT

Camera Parameters for Rain Removal (Cont’d) Reducing Rain using Multiple Parameters

Camera Parameters for Rain Removal (Cont’d) Reducing Heavy Rain

Camera Parameters for Rain Removal (Cont’d) This method will not be able to reduce rain in

scenes with very fast motion and when objects are very close to the camera.

Camera Based Rain Gauge

A camera-based rain gauge - a device that measures rain rate.

daavaadaahh

aav

36

3

410*6.3

200

Ref: R. Gunn and G.D. Kinzer. Terminal velocity for water droplet in stagnant air. J. Metero., 6:243–248, 1949.T. Wang and R.S Clifford. Use of rainfall-induced optical scintillations to measure path-averaged rain parameters. JOSA, 8:927–937, 1975.

rain. of density the

rain. ofradius the

:)(

:

a

a

Camera Based Rain Gauge (Cont’d)

Conclusion

The visibility of rain is affected by factors such as camera parameters, properties of rain, and the brightness of scene.

This method is not as effective in reducing rain from scenes with very heavy rain or scenes with fast-moving objects that are close to the camera.

Conclusion (Cont’d)

The visibility of rain can be enhanced to build an inexpensive and portable camera-based rain gauge that provides instantaneous rain rate.