Introduction to Computer Vision Introduction Zhigang Zhu, City College of New York...
-
Upload
scarlett-blankenship -
Category
Documents
-
view
221 -
download
4
Transcript of Introduction to Computer Vision Introduction Zhigang Zhu, City College of New York...
Introduction to
Computer Vision IntroductionIntroduction
Zhigang Zhu, City College of New York [email protected]
CSc I6716Spring 2012
Image Enhancement
Part IFeature Extraction (1)
Introduction to
Computer Vision What are Image Features?What are Image Features?
n Local, meaningful, detectable parts of the image.
Introduction to
Computer Vision More Color WoesMore Color Woes
Squares with dots in them are the same color
Introduction to
Computer Vision TopicsTopics
n Image Enhancementl Brightness mappingl Contrast stretching/enhancementl Histogram modificationl Noise Reductionl ……...
n Mathematical Techniquesl Convolutionl Gaussian Filtering
n Edge and Line Detection and Extractionn Region Segmentationn Contour Extractionn Corner Detection
Introduction to
Computer Vision Image EnhancementImage Enhancement
n Goal: improve the ‘visual quality’ of the imagel for human viewingl for subsequent processing
n Two typical methodsl spatial domain techniques....
u operate directly on image pixelsl frequency domain techniques....
u operate on the Fourier transform of the image
n No general theory of ‘visual quality’l General assumption: if it looks better, it is betterl Often not a good assumption
Introduction to
Computer Vision Spatial Domain MethodsSpatial Domain Methods
n Transformation Tl point - pixel to pixell area - local area to pixell global - entire image to pixel
n Neighborhoods l typically rectangularl typically an odd size: 3x3, 5x5, etcl centered on pixel I(x,y)
n Many IP algorithms rely on this basic notion
T(I(x,y))
I(x,y) I’(x,y)neighborhood N
I’(x,y) = T(I(x,y))
O=T(I)
Introduction to
Computer Vision Point Transforms: General IdeaPoint Transforms: General Idea
O = T(I)
Input pixel value, I, mapped to output pixel value, O, via transfer function T.
0 255
255
INPUT
OU
TP
UT
TransferFunction T
Introduction to
Computer Vision Grayscale TransformsGrayscale Transforms
n Photoshop ‘adjust curve’ command
Input gray value I(x,y)
Output gray value I’(x,y)
Introduction to
Computer Vision Point Transforms: BrightnessPoint Transforms: Brightness
0 0.5 10
0.5
1
0 0.5 10
0.5
1
0 0.5 10
0.5
1
0 100 200
0
2000
4000
0 0.5 1
0
2000
4000
0 0.5 1
0
2000
4000
Introduction to
Computer Vision Point Transforms:ThresholdingPoint Transforms:Thresholding
n T is a point-to-point transformationl only information at I(x,y) used to generate I’(x,y)
n Thresholding
Imax if I(x,y) > tImin if I(x,y) ≤ t
I’(x,y) =
t=89
t 25500
255
Introduction to
Computer Vision Point Transforms: Linear StretchPoint Transforms: Linear Stretch
0 255
255
INPUT
OU
TP
UT
Introduction to
Computer Vision Linear ScalingLinear Scaling
n Consider the case where the original image only utilizes a small subset of the full range of gray values:
n New image uses full range of gray values.n What's F? {just the equation of the straight line}
I'(x,y)
I(x,y) ImaxImin0
0 K
K
Output image I'(x,y) = F [ I(x,y)] Desired gray scale range: [0 , K]
Input image I(x,y) Gray scale range: [I , I ]min max
Introduction to
Computer Vision Linear ScalingLinear Scaling
n F is the equation of the straight line going through the point (Imin , 0) and (Imax , K)
n useful when the image gray values do not fill the available range.
n Implement via lookup tables
I' = mI + b
I'(x,y) = I(x,y) - minIKI max min- I
KI max min- I
Introduction to
Computer Vision Scaling Discrete ImagesScaling Discrete Images
n Have assumed a continuous grayscale.n What happens in the case of a discrete grayscale with
K levels?
?
Empty!
1 2 3 4 5 6 70
0
1
2
3
5
4
6
7
Input Gray Level
Out
put
Gra
y L
evel
Introduction to
Computer Vision Non-Linear Scaling: Power LawNon-Linear Scaling: Power Law
n < 1 to enhance contrast in dark regionsn > 1 to enhance contrast in bright regions.
O = Ig
0 0.5 10
0.5
1
=1g
<1g
>1g
Introduction to
Computer Vision Square Root Transfer: g=.5Square Root Transfer: g=.5
0 0.5 10
0.5
1
Introduction to
Computer Vision
g=3.0
g=3.0
0 50 100 150 200 250
0
500
1000
1500
2000
2500
3000
3500
4000
0 50 100 150 200 250
0
500
1000
1500
2000
2500
3000
3500
4000
0 0.5 10
0.5
1
Introduction to
Computer Vision ExamplesExamples
n Technique can be applied to color imagesl same curve to all color bandsl different curves to separate color bands:
Introduction to
Computer Vision Point Transforms:ThresholdingPoint Transforms:Thresholding
n T is a point-to-point transformationl only information at I(x,y) used to generate I’(x,y)
n Thresholding
Imax if I(x,y) > tImin if I(x,y) ≤ t
I’(x,y) =
t=89
t 25500
255
Introduction to
Computer Vision Threshold SelectionThreshold Selection
n Arbitrary selectionl select visually
n Use image histogram
Threshold
Introduction to
Computer Vision HistogramsHistograms
n The image shows the spatial distribution of gray values.n The image histogram discards the spatial information and
shows the relative frequency of occurrence of the gray values.
0 3 3 2 5 5
1 1 0 3 4 5
2 2 2 4 4 4
3 3 4 4 5 5
3 4 5 5 6 6
7 6 6 6 6 5
0 2 .05 1 2 .05 2 4 .11 3 6 .17 4 7 .20 5 8 .22 6 6 .17 7 1 .03
Image CountGray Value
Rel. Freq.
Sum= 36 1.00
Introduction to
Computer Vision Image HistogramImage Histogram
n The histogram typically plots the absolute pixel count as a function of gray value:
0 1 2 3 4 5 6 70
1
2
3
4
5
6
7
8
Pix
el C
ount
Gray Value
For an image with dimensions M by N
MNiHI
Ii
min
min
)(
Introduction to
Computer Vision Probability InterpretationProbability Interpretation
n The graph of relative frequency of occurrence as a function of gray value is also called a histogram:
n Interpreting the relative frequency histogram as a probability distribution, then:
0 1 2 3 4 5 6 70
0.05
0.1
0.15
0.2
0.25
Rela
tive F
requency
Gray Value
P(I(x,y) = i) = H(i)/(MxN)
Introduction to
Computer Vision Cumulative Density FunctionCumulative Density Function
n Interpreting the relative frequency histogram as a probability distribution, then:
n Curve is called the cumulative distribution function
0 1 2 3 4 5 6 70
0.05
0.1
0.15
0.2
0.25
Rela
tive F
requency
Gray Value
0 1 2 3 4 5 6 70
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Gray Value
P(I(x,y) = i) = H(i)/(MxN)
i
k
kPiQ0
)()(
i
k
kHiCH0
)()(
Introduction to
Computer Vision ExamplesExamples
1241
0 256
1693
0 256
Introduction to
Computer Vision Color HistogramsColor Histograms
Introduction to
Computer Vision Histogram EqualizationHistogram Equalization
0 50 100 150 200 250
0
500
1000
1500
2000
2500
3000
3500
4000
n Image histograms consist of peaks, valleys, and low plains
n Peaks = many pixels concentrated in a few grey levels
n Plains = small number of pixels distributed over a wider range of grey levels
Introduction to
Computer Vision Histogram EqualizationHistogram Equalization
n The goal is to modify the gray levels of an image so that the histogram of the modified image is flat.l Expand pixels in peaks over a wider range of gray-levels.l “Squeeze” low plains pixels into a narrower range of gray
levels.n Utilizes all gray values equallyn Example Histogram:
l Note low utilization of small gray values
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 150
200400600800
100012001400160018002000
Pix
el C
ount
Gray Value
Introduction to
Computer Vision Desired HistogramDesired Histogram
n All gray levels are used equally.n Has a tendency to enhance contrast.
0 1 2 3 4 5 6 7 8 9 1011 12 13 14 150
50
100
150
200
250
300
Pix
el C
ount
Gray Value
Introduction to
Computer Vision Brute Force Brute Force
0 0 256 22 from 7, 234 from 8 1 0 256 1 from 8, 255 from 9 2 0 256 195 from 9, 61 from 10 3 0 256 256 from 10 4 0 256 256 from 10 5 0 256 256 from 10 6 0 256 256 from 10 7 22 256 256 from 10 8 235 256 256 from 10 9 450 256 256 from 10
10 1920 256 67 from 10, 189 from 11 11 212 256 23 from 11, 10 from 12, 223 from 13 12 10 256 10 from 13, 246 from 14 13 233 256 256 from 14 14 672 256 170 from 14, 86 from 15 15 342 256 256 from 15
Gray Actual Desired How to get it Scale Count Count
Sum 4096 4096
How are the gray levels in the original image changed to produce the enhanced image?
Method 1. Choose points randomly.
Method 2. Choice depends on the graylevels of their neighboring points.
Computationally expensive.
Approximations.
Introduction to
Computer Vision Histogram EqualizationHistogram Equalization
n Mapping from one set of grey-levels, I, to a new set, O.n Ideally, the number of pixels, Np, occupying each grey
level should be:
n To approximate this, apply the transform
l Where CH is the cumulative histogram (see next slide)l j is the gray value in the source imagel i is the gray value in the equalized image
Np = M*N
G
i = MAX 0, roundCH(j)
Np
-1
Introduction to
Computer Vision ExampleExample
800
600
400
200 0
0 1 2 3 4 5 6 8
ideal
800
600
400
200 0
G=8MxN=2400Np=300
j H(j) CH(j) i
0 100 01 800 22 700 43 500 64 100 65 100 76 100 77 0 7
100900160021002200230024002400
CH(j) = S H(i)i=0
j
0 1 2 3 4 5 6 8
Introduction to
Computer Vision ExampleExample
0 50 100 150 200 250
0
2000
4000
0 50 100 150 200 250 3000
0.5
1
0 50 100 150 200 250
0
2000
4000
Introduction to
Computer Vision ComparisonComparison
Original >1
Histogram equalization
Introduction to
Computer Vision Why?Why?
n Given MxN input image I with gray scale p0….pk and histogram H(p).
n Desire output image O with gray scale q0….qk and uniform histogram G(q)
n Treat the histograms as a discrete probability density function. Then monotonic transfer function m = T(n) implies:
n The sums can be interpreted as discrete distribution functions.
From Image Processing, Analysis, and Machine Vision, Sonka et al.
S G(qi) = S H(pj)i=0 j=0
m n
Introduction to
Computer Vision Why? continuedWhy? continued
n The histogram entries in G must be:(because G is uniform)
n Plug this into previous equation to get:
n Now translate into continuous domain (can’t really get uniform distributions in the discrete domain)
MxNqk - q0
G(i) =
S = S H(pi)i=0 i=0
m n MxNqk - q0
MxNqk - q0
= H(s)dsp0
pn
q0
qm
ds
MN(qm-q0)qk - q0
Introduction to
Computer Vision Why? continuedWhy? continued
n Solve last equation for q to get:
n Map back into discrete domain to get:
n Leads to an algorithm…..
MNqk - q0
H(s)ds + q0p0
pn
qm = T(p) =
MNqk - q0 S H(i) + q0
qm = T(p) =i=p0
pn
Introduction to
Computer Vision Histogram Equalization AlgorithmHistogram Equalization Algorithm
n For an NxM image of G gray levels, say 0-255l Create image histograml For cumulative image histogram Hc
n Set
n Rescan input image and write new output image by setting
T(p) = round ( Hc (p))G-1NM
gq = T(gp)
Introduction to
Computer Vision ObservationsObservations
n Acquisition process degrades imagen Brightness and contrast enhancement implemented
by pixel operationsn No one algorithm universally useful n > 1 enhances contrast in bright imagesn < 1 enhances contrast in dark imagesn Transfer function for histogram equalisation
proportional to cumulative histogram
Introduction to
Computer Vision Noise ReductionNoise Reduction
n What is noise?n How is noise reduction performed?
l Noise reduction from first principlesl Neighbourhood operators
u linear filters (low pass, high pass)u non-linear filters (median)
image
+
noise
=
‘grainy’ image
Introduction to
Computer Vision NoiseNoise
50 100 150 200 250
100
200
300
50 100 150 200 250
-100
0
100
50 100 150 200 2500
100
200
300
n Plot of image brightness.n Noise is additive.n Noise fluctuations are
rapidl high frequency.
Image
Noise
Image + Noise
Introduction to
Computer Vision Sources of NoiseSources of Noise
n Sources of noise = CCD chip.n Electronic signal fluctuations
in detector.n Caused by thermal energy.n Worse for infra-red sensors.n Electronicsn Transmission
Radiation from the long wavelength IR band is used in most infrared imaging applications
Introduction to
Computer Vision Noise ModelNoise Model
0 50 100 150 200 250
0
500
1000
1500
2000
2500
2
n Plot noise histogramn Typical noise distribution is
normal or Gaussiann Mean(noise) = 0n Standard deviation
h(x) = exp - 1
2ps2
12
(x- )ms
2
Introduction to
Computer Vision Effect of sEffect of s
n Integral under curve is 1
=10
=20
=30
0 50 100 150 200 250
0
1000
2000
0 50 100 150 200 250
0
1000
2000
0 50 100 150 200 250
0
1000
2000
Introduction to
Computer Vision Two Dimensional GaussianTwo Dimensional Gaussian
Introduction to
Computer Vision NoiseNoise
n Noise varies above and below uncorrupted image.
70 80 90 100 110 120 130 140
160
170
180
190
200
210
220
230
Image
Image +
Noise
Introduction to
Computer Vision Noise Reduction - 1Noise Reduction - 1
n How do we reduce noise?
n Consider a uniform 1-d image and add noise.
n Focus on a pixel neighbourhood.
n Averaging the three values should result in a value closer to original uncorrupted data
n Especially if gaussian mean is zero
5 10 15 20 25 300
50
100
150
200
250
5 10 15 20 25 300
50
100
150
200
250
Ai-1 Ai Ai+1 Ci
Introduction to
Computer Vision Noise Reduction - 1Noise Reduction - 1
n Averaging ‘smoothes’ the noise fluctuations.
n Consider the next pixel Ai+1
n Repeat for remainder of pixels.
5 10 15 20 25 300
50
100
150
200
250
5 10 15 20 25 300
50
100
150
200
250
Ai-1 Ai Ai+1 Ai+2 Ci+1 321
1
iii
i
AAAC
Introduction to
Computer Vision Neighborhood OperationsNeighborhood Operations
n All pixels can be averaged by convolving 1-d image A with mask B to give enhanced image C.
n Weights of B must equal one when added together.l Why?
C = A * B
B = [ B1 B2 B3 ]
Ci = Ai-1 ´B1 + Ai ´B2 + Ai+1 ´B3
B = [1 1 1]
Ci =
13
Ai-1 + Ai + Ai+1
3
Introduction to
Computer Vision 2D Analog of 1D Convolution2D Analog of 1D Convolution
n Consider the problem of blurring a 2D imagen Here’s the image and a blurred version:
n How would we do this?l What’s the 2D analog of 1D convolution?
Introduction to
Computer Vision 2D Blurring Kernel2D Blurring Kernel
n C = A * Bn B becomes
1 1 11 1 11 1 1
B = 19
Introduction to
Computer Vision ConvolutionConvolution
n Extremely important concept in computer vision, image processing, signal processing, etc.
n Lots of related mathematics (we won’t do)n General idea: reduce a filtering operation to the repeated
application of a mask (or filter kernel) to the imagel Kernel can be thought of as an NxN imagel N is usually odd so kernel has a central pixel
n In practicel (flip kernel)l Align kernel center pixel with an image pixell Pointwise multiply each kernel pixel value with corresponding
image pixel value and add resultsl Resulting sum is normalized by kernel weightl Result is the value of the pixel centered on the kernel
Introduction to
Computer Vision
Image
Kernel
ResultImage
ExampleExample
n Kernel is aligned with pixel in image, multiplicative sum is computed, normalized, and stored in result image.
n Process is repeated across image.n What happens when kernel is near
edge of input image?
Introduction to
Computer Vision Border ProblemBorder Problem
n missing samples are zeron missing samples are grayn copying last linesn reflected indexing (mirror)n circular indexing (periodic)n truncation of the result
Introduction to
Computer Vision
Image size = M1 N1
Mask size = M2 N2
Convolution size = M1- M2 +1 N1-N2+1
N1
N2
N1-N2+1
Typical Mask sizes= 33, 5 5, 77,9 9, 1111
What is the convolved image size for a 128 128 image and 7 7 mask?
Convolution SizeConvolution Size
Introduction to
Computer Vision ConvolutionConvolution
Image Data:
10 12 40 16 19 10
14 22 52 10 55 41
10 14 51 21 14 10
32 22 9 9 19 14
41 18 9 22 27 11
10 7 8 8 4 5
Mask:
1
0
-1
-1 0 1
1 11
1 11
1 11
k=-1 m=-1
1
1S = M(k,m)
I (i,j) = I(i+k,j+m) *M(k,m)fk=-1 m=-1
1
11S = I M*
Introduction to
Computer Vision Properties of ConvolutionProperties of Convolution
n Commutative: f1(t) * f2 (t) = f2 (t) * f1 (t)n Distributive: f1(t) * [f2(t) + f3 (t)] = f1(t) * f2 (t) + f1 (t) * f3 (t)n Associative: f1(t) * [f2(t) * f3 (t)] = [f1(t) * f2(t)] * f3 (t) n Shift: if f1(t) * f2(t) = c(t), then
l f1(t) * f2(t-T) = f1(t-T) * f2(t) = c(t-T)
n Convolution with impulse: f(t) *d(t) = f(t)n Convolution with shifted impulse: f(t) *d(t-T) = f(t-T)
Introduction to
Computer Vision Noise Reduction - 1Noise Reduction - 1
70 80 90 100 110 120 130 140
160
170
180
190
200
210
220
230
Uncorrupted Image
Image + Noise
Image + Noise - Blurred
Introduction to
Computer Vision Noise Reduction - 1Noise Reduction - 1
n Technique relies on high frequency noise fluctuations being ‘blocked’ by filter. Hence, low-pass filter.
n Fine detail in image may also be smoothed.
n Balance between keeping image fine detail and reducing noise.
Introduction to
Computer Vision Noise Reduction - 1Noise Reduction - 1
n Saturn image coarse detailn Boat image contains fine
detailn Noise reduced but fine
detail also smoothed
Introduction to
Computer Vision hmmmmm…..hmmmmm…..Image Blurred Image
- =
Introduction to
Computer Vision Noise Reduction - 2: Median FilterNoise Reduction - 2: Median Filter
n Nonlinear Filtern Compute median of points in neighborhoodn Has a tendency to blur detail less than averagingn Works very well for ‘shot’ or ‘salt and pepper’ noise
Original Low-pass Median
Introduction to
Computer Vision Noise Reduction - 2Noise Reduction - 2
60 80 100 120 140 160 180
80
100
120
140
160
180
60 80 100 120 140 160 180
80
100
120
140
160
180
200
Low-pass Median
• Low-pass: fine detail smoothed by averaging• Median: fine detail passed by filter
Introduction to
Computer Vision Edge Preserving SmoothingEdge Preserving Smoothing
n Smooth (average, blur) an image without disturbingl Sharpness orl position
Of the edgesn Nagoa-Maysuyama Filtern Kuwahara Filtern Anisotropic Diffusion Filtering (Perona & Malik, .....)n Spatially variant smoothing
Introduction to
Computer Vision Nagao-Matsuyama FilterNagao-Matsuyama Filter
n Calculate the variance within nine subwindows of a 5x5 moving window
n Output value is the mean of the subwindow with the lowest variance
n Nine subwindows used:
Introduction to
Computer Vision Kuwahara FilterKuwahara Filter
n Principle:l divide filter mask into four regions (a, b, c, d). l In each compute the mean brightness and the variance l The output value of the center pixel (abcd) in the window is the
mean value of that region that has the smallest variance.
Introduction to
Computer Vision Kuwahara Filter ExampleKuwahara Filter Example
Original
Median(1 iteration)
Median(10 iterations)
Kuwahara
Introduction to
Computer Vision Anisotropic Diffusion FilteringAnisotropic Diffusion Filtering
Note: Original Perona-Malik diffusion process is NOT anisotropic, although they erroneously said it was.
Introduction to
Computer Vision Example: Perona-MalikExample: Perona-Malik
n In general:l Anisotropic diffusion estimates an image from a noisy imagel Useful for restoration, etc.l Only shown smoothing examplesl Read the literature
Introduction to
Computer Vision Observations on EnhancementObservations on Enhancement
n Set of general techniquesn Varied goals:
l enhance perceptual aspects of an image for human observationl preprocessing to aid a vision system achieve its goal
n Techniques tend to be simple, ad-hoc, and qualitativen Not universally applicable
l results depend on image characteristicsl determined by interactive experimentation
n Can be embedded in larger vision systeml selection must be done carefullyl some techniques introduce artifacts into the image data
n Developing specialized techniques for specific applications can be tedious and frustrating.
Introduction to
Computer Vision SummarySummary
Summary
n What is noise?l Gaussian distribution
n Noise reductionl first principles
n Neighbourhoodl low-passl median
n Averaging pixels corrupted by noise cancels out the noise.
n Low-pass can blur image.
n Median may retain fine image detail that may be smoothed by averaging.
Conclusion