Motivation - University of Hong Kong
Transcript of Motivation - University of Hong Kong
Image Intensity and Point Operations
Dr. Edmund Lam
Department of Electrical and Electronic EngineeringThe University of Hong Kong
ELEC4245: Digital Image Proessing(Second Semester, 2015–16)
http://www.eee.hku.hk/˜elec4245
E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 1 / 43
Motivation
A digital image is
a matrix of numbers, each corresponding to certain brightness.
E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 2 / 43
Motivation
A/D converter D/A convertersensor displaydigital image
finite dynamic range finite representation finite dynamic range
These numbersare called intensity, or gray levelsmust be nonnegativemust fall within a range of discrete values (dynamic range)are measured by the number of bits.
E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 3 / 43
Gray Levels
How many gray levels are “enough”?Often, 8-bit. 28 = 256 levelsYour computer likes it: 8 bits = 1 byteFor an image of size X × Y, your computer can store it with XYbytes (each pixel needs 1 byte to store its intensity). In reality, weneed much less, due to compression.
But other values exist:Printing: We may only have 1-bit (“ink” or “no ink” at a specificlocation)High dynamic range (HDR) imaging: With better sensors anddisplays, we may record and show a wider rangeWhat’s the limit? Our eyes can see about 14 orders of magnitude!
E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 4 / 43
Gray Levels
8-bits 7-bits 6-bits 5-bits
4-bits 3-bits 2-bits 1-bit
E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 5 / 43
Point operations
1 Point operations
2 Image histogram
3 Combining images
E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 6 / 43
Point operations
Gray level mapping
We will focus on discussing gray level images as the notations andconcepts are much easier to understand. For color images, we canalways perform such operations on the luminance channel. (morelater)
Let the input image be represented by Iin(x, y). We process theimage, and the output is represented by Iout(x, y).The simplest kind of processing is a point-wise operation:
Iout(x, y) = T {Iin(x, y)
}
where Tcan be a one-to-one mapping (reversible)can be a many-to-one mapping (irreversible)cannot be a one-to-many mapping
For every pixel, we change the intensity from value input to output
E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 7 / 43
Point operations
Gray-level mapping
The algorithm can be represented by an input-output plot
input intensity
output intensity
It can usually be implemented as a look-up table (LUT) for maximumefficiency.
E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 8 / 43
Point operations
Gray-level mapping
LUT is most flexible. But conceptually, let’s consider formulas:
Iout(x, y) =
{0 Iin(x, y) < T
255 Iin(x, y) ≥ T Threshold (1)
Iout(x, y) = 255 − Iin(x, y) Negative (2)
Iout(x, y) = c log[1 + Iin(x, y)
]Logarithm (3)
Iout(x, y) = c[Iin(x, y)
]γ Power-law (4)
Pick c and γ so that Iout(x, y) is within [0, 255].
E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 9 / 43
Point operations
Gray-level mapping
input
output
Thresholdinput
output
Negative
input
output
Logarithminput
output
Power-law
E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 10 / 43
Point operations
Threshold
Original Modified
E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 11 / 43
Point operations
Threshold
Output is a binary imageT can be set at the mid-point of the intensity range (i.e., 128), butany other number is also fine.Theoretically, we lost 7/8 of the total information! Butsurprisingly, we retain most of the useful information.Thresholding is often used as part of a computer vision process,e.g., in pattern recognition or defect detection.
E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 12 / 43
Point operations
Negative
Original Modified
Not used often. For “ordinary” images it would look funny.
More useful for images we don’t normally see, such as medical images.E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 13 / 43
Point operations
Logarithm
Original Modified
E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 14 / 43
Point operations
Power-law
Example (γ = 1.5):
Original Modified
E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 15 / 43
Point operations
Bit-plane slicing
Represent each pixel value in binary, and then create a binary imagefor each bit. Each such image is called a bit-plane.
1 0 1 1 0 1 0 0180 =
0 0 1 1 0 1 0 153 =...
plane 8 plane 1
Plane 8 is “most significant” while plane 1 is “least significant”E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 16 / 43
Point operations
Bit-plane slicing
Bit 8 Bit 7 Bit 6 Bit 5
Bit 4 Bit 3 Bit 2 Bit 1
E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 17 / 43
Point operations
Multiple bit-planes
Bit 8 Bit 8–7 Bit 8–6 Bit 8–5
Bit 8–4 Bit 8–3 Bit 8–2 Bit 8–1
E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 18 / 43
Point operations
Application: Watermarking
Replace bit-plane 1 with another binary image as digital “watermark”
Bit 8–2 Bit 1
+ =
Slicing this image:
Bit 8 Bit 7 Bit 6 Bit 5 Bit 4 Bit 3 Bit 2 Bit 1
E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 19 / 43
Point operations
Application: Watermarking
This is one form of digital watermarking: hiding information digitallyOften used for authentication: for example, to show that a certainpicture is owned by youA fancy word — steganography: art or practice of concealing amessage, image, or file within another message, image, or fileThe method using bit-plane slicing is simple, easy to implement,and the watermark is easy to detectDrawback: the watermark is not robust: it can easily be destroyedor replacedThere are much more sophisticated schemes
E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 20 / 43
Image histogram
1 Point operations
2 Image histogram
3 Combining images
E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 21 / 43
Image histogram
Histogram
Each pixel has a value (intensity). By collecting all the pixels together,we can form a histogram.
The spatial information is lost!The histogram can give us a vague idea of the intensityconcentrations.
E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 22 / 43
Image histogram
Histogram
Original Too dark Too bright Equalized
E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 23 / 43
Image histogram
Histogram
The histogram can be helpful to provide the “curve” for gray-levelmapping.
Histogram equalization: output image has (roughly) the samenumber of pixels of each gray level (hence “equalized”)
Good thing: make use of all available gray levels to the maximumextentReality: this is only approximate because we are not allowedone-to-many mapping (see the next example)Conceptually: (for 8-bit) lowest 1/256 intensity of all pixels map tointensity 0; next 1/256 map to intensity 1; next map to 2, etc.Mainly works when the illuminating condition has problem
E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 24 / 43
Image histogram
Histogram equalization
Example: 3-bit image, 64 × 64 pixels. Assume the following:
gray level number of pixels
0 7901 10232 8503 6564 3295 2456 1227 81
E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 25 / 43
Image histogram
Histogram equalization
1 Gray levels: [0, . . . , 7], total 4096 pixels2 Proportion of input pixels at level 0: 790/4096 ≈ 0.19. We need to
fill the entire range of 0 to 7, so such pixels should map to0.19 × 7 ≈ 1.33. Round to the nearest integer, we map them to 1.
3 Proportion of input pixels at level 0 and 1: (790 + 1023)/4096 ≈ 0.44.Level 1 should map to 0.44 × 7 ≈ 3.08→ 3.
4 Proportion of input pixels at level 0 to 2:(790 + 1023 + 850)/4096 ≈ 0.65. Level 2 should map to0.65 × 7 ≈ 4.55→ 5.
5 Similarly: Level 3→ 6, Level 4→ 6, Level 5→ 7, Level 6→ 7,Level 7→ 7
E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 26 / 43
Image histogram
Histogram equalization
Generally,Assume L levels, and j = 0, . . . ,L− 1; image is of size M×N. Let n jdenote the number of pixels at level j.We compute, for each k,
sk =L − 1MN
k∑
j=0
n j k = 0, 1, . . . ,L − 1 (5)
so each sk is the ideal output level for an input level k.We are limited to integer output levels, so we quantize sk.
E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 27 / 43
Image histogram
Histogram equalization
input intensity
count
input intensity
output intensity
00
1
1
2
2
3
3
4
4
5
5
6
6
7
7
output intensity
count
Note that output histogram is roughly flat, but not strictly.E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 28 / 43
Image histogram
Histogram equalization
Research: What is undesirable, and how to improve the algorithm?
Original Histogram equalization
E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 29 / 43
Image histogram
Adaptive histogram equalization
Modification: adaptive histogram equalizationHistogram equalization based on a portion of the image, e.g.,every non-overlapping 16 × 16 block (tile). Limit contrastexpansion in flat regions by clipping values. Smooth blending(bilinear interpolation) between neighboring tiles.
Original Global equalization Adaptive equalization
E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 30 / 43
Combining images
1 Point operations
2 Image histogram
3 Combining images
E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 31 / 43
Combining images
Pointwise operations
We can perform point-by-point (also known as pointwise) operations tocombine several images.
Assume the images are of the same size:
addition: I(x, y) = a(x, y) + b(x, y)
subtraction: I(x, y) = a(x, y) − b(x, y)
multiplication: I(x, y) = a(x, y) · b(x, y)
division: I(x, y) =a(x, y)b(x, y)
E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 32 / 43
Combining images
Addition and averaging
Average images to reduce noise
1 image 8 images 32 images
E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 33 / 43
Combining images
Addition and averaging
Assume each image is corrupted by additive white Gaussian noise:
fi(x, y) = g(x, y) + ni(x, y) (6)
g(x, y) is the “ideal” noise-free imagefi(x, y) is what we capture (subscript i to denote the ith one)ni(x, y) is the noise. Every pixel of the noise follows a Gaussiandistribution with mean zero and the same standard deviation σ. Thestandard deviation (or variance σ2) of the noise indicates howsevere the image is corrupted.We use the expected value E, such that
E[ni(x, y)] = 0 (7)
E[n2i (x, y)] = σ2 (8)
E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 34 / 43
Combining images
Addition and averaging
Example
σ2 = 0.001 × 2552 σ2 = 0.01 × 2552 σ2 = 0.1 × 2552
E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 35 / 43
Combining images
Addition and averaging
Assume we now have K images, f1(x, y), . . . , fK(x, y)
f̃ (x, y) =1K
K∑
i=1
fi(x, y)
Noise in one image:
E[( f1(x, y) − g(x, y))2] = E[n21(x, y)] = σ2 (9)
E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 36 / 43
Combining images
Addition and averaging
Noise in the averaged image:
E[( f̃ (x, y) − g(x, y))2] = E[(1K
K∑
i=1
fi(x, y) − 1K
K∑
i=1
g(x, y))2] (10)
= E[(1K
K∑
i=1
ni(x, y))2] (11)
=1
K2
K∑
i=1
E[(ni(x, y))2] (12)
=1
K2 · K · σ2 =1Kσ2 (13)
Eq. (12) is valid provided E[nin j] = 0 when i , j
E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 37 / 43
Combining images
Subtraction
Spot the difference:
No defect, f1(x, y) With defect f2(x, y)
E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 38 / 43
Combining images
Subtraction
Take the difference: | f1(x, y) − f2(x, y)|No alignment Properly aligned and thresholded
Research: How to align?E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 39 / 43
Combining images
Multiplication
We can think about how an image is formed: (the imaging process)
f (x, y) = i(x, y)r(x, y) (14)
i(x, y) is the illumination source: 0 < i(x, y) < ∞r(x, y) is the reflectance: 0 < r(x, y) < 1Some images are formed with transmission (e.g. x-ray), thenr(x, y) is the transmissivityf (x, y) are confined to the available dynamic range when capturedby a detector
E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 40 / 43
Combining images
Other combinations
High dynamic range (HDR) imaging: combining images from differentexposures
E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 41 / 43
Combining images
Other combinations
Removing occlusion
−→
(source: Herley, “Automatic occlusion removal from minimum number ofimages,” ICIP 2005)
E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 42 / 43
Combining images
Summary
We looked at image enhancement with one or more images as input.
We consider each pixel location as unrelated to its neighbors.Next: We look at image processing that involves the neighborpixels.
E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 43 / 43