Image Processing on GPU
Transcript of Image Processing on GPU
![Page 1: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/1.jpg)
Image Image ProcessingProcessing
on GPUon GPU
![Page 2: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/2.jpg)
Image ProcessingImage Processing• Define new images from existing image
for different purposes• Used for texture preprocessing for 3D
graphics and visualization• Simple processing:
– Transform each pixel independently• Pixel to pixel operations• Example: RGB image to grey scale image
– Move pixels inside without modifying intensities
• Complex processing– Many advanced technologies for different
applications
![Page 3: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/3.jpg)
Noise ReductionNoise Reduction• An important application for image
processing• Important in many areas• Sources:
–Digital camera devices–Scanned documents–Satellite images–Medical imaging–…
![Page 4: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/4.jpg)
Example Noise TypesExample Noise Types• Salt and pepper:
random white and black pixels
• Impulse: random white pixels
• Gaussian noise: variation in intensity from Gaussian distribution
• …• We need to “SMOOTH”
the image to reduce noises
![Page 5: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/5.jpg)
Linear FiltersLinear Filters• Construct a new but same-size image
from original one• Fill new pixel with a weighted sum of
old pixels -- filtering• Using the same weights for each pixel• Linear filtering
–Output for the sum of tow images is the same as the sum of the outputs obtained for the images separately
![Page 6: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/6.jpg)
ConvolutionConvolution• Filter kernel: the pattern of weights• Convolution: the process of applying
the filter on an image• Mathematically:
![Page 7: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/7.jpg)
SmoothingSmoothing• Reduce noise• Replace each pixel with a weighted
average of its neighbors• Local blur effects to remove noise• Low frequency filter to remove high
frequency noise
![Page 8: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/8.jpg)
Averaging SmoothAveraging Smooth• (2k+1)*(2k+1) region
• Poor blurring• Too simple
∑ ∑+
−=
+
−=+=
ki
kiu
kj
kjvuvij F
kR 2)12(
1
![Page 9: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/9.jpg)
Gaussian SmoothingGaussian Smoothing• Symmetric Gaussian kernel
– Weights distributed with Gaussian function– Sigma: standard deviation
– In applications, discrete kernel used:
)2
)(exp(2
1),( 2
22
2 σπσσyxyxG +
−=
)2
))1()1((exp(2
12
22
2 σπσ−−+−−
−=kjkiHij
![Page 10: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/10.jpg)
Gaussian Lookup TableGaussian Lookup Table–A lookup table can be pre-generated to
accelerate the computation, such as
![Page 11: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/11.jpg)
Gaussian KernelGaussian Kernel• If standard deviation is very small
(smaller than one pixel), the smoothing have very little effect
• For a larger one, the average will be strongly biased toward a consensus of the neighbors. Noise will largely disappear, at the cost of some blurring
• Finally, if it is very large, much of detail disappear along with noise
![Page 12: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/12.jpg)
Why use GaussianWhy use Gaussian• Not the only smoothing filter• Convenient to use:
– Efficiency– Separable
– Performed separately for x, y, z dimensions
22 2121 σσσσ +=∗ GGG
![Page 13: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/13.jpg)
Boundary EffectsBoundary Effects• In practical system, image has
boundaries• When computing convolution, need
to handle boundaries where the neighbors do not exist– Ignore these locations-image may
shrink–Pad the image with constant values–Pad with other methods: periodical …
![Page 14: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/14.jpg)
ExampleExample
• Robert J. Woodham, UBC
![Page 15: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/15.jpg)
Edge DetectionEdge Detection• Edges are important information on
images–Object detection …–Large gradient– Intensity derivatives
• Edge detection: estimate derivative and an image represented by a discrete set of pixels
![Page 16: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/16.jpg)
What is an edgeWhat is an edge
• Easy for human• Difficult for computer
![Page 17: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/17.jpg)
Estimate DerivativeEstimate Derivative• Finite difference
• The similar convolution kernel is
• Gradient vector
jiji hhxh
,1,1 −+ −≈∂∂
⎪⎭
⎪⎬
⎫
⎪⎩
⎪⎨
⎧−=000101
000G
⎟⎟⎠
⎞⎜⎜⎝
⎛∂∂
∂∂
=∇yf
xfyxf ,),(
![Page 18: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/18.jpg)
SobelSobel KernelKernel• A popular gradient magnitude
computation
![Page 19: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/19.jpg)
Second DerivativesSecond Derivatives• Sobel operator can produce thick edges• Ideally, we want thin boundaries• Alternative approach:
– Looking for local extrema in first derivative– Where the change in gradient is highest
![Page 20: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/20.jpg)
2D 2D LaplacianLaplacian• An equivalent measure of second
derivative
![Page 21: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/21.jpg)
Edge detection cg codeEdge detection cg code• Laplacian edge detection
static const char *edgeFragSource = {"uniform sampler2D texUnit;"
"void main(void)""{"" const float offset = 1.0 / 512.0;"" vec2 texCoord = gl_TexCoord[0].xy;"" vec4 c = texture2D(texUnit, texCoord);"" vec4 bl = texture2D(texUnit, texCoord + vec2(-offset, -offset));"" vec4 l = texture2D(texUnit, texCoord + vec2(-offset, 0.0));"" vec4 tl = texture2D(texUnit, texCoord + vec2(-offset, offset));"" vec4 t = texture2D(texUnit, texCoord + vec2( 0.0, offset));"" vec4 ur = texture2D(texUnit, texCoord + vec2( offset, offset));"" vec4 r = texture2D(texUnit, texCoord + vec2( offset, 0.0));"" vec4 br = texture2D(texUnit, texCoord + vec2( offset, offset));"" vec4 b = texture2D(texUnit, texCoord + vec2( 0.0, -offset));"" gl_FragColor = 8.0 * (c + -0.125 * (bl + l + tl + t + ur + r + br + b));""}"
};
![Page 22: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/22.jpg)
Results not satisfiedResults not satisfied• Noise are emphasized
![Page 23: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/23.jpg)
How to improve?How to improve?• Add smoothing filter• Smoothing a differentiated image
tends to get good result we want• Smooth the edge detected due to
noises
![Page 24: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/24.jpg)
Improved ResultsImproved Results
![Page 25: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/25.jpg)
Derivative of Gaussian FilterDerivative of Gaussian Filter• The improvement is implemented as:
Gaussian smoothing first and then use differential filter
• This can be replaced by convolution with the derivative of Gaussian
xIG
∂∗∂ )( σ
Ix
Gx
IG∗
∂∂
=∂∗∂ σσ )(
![Page 26: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/26.jpg)
Steps in edge detectionSteps in edge detection• Typical steps:
–Filtering: cut down on noise–Enhancement: amplify the difference
between edges and non-edges–Detection: use threshold–More: estimate geometry and other
operations
![Page 27: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/27.jpg)
Image Processing on GPUImage Processing on GPU• Processing an image is fairly simple and
straightforward. • The filter renders a screen-aligned quad
into an invisible pixel buffer.– The screen-aligned quad has the input image
bound as a texture. – For each rendered pixel, a Cg fragment
program is executed, which does the actual image processing in a local neighborhood.
– Because the quad is rendered using the input image as a texture, the Cg program can access the pixels in the input image though simple texture reads.
![Page 28: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/28.jpg)
Convolution on GPUConvolution on GPU
![Page 29: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/29.jpg)
Typical GPU FrameworkTypical GPU Framework• Load Sources• Apply filters as cg
programs• Render image on
screen• Must also provide
operators for saving images
![Page 30: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/30.jpg)
An Example: An Example: ScotopicScotopic FilterFilter• Vision under reduced illumination,
– lose most of our ability to see color– Scenes appear blurred and shaded in blue
• Blur use Gaussian filter• Blue shaded use night-vision filter• Then edge detection• This procedure mimic the human night
vision system
• GPU implementation enable immediate processing, useful for many real applications, such as military
![Page 31: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/31.jpg)
ExampleExample
![Page 32: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/32.jpg)
More Techniques and ExamplesMore Techniques and Examples
![Page 33: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/33.jpg)
Image GlowImage Glow
![Page 34: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/34.jpg)
Add Glow on GPUAdd Glow on GPU• (a) The scene is rendered normally.• (b) A rendering of glow sources is blurred to
create • (c) a glow texture, which is added to the ordinary
scene to produce • (d) the final glow effect.
![Page 35: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/35.jpg)
Glow on GPUGlow on GPU• The alpha channel of an object's
texture can be used to specify glow sources and the glow source brightness.
![Page 36: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/36.jpg)
Color ControlColor Control• Color correction is part of almost all print
and film imaging applications. – move color images from one color space to
another (say, from Adobe RGB to sRGB); – to stylize images (creating illusions such as
faded film, cross-processing, or other stylistic variations);
– to combine art elements from different sources (such as matching color palettes between different video sources, or models made by different artists);
– to give broad coherence and mood to entire parts of a game or scene without changing the underlying models and textures.
![Page 37: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/37.jpg)
General IdeaGeneral Idea• Change color of individual pixels
– Per channel: alter red, green, and blue components individually
– color-mixing: each output channel may be an operation based on the red, green, and blue components simultaneously
• The mathematics of color corrections can be compactly and easily described in a GPU shader.
• Just as important, they can be controlled effectively with common tools used widely by computer artists and programmers. – E.g.Relying on Adobe Photoshop to let artists
create control resources, which can then be applied in real time via pixel shaders.
![Page 38: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/38.jpg)
Gamma AdjustmentGamma Adjustment• Adjusting Image Gamma and Overall Dynamic
Range in photoshop• Cg program implement: • outPixel = (pow(((inPixel * 255.0) - inBlack) / (inWhite - inBlack), inGamma) *
(outWhite - outBlack) + outBlack) / 255.0;
![Page 39: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/39.jpg)
Curved Level AdjustmentCurved Level Adjustment• Curves tool in Photoshop provides arbitrary remapping of
the color channels
![Page 40: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/40.jpg)
Curved Level AdjustmentCurved Level Adjustment• A 1x256 texture map should be defined,
although smaller maps can often be used effectively
• Before creating the map, the artist must first define the color transformation, typically by applying adjustment layers to existing still images using the Curves tool in Photoshop.
• Create a ramp texture from an original white-black ramp according to the color transformation
![Page 41: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/41.jpg)
Curved Level AdjustmentCurved Level Adjustment• Cg code• float3 InColor = tex2D(inSampler,
IN.UV).xyz; • float3 OutColor; • OutColor.r = tex1D(ColorCorrMap,
InColor.r).r; • OutColor.g = tex1D(ColorCorrMap,
InColor.g).g; • OutColor.b = tex1D(ColorCorrMap,
InColor.b).b;
• Channel-by-Channel Results of the Red, Green, and Blue Remappings
![Page 42: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/42.jpg)
PDE and Image ProcessingPDE and Image Processing• We introduce simple local pixel
filtering for image processing• Some more complex image
processing techniques involves PDEs• Thus, GPU-based PDE solver is very
important– Image diffusion– Image editing–…
![Page 43: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/43.jpg)
Repeated Smoothing: DiffusionRepeated Smoothing: Diffusion• Usually we need to repeat smoothing
operation• Such operation actually is a solution to
the diffusion equation
Φ∇=∂Φ∂
+∂Φ∂
=∂Φ∂ 2
2
2
2
2
yxt
![Page 44: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/44.jpg)
Anisotropic DiffusionAnisotropic Diffusion• Edge preserving smoothing• Use a function g to control diffusion• Smoothing noise while conserve edges
)),,(( Φ∇⋅∇=∂Φ∂ σyxct
![Page 45: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/45.jpg)
Diffusion FunctionDiffusion Function• c() is the diffusion function • A monotonically decreasing function
of the image gradient magnitude • Generally g() can be implemented
![Page 46: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/46.jpg)
Gaussian Diffusion FunctionGaussian Diffusion Function• g() can be implemented by the
function of derivative of Gaussian distribution
• Smoothing noise while preserving edges
![Page 47: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/47.jpg)
DemosDemos• Copyright by Joachim Weickert
![Page 48: Image Processing on GPU](https://reader036.fdocuments.in/reader036/viewer/2022071602/613d6437736caf36b75cc769/html5/thumbnails/48.jpg)
How to solve the equation?How to solve the equation?• Typical Partial Differential equation• Basic finite difference operations to
discretize the equation• Thus, a linear system is achieved
–Show how?
• How to use GPU solving linear system?–Linear algebra on GPU