Why D.I.P.?
43
Why D.I.P.? Why D.I.P.? Reasons for compression Reasons for compression – Image data need to be accessed at a different time Image data need to be accessed at a different time or location or location – Limited storage space and transmission bandwidth Limited storage space and transmission bandwidth Reasons for manipulation Reasons for manipulation – Image data might experience nonideal acquisition, Image data might experience nonideal acquisition, transmission or display (e.g., restoration, transmission or display (e.g., restoration, enhancement and interpolation) enhancement and interpolation) – Image data might contain sensitive content (e.g., Image data might contain sensitive content (e.g., fight against piracy, conterfeit and forgery) fight against piracy, conterfeit and forgery) – To produce images with artistic effect (e.g., To produce images with artistic effect (e.g., pointellism) pointellism) Reasons for analysis Reasons for analysis – Image data need to be analyzed automatically in Image data need to be analyzed automatically in order to reduce the burden of human operators order to reduce the burden of human operators – To teach a computer to “see” in A.I. tasks To teach a computer to “see” in A.I. tasks
description
Why D.I.P.?. Reasons for compression Image data need to be accessed at a different time or location Limited storage space and transmission bandwidth Reasons for manipulation - PowerPoint PPT Presentation
Transcript of Why D.I.P.?
Why D.I.P.?Why D.I.P.?
Reasons for compressionReasons for compression– Image data need to be accessed at a different time or Image data need to be accessed at a different time or
locationlocation– Limited storage space and transmission bandwidthLimited storage space and transmission bandwidth
Reasons for manipulationReasons for manipulation– Image data might experience nonideal acquisition, Image data might experience nonideal acquisition,
transmission or display (e.g., restoration, enhancement transmission or display (e.g., restoration, enhancement and interpolation) and interpolation)
– Image data might contain sensitive content (e.g., fight Image data might contain sensitive content (e.g., fight against piracy, conterfeit and forgery) against piracy, conterfeit and forgery)
– To produce images with artistic effect (e.g., pointellism)To produce images with artistic effect (e.g., pointellism)
Reasons for analysisReasons for analysis– Image data need to be analyzed automatically in order Image data need to be analyzed automatically in order
to reduce the burden of human operators to reduce the burden of human operators – To teach a computer to “see” in A.I. tasksTo teach a computer to “see” in A.I. tasks
Beyond Image ProcessingBeyond Image Processing
The way of thinkingThe way of thinking– From art (heuristics) to science (principles)From art (heuristics) to science (principles)– The key is mathematics (I will write a separate The key is mathematics (I will write a separate
blog about the role of mathematics in DIP)blog about the role of mathematics in DIP)
The holistic viewThe holistic view– Everything is connected (recall the six-degree Everything is connected (recall the six-degree
phenomenon in social science)phenomenon in social science)
The “Google”-style re-searchThe “Google”-style re-search– Ability to search is a basic part of learning Ability to search is a basic part of learning
Image Compression
Image Manipulation
Image Analysis
Image Acquisition
Image Perception
Image Display
Image Generation
D.I.P.ThemePark
DIP is also about connecting dots – in image compression, you will see why you need to learn matrix theory and statistics
The Art of Image CompressionThe Art of Image Compression
Why are images compressible?Why are images compressible?– Redundancy in images (NOT random)Redundancy in images (NOT random)
How data compression works?How data compression works?– Probability theory and statisticsProbability theory and statistics– Shannon’s information theoryShannon’s information theory
What about the future of image What about the future of image compression?compression?– I will discuss this in my weblog and facebookI will discuss this in my weblog and facebook
(Google “(Google “what happened to Iterated Systems Incorporatedwhat happened to Iterated Systems Incorporated?”) ?”)
Shannon’s Picture on Shannon’s Picture on Communication (1948)Communication (1948)
sourceencoder
channel
sourcedecoder
source destination
Examples of source: Human speeches, photos, text messages, computer programs …
Examples of channel: storage media, telephone lines, wireless transmission …
super-channel
channelencoder
channeldecoder
The goal of communication is to move informationfrom here to there and from now to then
Lossless vs. Lossy CompressionLossless vs. Lossy Compression
Lossless: zero error toleranceLossless: zero error tolerance– No information lossNo information loss– Shannon’s Shannon’s entropyentropy formula formula– For photographic images, compression ratio is For photographic images, compression ratio is
modest (about 2:1)modest (about 2:1)
Lossy: the goal is to preserve the visual quality of Lossy: the goal is to preserve the visual quality of imagesimages– Information loss Information loss visuallyvisually acceptable acceptable– Shannon’s rate-distortion functionShannon’s rate-distortion function– For photographic images, compression ratio is For photographic images, compression ratio is
typically around 10-100typically around 10-100
Popular Lossless Image Compression Techniques
WinZip
- Based on the celebrated Lempel-Ziv algorithminvented nearly 30 years ago
-Based on an enhanced version of LZ algorithmby Welch in 1983-Was introduced by CompuServe in 1987 and madepopular until it was not royalty-free in 1994
GIF (Graphic Interchange Format)
PNG (Portable Network Graphics)
GIF Liberation Day: June 20, 2003
Lossy Image Compression
JPEGdecoder
original raw image (262,144 bytes)
compressed JPEG file (20,407 bytes)
decompressed image
high compression ratio
low compression ratio
low quality
high quality
QQ100
0
From JPEG to JPEG2000
JPEG (CR=64) JPEG2000 (CR=64)
discrete cosine transform based wavelet transform based
Image Compression
Image Manipulation
Image Analysis
Image Acquisition
Image Perception
Image Display
Image Generation
D.I.P.ThemePark
DIP is also about connecting dots – in image manipulation, you will see why you need to learn calculus and Fourier transform
salt and pepper (impulse) noise
Image Manipulation (I): Noise Removal
Noise contamination is often inevitable during the acquisition
additive white Gaussian noise
You will learn how to design image filter in a principled way
License plate is barely legible due to motion blurring
Image Manipulation (II): Deblurring
You will learn the use of FT and the necessity of regularization
overly-exposed image
Image Manipulation (III): Contrast Enhancement
under-exposed image
You will learn how to modify the histogram of an image
Example: aliasing artifacts in MRI image acquisition
Tradeoff between scanning time and image quality(image reconstruction is covered by EE425)
Ideal quality, slow scanning
nonideal quality,fast scanning
Image Manipulation (IV): Aliasing Reduction
small
large
digital zooming
1M pixels
4M pixels
Resolution enhancement can be obtained by common imageprocessing software such as Photoshop or Paint Shop Pro
Image Manipulation (V): Image Interpolation
You will learn the difference between digital and optical zooming
F.Y.I.: search “Gigapixel images” by Google
http://triton.tpd.tno.nl/gigazoom/Delft2.htm
=+
Merge multiple images of the same scene into one with larger FOV
Image Manipulation (VI): Image Mosaicing
There exist several mosaicing software for automatic stitching
blocks contaminated by channel errors(this problem is covered in EE565)
Image Manipulation (VII): Error Concealment
Block artifacts
Image Manipulation (VIII): Deblocking/Deringing
Ringing artifactsYou will learn how to suppress those artifacts by nonlinear diffusion
jittering noise (you will see it in either bonus assignment or final project)
Image Manipulation (IX): Dejittering
Image Manipulation (X): Image Inpainting
You will learn how to do image inpainting in EE565
Image Inpainting Application: Restore Old Photos
25,680 colors (24 bits) 256 colors (8 bits)
Applications: video cell-phone, gameboy, portable DVD
Image Manipulation (XI): Color Quantization
grayscale: 0-255 halftoned: 0/255
Image Manipulation (XII): Image Halftoning
You will learn the famous Floyd-Steinberg diffusion in CA
original scrambled
Image Manipulation (XIII): Image Scrambling/Hashing*
Original image Modified image
Image Manipulation (XIV): Image Watermarking*
Image Manipulation (XV): Image Stylization*
Abysscomputergenerated
Image Manipulation (XVI): Image Rendering*
Image-based Rendering*
Image Compression
Image Manipulation
Image Analysis
Image Acquisition
Image Perception
Image Display
Image Generation
D.I.P.ThemePark
DIP is also about connecting dots – in image analysis, you will see why you need to know about neuroscience and psychology
Image Analysis (I): Edge Detection
You will learn basic edge detectors based on derivatives
Image Analysis (II): Face Detection
Deceivingly simple for humans but notoriously difficult for machines
Image Analysis (III): Change Detection
Change Detection in Medical Application
Image Analysis (IV): Image Matching
Antemortem dental X-ray record Postmortem dental X-ray record
Image Matching in Biometrics
Two deceivingly similar fingerprints of two different people
Image Analysis (V): Image Segmentation
License number can be automaticallyextracted from the image of license plate
Image Analysis (VI): Object Recognition
Object Recognition in Military Applications
Image-based monitoring system prevents drowning
Image Analysis (VII): Event Recognition
Only send out “important” motion pictures such as home-runs
Image Analysis (VIII): Video Summarization
retrieved building images
Image Analysis (IX): Content-based Image Retrieval
SummarySummary
In EE465, you will learnIn EE465, you will learn– Image compression: Lempel-Ziv, Huffman coding, run-length Image compression: Lempel-Ziv, Huffman coding, run-length
coding and JPEGcoding and JPEG– Image manipulation: linear/nonlinear filtering, histogram-based Image manipulation: linear/nonlinear filtering, histogram-based
processing, linear interpolationprocessing, linear interpolation– Image analysis: edge/corner detection, circle/ line/ellipse Image analysis: edge/corner detection, circle/ line/ellipse
detection, chain codes/shape numbersdetection, chain codes/shape numbers
In EE565, you will learnIn EE565, you will learn– Advanced algorithms/techniques with stronger mathematical Advanced algorithms/techniques with stronger mathematical
emphasisemphasis
Not covered by the courses I offeredNot covered by the courses I offered– CBIR (multimedia database), face/pedestrian detection CBIR (multimedia database), face/pedestrian detection
(Advanced biometrics), PDE-based image processing (medical (Advanced biometrics), PDE-based image processing (medical image analysis) image analysis)
General Cooking RecipeGeneral Cooking Recipe
MotivationsMotivationsProblem statementProblem statementHeuristic observationsHeuristic observationsExamples/IllustrationsExamples/IllustrationsPrincipled approaches (via mathematics)Principled approaches (via mathematics)MATLAB implementationsMATLAB implementationsComputer assignment reviewsComputer assignment reviewsCurrent state-of-the-art and future Current state-of-the-art and future directionsdirections
