Why D.I.P.?Why D.I.P.?

Reasons for compressionReasons for compression– Image data need to be accessed at a different time or Image data need to be accessed at a different time or

locationlocation– Limited storage space and transmission bandwidthLimited storage space and transmission bandwidth

Reasons for manipulationReasons for manipulation– Image data might experience nonideal acquisition, Image data might experience nonideal acquisition,

transmission or display (e.g., restoration, enhancement transmission or display (e.g., restoration, enhancement and interpolation) and interpolation)

– Image data might contain sensitive content (e.g., fight Image data might contain sensitive content (e.g., fight against piracy, conterfeit and forgery) against piracy, conterfeit and forgery)

– To produce images with artistic effect (e.g., pointellism)To produce images with artistic effect (e.g., pointellism)

Reasons for analysisReasons for analysis– Image data need to be analyzed automatically in order Image data need to be analyzed automatically in order

to reduce the burden of human operators to reduce the burden of human operators – To teach a computer to “see” in A.I. tasksTo teach a computer to “see” in A.I. tasks

Beyond Image ProcessingBeyond Image Processing

The way of thinkingThe way of thinking– From art (heuristics) to science (principles)From art (heuristics) to science (principles)– The key is mathematics (I will write a separate The key is mathematics (I will write a separate

blog about the role of mathematics in DIP)blog about the role of mathematics in DIP)

The holistic viewThe holistic view– Everything is connected (recall the six-degree Everything is connected (recall the six-degree

phenomenon in social science)phenomenon in social science)

The “Google”-style re-searchThe “Google”-style re-search– Ability to search is a basic part of learning Ability to search is a basic part of learning

Image Compression

Image Manipulation

Image Analysis

Image Acquisition

Image Perception

Image Display

Image Generation

D.I.P.ThemePark

DIP is also about connecting dots – in image compression, you will see why you need to learn matrix theory and statistics

The Art of Image CompressionThe Art of Image Compression

Why are images compressible?Why are images compressible?– Redundancy in images (NOT random)Redundancy in images (NOT random)

How data compression works?How data compression works?– Probability theory and statisticsProbability theory and statistics– Shannon’s information theoryShannon’s information theory

What about the future of image What about the future of image compression?compression?– I will discuss this in my weblog and facebookI will discuss this in my weblog and facebook

(Google “(Google “what happened to Iterated Systems Incorporatedwhat happened to Iterated Systems Incorporated?”) ?”)

Shannon’s Picture on Shannon’s Picture on Communication (1948)Communication (1948)

sourceencoder

channel

sourcedecoder

source destination

Examples of source: Human speeches, photos, text messages, computer programs …

Examples of channel: storage media, telephone lines, wireless transmission …

super-channel

channelencoder

channeldecoder

The goal of communication is to move informationfrom here to there and from now to then

Lossless vs. Lossy CompressionLossless vs. Lossy Compression

Lossless: zero error toleranceLossless: zero error tolerance– No information lossNo information loss– Shannon’s Shannon’s entropyentropy formula formula– For photographic images, compression ratio is For photographic images, compression ratio is

modest (about 2:1)modest (about 2:1)

Lossy: the goal is to preserve the visual quality of Lossy: the goal is to preserve the visual quality of imagesimages– Information loss Information loss visuallyvisually acceptable acceptable– Shannon’s rate-distortion functionShannon’s rate-distortion function– For photographic images, compression ratio is For photographic images, compression ratio is

typically around 10-100typically around 10-100

Popular Lossless Image Compression Techniques

WinZip

- Based on the celebrated Lempel-Ziv algorithminvented nearly 30 years ago

-Based on an enhanced version of LZ algorithmby Welch in 1983-Was introduced by CompuServe in 1987 and madepopular until it was not royalty-free in 1994

GIF (Graphic Interchange Format)

PNG (Portable Network Graphics)

GIF Liberation Day: June 20, 2003

Lossy Image Compression

JPEGdecoder

original raw image (262,144 bytes)

compressed JPEG file (20,407 bytes)

decompressed image

high compression ratio

low compression ratio

low quality

high quality

QQ100

0

From JPEG to JPEG2000

JPEG (CR=64) JPEG2000 (CR=64)

discrete cosine transform based wavelet transform based

Image Compression

Image Manipulation

Image Analysis

Image Acquisition

Image Perception

Image Display

Image Generation

D.I.P.ThemePark

DIP is also about connecting dots – in image manipulation, you will see why you need to learn calculus and Fourier transform

salt and pepper (impulse) noise

Image Manipulation (I): Noise Removal

Noise contamination is often inevitable during the acquisition

additive white Gaussian noise

You will learn how to design image filter in a principled way

License plate is barely legible due to motion blurring

Image Manipulation (II): Deblurring

You will learn the use of FT and the necessity of regularization

overly-exposed image

Image Manipulation (III): Contrast Enhancement

under-exposed image

You will learn how to modify the histogram of an image

Example: aliasing artifacts in MRI image acquisition

Tradeoff between scanning time and image quality(image reconstruction is covered by EE425)

Ideal quality, slow scanning

nonideal quality,fast scanning

Image Manipulation (IV): Aliasing Reduction

small

large

digital zooming

1M pixels

4M pixels

Resolution enhancement can be obtained by common imageprocessing software such as Photoshop or Paint Shop Pro

Image Manipulation (V): Image Interpolation

You will learn the difference between digital and optical zooming

F.Y.I.: search “Gigapixel images” by Google

http://triton.tpd.tno.nl/gigazoom/Delft2.htm

=+

Merge multiple images of the same scene into one with larger FOV

Image Manipulation (VI): Image Mosaicing

There exist several mosaicing software for automatic stitching

blocks contaminated by channel errors(this problem is covered in EE565)

Image Manipulation (VII): Error Concealment

Block artifacts

Image Manipulation (VIII): Deblocking/Deringing

Ringing artifactsYou will learn how to suppress those artifacts by nonlinear diffusion

jittering noise (you will see it in either bonus assignment or final project)

Image Manipulation (IX): Dejittering

Image Manipulation (X): Image Inpainting

You will learn how to do image inpainting in EE565

Image Inpainting Application: Restore Old Photos

25,680 colors (24 bits) 256 colors (8 bits)

Applications: video cell-phone, gameboy, portable DVD

Image Manipulation (XI): Color Quantization

grayscale: 0-255 halftoned: 0/255

Image Manipulation (XII): Image Halftoning

You will learn the famous Floyd-Steinberg diffusion in CA

original scrambled

Image Manipulation (XIII): Image Scrambling/Hashing*

Original image Modified image

Image Manipulation (XIV): Image Watermarking*

Image Manipulation (XV): Image Stylization*

Abysscomputergenerated

Image Manipulation (XVI): Image Rendering*

Image-based Rendering*

Image Compression

Image Manipulation

Image Analysis

Image Acquisition

Image Perception

Image Display

Image Generation

D.I.P.ThemePark

DIP is also about connecting dots – in image analysis, you will see why you need to know about neuroscience and psychology

Image Analysis (I): Edge Detection

You will learn basic edge detectors based on derivatives

Image Analysis (II): Face Detection

Deceivingly simple for humans but notoriously difficult for machines

Image Analysis (III): Change Detection

Change Detection in Medical Application

Image Analysis (IV): Image Matching

Antemortem dental X-ray record Postmortem dental X-ray record

Image Matching in Biometrics

Two deceivingly similar fingerprints of two different people

Image Analysis (V): Image Segmentation

License number can be automaticallyextracted from the image of license plate

Image Analysis (VI): Object Recognition

Object Recognition in Military Applications

Image-based monitoring system prevents drowning

Image Analysis (VII): Event Recognition

Only send out “important” motion pictures such as home-runs

Image Analysis (VIII): Video Summarization

retrieved building images

Image Analysis (IX): Content-based Image Retrieval

SummarySummary

In EE465, you will learnIn EE465, you will learn– Image compression: Lempel-Ziv, Huffman coding, run-length Image compression: Lempel-Ziv, Huffman coding, run-length

coding and JPEGcoding and JPEG– Image manipulation: linear/nonlinear filtering, histogram-based Image manipulation: linear/nonlinear filtering, histogram-based

processing, linear interpolationprocessing, linear interpolation– Image analysis: edge/corner detection, circle/ line/ellipse Image analysis: edge/corner detection, circle/ line/ellipse

detection, chain codes/shape numbersdetection, chain codes/shape numbers

In EE565, you will learnIn EE565, you will learn– Advanced algorithms/techniques with stronger mathematical Advanced algorithms/techniques with stronger mathematical

emphasisemphasis

Not covered by the courses I offeredNot covered by the courses I offered– CBIR (multimedia database), face/pedestrian detection CBIR (multimedia database), face/pedestrian detection

(Advanced biometrics), PDE-based image processing (medical (Advanced biometrics), PDE-based image processing (medical image analysis) image analysis)

General Cooking RecipeGeneral Cooking Recipe

MotivationsMotivationsProblem statementProblem statementHeuristic observationsHeuristic observationsExamples/IllustrationsExamples/IllustrationsPrincipled approaches (via mathematics)Principled approaches (via mathematics)MATLAB implementationsMATLAB implementationsComputer assignment reviewsComputer assignment reviewsCurrent state-of-the-art and future Current state-of-the-art and future directionsdirections