EE3414 Multimedia Communication Systems – Part...

EE3414Multimedia Communication Systems –

Part I

Spring 2003Final Review

Yao WangElectrical and Computer Engineering

Polytechnic University

EE3414, S03 © Yao Wang, ECE, Polytechnic University 2

Color Principle

• How a human perceives color– Three types of cones sensitive to red, green, and blue respectively

• How to generate different colors in display– By mixing three primary colors, Red, Green and Blue– The mixing following additive rules – Know some simple mixing rule:

• R+G+B light in equal proportion = white• How to generate different colors in print

– By mixing three primary colors, Cyan, Magenta, Yellow– The mixing follows subtractive rule– Know some simple mixing rule:

• C+M+M dye in equal proportion = black• How is a color image stored

– Consists of three separate component images: ex. R,G,B– Convert to CMY+K for printing


Color Quantization

• Uniform scalar quantization– Each color component quantized independently using a uniform

quantizer• Vector quantization (adaptive quantization, adaptive palette

– Three color components quantized jointly– Finding the best set of colors in an image to represent all colors in an

image• Color dithering

– Randomly perturb quantized color pixels to reduce the contour effect• Pseudo color image display

– Difference from true color images– Usage for revealing image features


Image Contrast Enhancement

• What is a histogram?– How to compute the histogram of a given image?

• Can tell whether an image has a good contrast from its histogram• Given a histogram, can sketch a transformation that will likely

improve the image contrast.• Know the principle of histogram equalization

– The transformation function determined from the accumulated histogram

• How to enhance color images?– To the Intensity component only while keeping the Hue and

Saturation in the HSI coordinate

Histogram vs. Image Contrast

Images with figure captions in this and other slides are from [Gonzalez02]


Enhancement of Too-Dark Images

fmax

H(f)

gmax g

H(g)

fmax

gmax

f

g


Enhancement of Too-Bright Images

fmax

H(f)

gmax g

H(g)

fmax

gmax

f

g


Enhancement of Images Centered near the Middle Range

fmax

H(f)

gmax g

H(g)

fmax

gmax

f

g


Noise Removal

• How does averaging and weighted averaging filter works?– How to apply a filter mask

• How does median filter works?• What method is better for additive Gaussian noise?• What method is better for salt-and-pepper noise?• What is the challenge in noise removal?

– Trade-off between noise removal and detail preserving


Example: Weighted Average

100100100100100

100195205200100

100200200195100

100203205200100

100100100100100

100100100100100

100156175156100

100175201174100

100158176156100

100100100100100

121

242

121

×161

100100100100100

100144166144100

100168200167100

100145167144100

100100100100100

111

111

111

×91


Example: 3x3 Median

100100100100100

100195205200100

100200200195100

100203205200100

100100100100100

100100100100100

100100195100100

100200200200100

100100200100100

100100100100100

Matlab command: medfilt2(A,[3 3])


DFT

• Understand how does 1D DFT represents a vector as a sum of some basis vectors, and how to compute the DFT coefficients by inner product, and how to construct the original vector using the basis vectors and the DFT coefficients

• Understand how does 2D DFT represents an image as sum of some basis images, and how to compute the DFT coefficients by inner product, and how to construct the original image using the basis images and the DFT coefficients

• Understand the relation between the frequency (number of cycles) and the DFT index in 1D and 2D cases.

• Know how to display the 2D DFT of an image properly: shifting + log transformation


DFT Domain Filtering

• Understand how to perform filtering in the DFT domain: – 3 step: DFT-> modifying the DFT coefficients -> IDFT

• Understand how to perform low-pass, high-pass, and high-emphasis filtering in DFT domain (all based on the low-pass filter)

– How to generate high-pass and high emphasis filters from low-pass filters• Understand the relation between spatial filtering and DFT domain filtering

– Spatial convolution using h === DFT domain multiplication using H=DFT(h)


Image Resizing

• What does image down- and up-sampling mean• Need for prefiltering before down-sampling

– Understand what aliasing mean – Know what happens in 2D frequency domain after down-sampling– Know what is the ideal prefilter and what are practical prefilter

• Can perform down-sampling without and with simple averaging filter usingmatlab

• Know how different interpolation filters work– Nearest neighbor– Bilinear– Bicubic (Bicubic filter is NOT required in the final exam)

• Can perform up-sampling with different interpolation filters using matlab• Know the tradeoff between aliasing and blurring by using different

prefilters and interpolation filters


2D Sampling in Spectral Domain

From: Y. Wang, J. Ostermann, Y.Q. Zhang, Video Processing and Communications, Prentice Hall, 2001. Figure 3.4.

Sampling w/o prefiltering, leading to aliasing

Sampling with prefiltering, avoiding aliasing.


Example: Factor of 2 Up-Sampling

(x+1,y)(x,y)

(x+1,y+1)(x,y+!)

(2x,2y) (2x+1,2y)

(2x,2y+1) (2x+1,2y+1)

Green samples are retained in the interpolated image;Orange samples are estimated from surrounding green samples.

Know how to obtain the orange samples using: nearest neighbor (pixel replication) and bilinear interpolation


DCT

• Difference between DCT and DFT– Using different basis vectors/images– DCT basis images consist of real numbers only

• How to perform 2D DCT: forward and inverse transform– Manual calculation for small sizes, using inner product notation– Using Matlab: dct2, idct2

• Why DCT is good for image coding– Real transform, easier than DFT– Most high frequency coefficients are nearly zero and can be ignored– Different coefficients can be quantized with different accuracy based

on human sensitivity– You should know how to quantize DCT coefficients using a given

quantization table


Example: Solution of Quiz 9


DCT-based Image Coding

• Three steps in transform coding– Transform, quantization of DCT coefficients, run-length coding

of quantized indices• How to quantize DCT coefficients

– A quantization matrix specifies the default quantization stepsize for each coefficient

– The matrix can be scaled using a user chosen parameter (QP)– Know how to apply uniform quantization on DCT coefficients

based on a given quantization matrix• Run-length coding of quantized DCT coefficient indices

are not required in the final– DCT transform and quantization are required.


JPEG

• JPEG Processing for a gray scale image:– Divide the image into data units of 8x8 blocks– Each 8x8 block go through 3 steps:

• Transform (DCT), quantization, run-length+Huffman coding– Quality vs. bit rate trade-off can be controlled by setting a quality factor, which

scales a predefined quantization matrix.• How does JPEG handle color images

– The RGB coordinate is converted to YCbCr • How to do the conversion?

– The Cb and Cr components are typically down sampled by a factor of 2 in both horizontal and vertical directions

• Why should we down-sample chrominance components?– Each component is divided into 8x8 blocks and coded as above

• Performance of JPEG– Very good quality at 1bpp for 24 bpp color images, acceptable quality at 0.5

bpp, poor quality at 0.25 bpp or below.• JPEG2000 not required on the final


Internet Technology

• Know the different layers in the Internet and protocols associated with each layer

• Know the difference between physical address (e.g. Ethernet address) and IP address

• Understand the basic process in sending a data packet– Application packet -> TCP segment or UDP packet -> IP packet (datagram) ->

Ethernet frames -> actual bits being sent• Understand the functions of the transport layer

– Flow control: not to push too many packets onto the network, to avoid congestion

– Error control: retransmit a packet that is not received– Connection establishment and termination: through handshaking between two

terminals• Understand the difference between UDP and TCP

– UDP: no retransmission– TCP: retransmit packets that were not acknowledged by the receiver after a

certain timeout. Uses sliding window with credit allocation for flow control.


Different Layers in a Network

Application

Transport

Network

HOST

Data Link Data Link Data Link

Network

Application

Transport

Network

HOST

Data Link

Physical layer at the bottom


Different Layers in Internet

Application

TCP

IP

NetworkAccess

Application

TCP

IP

NetworkAccess

Application protocol

TCP protocol

IP protocol IP protocol

DataLink

NetworkAccess

IP

NetworkAccess

NetworkAccess

IP

NetworkAccess

DataLink

DataLink

IP protocol

RouterRouter HostHost

Application

TCP

IP

NetworkAccess

Application

TCP

IP

NetworkAccess

Application protocol

TCP protocol

IP protocol IP protocol

DataLink

NetworkAccess

IP

NetworkAccess

NetworkAccess

IP

NetworkAccess

DataLink

DataLink

IP protocol

RouterRouter HostHost

IP is the highest layer protocol that is implemented at both routers and hosts


IP Protocol Stack

NetworkLayer

Link Layer

IP

ARP NetworkAccess RARP

Media

ICMP IGMP

TransportLayer

TCP UDP


UDP vs. TCP

UDP UDP -- User Datagram User Datagram ProtocolProtocol

• datagram oriented• Unreliable (best-effort),

connectionless• simple• unicast and multicast• Low-delay, hence good for

multimedia applications• used a lot for services

– network management (SNMP), routing (RIP), naming (DNS), etc.

TCP - Transmission Control Protocol

• stream oriented, in sequence• reliable, connection-oriented• complex• only unicast• used for most Internet

applications:– web (http), email (smtp), file

transfer (ftp), terminal (telnet), etc.


Final Exam

• Time: Thursday 5/1: 11-12:50• Closed book, 1 sheet of notes allowed (double sided

OK)• Extra office hour:

– Tuesday 4/29: 4-6PM Yao Wang, LC265– Wednesday 4/30: 11 AM – 1PM, Hai Tian, LC008

EE3414 Multimedia Communication Systems – Part...

Documents

Transcript of EE3414 Multimedia Communication Systems – Part...