Niranjan Damera-Venkata (HP Labs) Thomas D. Kite (Audio Precision) Ph.D. Graduates: Niranjan...

Ph.D. Graduates: Niranjan Damera-Venkata Niranjan Damera-Venkata (HP Labs)(HP Labs) Thomas D. Kite Thomas D. Kite (Audio Precision)(Audio Precision)

Graduate Student: Vishal MongaVishal Monga

Support: HP LabsHP Labs National Science Foundation National Science Foundation

Embedded Signal Processing Laboratory

The University of Texas at Austin

Austin TX 78712-1084

http:://www.ece.utexas.edu/~bevans

Brian L. EvansBrian L. Evans

How to Make Printed and Displayed Images How to Make Printed and Displayed Images Have High Visual QualityHave High Visual Quality

UT Center for Perceptual Systems

2

OutlineOutline

• Introduction

• Grayscale halftoning for printing– Screening– Error diffusion– Direct binary search– Linear human visual system model

• Color halftoning for display– Optimal design– Linear human visual system model

• Conclusion


3

Need for Digital Image HalftoningNeed for Digital Image Halftoning

• Many devices incapable of reproducing grayscale (e.g. eight bits/pixel or gray levels from 0 to 255)– Laser and inkjet printers

– Facsimile machines

– Low-cost liquid crystal displays

• Grayscale imagery binarized for these devices

• Halftoning tries to reproduce full range of gray while preserving quality and spatial resolution– Screening methods are fast and simple

– Error diffusion gives better results on some media


4

Digital Halftoning MethodsDigital Halftoning Methods

Clutered Dot ScreeningAM Halftoning

Blue-noise MaskFM Halftoning 1993

Dispersed Dot ScreeningFM Halftoning

Green-noise HalftoningAM-FM Halftoning 1992

Error DiffusionFM Halftoning 1975

Direct Binary SearchFM Halftoning 1992


5

Screening (Masking) MethodsScreening (Masking) Methods

• Periodic array of thresholds smaller than image– Spatial resampling leads to aliasing (gridding effect)

– Clustered dot screening produces a coarse image that is more resistant to printer defects such as ink spread

– Dispersed dot screening has higher spatial resolution

– Blue noise masking uses large array of thresholds


6

current pixel

weights

3/16

7/16

5/16 1/16

+ _

_+

e(m)

b(m)x(m)

difference threshold

compute error

shape error

u(m)

)(mh

Error Diffusion

Spectrum

Grayscale Error DiffusionGrayscale Error Diffusion

• Shape quantization noise into high frequencies

• Design of error filter key to quality

• Not a screening technique

2-D sigma-delta modulation


7

1 sampledelay

Inputwords

To output device

4 2

2 2

Assume input = 1001 constant

Time Input Feedback Sum Output 1 1001 00 1001 10 2 1001 01 1010 10 3 1001 10 1011 10 4 1001 11 1100 10

Periodic

Average output = 1/4(10+10+10+11)=10014-bit resolution at DC!

Added noise

f

12 dB(2 bits)

If signal is in this band, you are better off

Simple Noise Shaping ExampleSimple Noise Shaping Example

• Two-bit output device and four-bit input words– Going from 4 bits down to 2 increases noise by ~ 12 dB

– Shaping eliminates noise at DC at expense of increased noise at high frequency.


8

Modeling Grayscale Error DiffusionModeling Grayscale Error Diffusion

• Sharpening caused by a correlated error image [Knox, 1992]

Floyd-Steinberg

Jarvis

Error images Halftones


9

Modeling Grayscale Error DiffusionModeling Grayscale Error Diffusion

• Apply sigma-delta modulation analysis to 2-D– Linear gain model for quantizer in 1-D [Ardalan and Paulos, 1988]

– Linear gain model for grayscale image [Kite, Evans, Bovik, 2000]

– Signal transfer function (STF), noise transfer function (NTF)

Ks

us(m)

Signal Path

Ks us(m)

un(m)

+

n(m)

Kn un(m) + n(m)

Noise Path

)(1)(

zz

zH

N

BNTF n

zz

z

HK

K

X

BSTF

s

ss

11)(

Q(.)u(m) b(m) {

1 – H(z) is highpass so H(z) is lowpass


10

Problems with Error DiffusionProblems with Error Diffusion

• Objectionable artifacts– Scan order affects results– “Worminess” visible in constant graylevel areas

• Image sharpening– Error filters due to [Jarvis, Judice & Ninke, 1976] and [Stucki, 1980]

reduce worminess and sharpen edges– Sharpening not always desirable: may be adjustable by

prefiltering based on linear gain model [Kite, Evans, Bovik, 2000]

• Computational complexity– Larger error filters require more operations per pixel– Push towards simple schemes for fast printing


11

Direct Binary SearchDirect Binary Search

• Minimize mean-squared error between lowpass filtered versions of grayscale and halftone images– Lowpass filter is based a linear model of the human visual

system (contrast sensitivity function)

– Iterative method that gives a practical upper bound on the achievable quality for a halftone of an original image

• Each iteration visits every pixel– At each pixel, consider changing state of the pixel (toggle)

or swapping it with each of its 8 nearest neighbors that differ in state from it

– Terminate when if no pixels are changed in an iteration


12

Direct Binary SearchDirect Binary Search

• Advantages– Significantly improved

image quality over screening and error diffusion methods

– Quality of final solution is relatively insensitive to the choice of starting point (initial halftone)

– Has application in off-line design of threshold arrays for screening methods

• Disadvantages– Computational cost and

memory usage is very high in comparison to error diffusion and screening methods

– Increase complexity makes it unsuitable for real-time applications such as printing


13

Contrast Sensitivity FunctionContrast Sensitivity Function

• Contrast needed at a particular spatial frequency for visibility– Angular dependence modeled

with a cosine function

– Modify it at low frequency to be lowpass, which gives better correlation with psychovisual results

– Useful in image quality metrics for optimization and performance evaluation


14

Color Halftoning by Error Diffusion for DisplayColor Halftoning by Error Diffusion for Display

• Input image has a vector of values (e.g. Red-Green-Blue) at each pixel– Error filter has matrix-valued coefficients

– Algorithm for adapting matrix coefficients[Akarun, Yardimci, Cetin 1997]

vectormatrix

kmekhmtk

+ _

_+

e(m)

b(m)x(m)

difference threshold

compute error

shape error

u(m)

)(mh

t(m)


15

Matrix Gain Model for the QuantizerMatrix Gain Model for the Quantizer

• Replace scalar gain with a matrix

– Noise uncorrelated with signal component of quantizer input

– Convolution becomes matrix–vector multiplication in frequency domain

12minarg

uubu

A

CCmuAmbK

Es

IK

n

zNzHIzB

n

zXIKzHIKzB1

s

Noise component of output

Signal component of output

u(m) quantizer inputb(m) quantizer output


16

Optimum Color Noise ShapingOptimum Color Noise Shaping

• Vector color error diffusion halftone model– We use the matrix gain model [Damera-Venkata and Evans, 2001]– Predicts signal frequency distortion– Predicts shaped color halftone noise

• Visibility of halftone noise depends on– Model predicting noise shaping– Human visual system model (assume linear shift-invariant)

• Formulation of design problem– Given HVS model and matrix gain model find the color

error filter that minimizes average visible noise power subject to certain diffusion constraints


17

Linear Color Vision ModelLinear Color Vision Model

• Pattern-Color separable model [Poirson and Wandell, 1993] – Forms the basis for S-CIELab [Zhang and Wandell, 1996]

– Pixel-based color transformation

B-W

R-G

B-Y

Opponentrepresentation

Spatialfiltering

E


18

Linear Color Vision ModelLinear Color Vision Model

• Undo gamma correction on RGB image

• Color separation– Measure power spectral distribution of RGB phosphor excitations

– Measure absorption rates of long, medium, short (LMS) cones

– Device dependent transformation C from RGB to LMS space

– Transform LMS to opponent representation using O

– Color separation may be expressed as T = OC

• Spatial filtering is incorporated using matrix filter

• Linear color vision model

Tmdmv

)( where )(md

is a diagonal matrix

)(md


19

Original Image

Sample Images and optimumcoefficients for sRGB monitor

available at:http://signal.ece.utexas.edu/~damera/col-vec.html


20

Optimum FilterFloyd-Steinberg


21

ConclusionsConclusions

• Design of “optimal” color noise shaping filters– We use the matrix gain model [Damera-Venkata and Evans, 2001]

• Predicts shaped color halftone noise

– HVS could be modeled as a general LSI system

– Solve for best error filter that minimizes visually weighted average color halftone noise energy

• Future work– Above optimal solution does not guarantee “optimal” dot

distributions• Tone dependent error filters for optimal dot distributions

– Improve numerical stability of descent procedure


22

Designing the Error FilterDesigning the Error Filter

• Eliminate linear distortion filtering before error diffusion

• Optimize error filter h(m) for noise shaping

Subject to diffusion constraints

where

22min mnmhImvmb

EE n

11mhm

mv

linear model of human visual system

* matrix-valued convolution

Backup Slides


23

Generalized Optimum SolutionGeneralized Optimum Solution

• Differentiate scalar objective function for visual noise shaping w/r to matrix-valued coefficients

• Write norm as trace and differentiate trace usingidentities from linear algebra

i0ih

mb

d

Ed n

2

xxx Tr

A

X

XA

d

Trd

BA

X

BXA

d

Trd

BXABXA

X

BXAX

d

Trd

ABBA

TrTr

Backup Slides


24

Generalized Optimum Solution (cont.)Generalized Optimum Solution (cont.)

• Differentiating and using linearity of expectation operator give a generalization of the Yule-Walker equations

where

• Assuming white noise injection

)()()()()()( qpsirphqvsvkirkv nnp q s

ank

)()()( mnmvma

kkmnmnkrnn )( )()( E

kvkmnmakran )( )()( E

• Solve using gradient descent with projection onto constraint set

Backup Slides


25

Implementation of Vector Color Error DiffusionImplementation of Vector Color Error Diffusion

)()()(

)()()(

)()()(

zzz

zzz

zzz

zH

bbbgbr

gbgggr

rbrgrr

HHH

HHH

HHH

Hgr

Hgg

Hgb

+

b

g

r

g

Backup Slides

Niranjan Damera-Venkata (HP Labs) Thomas D. Kite (Audio Precision) Ph.D. Graduates: Niranjan...

Documents

Transcript of Niranjan Damera-Venkata (HP Labs) Thomas D. Kite (Audio Precision) Ph.D. Graduates: Niranjan...