Color image processing challenges zewail city workshop 7 march 2015

Scientific Research Group in Egypt http://www.egyptscience.net

Image Processing: Challenges for Better Life

Noura Abd El-Moez Semary

A S S I S TA N T P R O F E S S O R ( P H . D. ) S C I E N T I F I C R E S E A R C H G R O U P I N EGY P T

FAC U LT Y O F C O M P U T E R S A N D I N F O R M AT I O N M E N O F I A U N I V E R S I T Y –


Agenda Color image principles

Colorization

Compression

Assistive Technologies

Color Blindness

Visual Impaired


Color Principles

3

Gray values

0 . .50 . .

100 . .

150 . .

200 . .

250 .

255

Red Green Blue


From Color to Gray (RGB)

4

Scientific Research Group in Egypt http://www.egyptscience.netHue SaturationLuminance

From Color to Gray (HSL)

5


Image Colorization !!


Hue Saturation Luminance

Colorization Problem

7


Colorization Problem

There are two definitions to describe the gray value as an equation of the three basic components of RGB colormodel (red, green, blue):

1: Intensity (most common used)

Gray = (Red + Green + Blue) /3

2: Luminance (NTSC standard for luminance)

Gray = (0.299 × Red) + (0.587 × Green) + (0.114 × Blue)

RGB Color R, G, B values Gray value Gray Color

100, 150, 87 128

147, 87, 149 128

149, 147, 87 128

8

Scientific Research Group in Egypt http://www.egyptscience.net9

Colorization TechniquesAutomatic coloring

i. Transformational coloring

ii. Matched image coloring

iii. User selected coloring

iv. Recognition based


Texture Recognition Image Colorization System (TRICS)

10


TRICS System Structure


Structure Database set

The training set consists of 9 classes ‘cloud, sky, sea, sand, tree, grass, stone, water, and wood’

Each class has number of samples from 12 to 25 samples.

These samples are taken from real natural images as random 64x64 rectangles.


Results


Results PANN Database:

15


Movie Colorization !!


Coloring System 1

. .

Coloring System 2

Coloring System 3

Coloring System 4

Coloring System n

Recognition based movie Coloring

. .

Indoor Outdoor Natural Manmade . . People

TRICS

Object Tracking, Motion detection

Scene cut detection

Classification

Key frames


Coloring System

. .

User selection movie Coloring

Object Tracking, Motion detection

Scene cut detection

Seeds selection

Key frames


Load movie

1) Shot cut detection

2) Colorize key frames

3) Colorize shot

Save Movie

Detect The Motion Vector

Colorize Shot Frames

Key Frames

Shot Frames



. . . .

. . . .

… … … … … … … …

. . . .

… … … … … … … …

. . . .

. . . .

1 3 5 7 9

11 13 15 17 19

21 23 25 27 29

31 33 35

36 38 40 42 44

46 48 50 52 54

56 58 60 62 64

66 68 70 72 74

36 38 40 42 44

46 48 50 52 54

56 58 60 62 64

66 68 70 72 74



MethodsCross correlation between the adjacent frames FA and FB is used for shot cut detection.

The colorization method used in this stage is the colorization by optimization proposed by A. Levin.

• Three Step Method (TSM) is used to estimate the motion vector between each adjacent frames.

21


Results


Color to Gray = Compression !!


Decolorization Concept

26


Colors Hiding- Encoding H and S channels and then Hiding their coefficients inside I channel:

H >> Object Compression Encoder (OCP).

S >> A: Minimum Color Difference (MCD)

B: Y(Luma) Intensity Difference (YID)

HSL image

H S I

Color Correction,

Segmentation, Median

Filters

Color correction,

Transformation

OCP MCD/YID

RGB image


1) Hue Encoding

Color Correction Segmentation Object Compression

Cluster no1:0 Cluster no2:87

Cluster no3:40 Cluster no4:168

Hue Before

0 50 100 150 200 250 3000

5

10

15x 10

4

Hue After update: 253708

0 50 100 150 200 250 3000

0.5

1

1.5

2

2.5

3x 10

5

Hue Before

0 50 100 150 200 250 3000

5

10

15x 10

4

Hue After update: 253708

0 50 100 150 200 250 3000

0.5

1

1.5

2

2.5

3x 10

5

Object compression results

Compression ratio : 0.0241


MCD METHOD YID METHOD

PSNR= 43.3784 , MSE=3.3442, CR= 042

CR = 0.0187

PSNR= 39.08, MSE=8.024, CR=0.025.

RGB Before Encoding RGB after DecodingCR = 0.0116

Original Saturation

2) Saturation Encoding


More Results

PSNR MSE CR MCD 33.57 17.16 0.099YID 35.63 13.9 0.131

Original (489 kb): PSNR=37.6845, MSE=13.6254,

CR: 0.039, 0.0067 , 0.041 => 0.0289

Original (355 kb): PSNR=39.4457, MSE=7.7113,

CR: 0.02, 0.0128 ,0.0435 => 0.0254


HSI Color HidingLeast Significant Bit (LSB) hiding technique is used due to its simplicity and for testing our idea.

Number of plans needed :

16 bit 16 bit 16 bit × L1/2 32 bit × L1/2 8 bit × L2no.Sat

coefficients (L1)

no.OCP coefficients

(L2)

Sat coefficients positions

Sat coefficients values

OCP coefficients


Just One bit plan is needed.

Original Host

Stego Image

PSNR= 50.9598,

MSE= 0.1426,

SSIM= 0.9984.

HSI Color Hiding


Gray Image

Color Seeds

Encoder : Automatically select seeds Decoder : Colorization by seeds

Color Image Encoding by Decolorization


The proposed Morphological Decolorization system

Y

Cb, Cr

Clustered image (B)

Regions (C)

Boundary Image (D)

Marked Image (E)

KMeans

Clustering Technique

Convert to

YCbCr

Split Clusters to regions

Inner Boundary Extraction

Extract Seeds

Colorize

Recolored Image (F)

Original RGB Image

K

R

f Marked Image

(A)

(B)

(C)

(D)

(E)

(F)

...

Scientific Research Group in Egypt http://www.egyptscience.net 37

Automatic Seeds Selection Using Morphology

1. Using mean shift or adaptive k-mean segmentation techniques◦ The KMean++ segmentation

technique proposed by D. Arthur [3] is used

ARTHUR, D., AND VASSILVITSKII, S. k-means++: the advantages of careful seeding. In Proceedings of the eighteenth annual ACM-SIAM

symposium on Discrete algorithms (Philadelphia, PA, USA, 2007), SODA ’07, Society for Industrial and Applied Mathematics, pp. 1027–

1035


Automatic Seeds Selection Using Morphology

2. For each color cluster, extract the inner boundary and/or skeleton.

3. Select the seeds from boundaries by sampling, with a sampling rate = S.

Allen Y. Yang , John Wright, Yi Ma, S. Shankar Sastry ,‘Unsupervised segmentation of natural images via lossy data compression’

,Computer Vision and Image Understanding (2007)


RESULTS

Recolored

K S R M N PSNR MSE CM Chroma_CR CR

(a) 5 1% 3 100 3598 40.698 4.348 1.014 0.04 0.41

(b) 5 10% 3 100 467 35.97 20.64 4.301 0.012 0.345

(c) 7 50% 3 100 126 32.74 33.68 6.301 0.004 0.337

(d) 10 50% 3 100 173 37.327 26.675 3.927 0.005 0.338

(e) 10 100% 3 100 78 33.106 34.966 5.907 0.003 0.336

(a) (b) (c) (d) (e)


Comparison with colorization methods ofAnat Levin 1 and Liron Yatzif 2


Seeds Hiding◦ Only one bit plan needed.

◦ PSNR=49.87, MSE=0.49 and MSSIM= 0.996 .

Gray ‘boy' image Stego Image


Color Blindness


Color Blindness What is Color Blindness?◦ Color-blindness is the inability to distinguish the differences

between certain colors. This condition results from an absence of color-sensitive pigment in the cone cells of the retina, the nerve layer at the back of the eye.

◦Most color vision problems are inherited and are present at birth. Approximately 1 out of 12 males and 1 out of 20 women are color blind.

43


Color BlindnessWhat does a color-blind person see?◦ A person with color-blindness has trouble seeing red, green, blue, or mixtures

of these colors. The most common type is red-green color-blindness, where red and green are seen as the same color.

44


What does a color-blind person see?

45


Tests for Color-BlindnessIshihara plates

(Detects only Red-Green defect)

(Couldn’t estimate the majority of defect)

46


Start : P =1 , Correct =0

P <= 21?YesNo

NoYes

Correct >= 17

Normal RG-CVD

Correct

<=13

Other Vision

Problem

No

YesCorrect ++ A = B ?

Manual Input =

A

Show plate

PDelay 3

sHide plate P

P ++

Correct Answer

of plate P = B

No

Yes


The same number of volunteers (21) were diagnosed as red green CVD by both tests, with 100% sensitivity of the computer based test compared to the paper based test.

While 243 volunteers were diagnosed as normal in computer based test, when compared to the 246 volunteers diagnosed as normal by the paper based test gave a 98.78% specificity for the computer based test


Visual Impaired !!

Problem Definition

Apart from significant medical progress in applying medicine and surgery for treating several causesof blindness, much of the pioneering work on blindness has focused on two main issues :

1- Reading and writing :

Louis Braille (1809-1852) in the 19thcentury developed the well-known raiseddot code now named after him. This tactilerepresentation was later supplemented byblindness aids like the mechanical Brailleprinter and typewriter (Brailler), againfollowed by more versatile electronicimplementations.

Objectives

o The goal is to make daily activities such aseducation e.g. (identifying drawings and colors)besides reading text from books or computerpossible more interesting and easy for blindstudent.

o Targeting visually impaired that have Priorexperience of shapes and colors .

o Enhance Perception of blind people and givethem a new experience of vision .

Red

PurpleBlue

5see number five

as bluehear a music track as

purple

For example You might

OR


An illustration of the EyeMusic sensory–substitutiondevice (SSD), showing a user with a camera mounted on his glasses, and using bone conductance

headphones, hearing musical notes that create a mental image of the visual scene in front of him. He is reaching for the red apple in a pile of green ones..

sensory–substitution device (SSD)


Image to sound mapping

Color/ intensity Mapping

pixel brightness is coded by loudness.

A single bright object on an otherwise dark surface will thus generate a sound pattern whose loudness reflects its brightness.

Intensity pixel light intensity


What is next ??


Multispectral imaging …


Scientific Research Group in Egypt

ThanksAny Questions??

For further Contact:[email protected]

Color image processing challenges zewail city workshop 7 march 2015

Technology

Transcript of Color image processing challenges zewail city workshop 7 march 2015