Digital image capture Digital image capture –– Camera Camera testing and quality assurancetesting and quality assurance

nnPsychophysical experimental Psychophysical experimental method to estimate image qualitymethod to estimate image quality

nn Hitoshi UrabeHitoshi UrabeFuji Photo Film Co.,Fuji Photo Film Co., Ltd. Ltd.

OutlineOutline

nn BackgroundBackgroundnn Requirements for psychophysical Requirements for psychophysical

experimental methods experimental methods nn Two stages procedure and a triplet Two stages procedure and a triplet

comparisoncomparisonnn ConclusionConclusionnn Future workFuture work

Psychophysical experimentsPsychophysical experiments

Method Stress

Categorical

Rank Order

Paired comparison

Stability

Low

High

Scalability

Low

High

Small

Large smaller

Magnitude

RequirementsRequirements

nn Enables a large number of samples to be Enables a large number of samples to be exampledexampled

nn Provides precise scalabilityProvides precise scalabilitynn Provides low observer stressProvides low observer stressnn Suitable for nonSuitable for non--expert observersexpert observersnn Provides high repeatability of the resultsProvides high repeatability of the results

Two stages procedureTwo stages procedure

Categorical session

Triplet comparison method session

Reduction of the number of the samples

Interval scaling

Two stages procedureTwo stages procedure

(1) The 1(1) The 1stst stage;stage;A categorical stage to reduce unnecessary A categorical stage to reduce unnecessary comparison (remove extremes) .comparison (remove extremes) .

(2) The 2(2) The 2ndnd stage;stage;A precise comparison stage, which A precise comparison stage, which 　　　　enables to compare samples efficiently enables to compare samples efficiently by a triplet comparison.by a triplet comparison.

The The Number of CombinationNumber of Combination

AA

BB

CC

DDEE

GG

FF

7 samples 　　　　　　　　　　　　　　　　　　7 Combinations

DE,DG,EGDE,DG,EGDEGDEG

CD,CF,DFCD,CF,DFCDFCDF

BF,BG,FGBF,BG,FGBFGBFG

BC,BE,CEBC,BE,CEBCEBCE

AE,AF,EFAE,AF,EFAEFAEF

AC,AG,CGAC,AG,CGACGACG

AB,AD,BDAB,AD,BDABDABD

The Number of ObservationsThe Number of Observations

11761176392392494930030010010025251275127542542551513513511171172727

4654651551553131212177775285281761763333363612129966666622222237377878262613137417412472473939105105353515159039033013014343171171575719199909903303304545210210７０７０2121

Pair Pair Comp.Comp.

Triplet Triplet Comp.Comp.

No. of No. of SamplesSamples

Pair Pair Comp.Comp.

Triplet Triplet Comp.Comp.

No. of No. of SamplesSamples

　　ExperimentalExperimental

nn To examine the validity and To examine the validity and reliability of the proposed methodreliability of the proposed method

As a case studyAs a case study

nn To investigate the preferred skin To investigate the preferred skin colors on various mediacolors on various media

ObjectivesObjectives

Original portrait sceneOriginal portrait scene

Facial area of the standard portrait: L*=58.8, a*=17.0, b*=22.0

11stst Stage; ExperimentalStage; Experimental

nn 102 samples102 samplesnn 45 observers45 observersnn 3 categories 3 categories

(Favorable(Favorable, , Acceptable, UnacceptableAcceptable, Unacceptable))nn Viewing illuminant: fluorescent lamp Viewing illuminant: fluorescent lamp

for color evaluation (1000lx)for color evaluation (1000lx)

Test samplesTest samples

a*

b*

2yg

6yg

4yg

2y

4y

6y

2o

4o

6o

2r 4r 6r2g4g6g

2c

4c

6c

2b

4b

6b

2m

4m

6m

S SS

--33

+3+3

+6+6

+9+9

+12+12

1717　　xx　　66　　==　　102102　　SamplesSamples

ΔΔL*L*

+4+4

+6+6

+2+2

Results Results

４Ｒ４Ｒ－３－３－３３－３３１０２１０２２Ｙ２Ｙ＋６＋６＋３＋３1414

４Ｍ４Ｍ＋１２＋１２－３１－３１１０１１０１２Ｂ２Ｂ＋６＋６＋４＋４1313

２ＹＧ２ＹＧ＋６＋６＋４＋４1212

２Ｇ２Ｇ＋３＋３－２－２２５２５４Ｍ４Ｍ＋９＋９＋４＋４1111

２Ｒ２Ｒ＋３＋３－２－２２４２４２Ｍ２Ｍ＋３＋３＋６＋６1010

ＳＳ００－１－１２３２３ＳＳ＋３＋３＋６＋６99

２Ｍ２Ｍ＋９＋９－１－１２２２２２Ｒ２Ｒ＋６＋６＋６＋６88

２Ｒ２Ｒ＋９＋９－１－１２１２１２Ｏｒ２Ｏｒ＋９＋９＋６＋６77

２ＹＧ２ＹＧ＋３＋３００２０２０２２ＧＧ＋６＋６＋７＋７66

２Ｙ２Ｙ＋９＋９００１９１９２２OrOr＋６＋６＋７＋７55

４Ｂ４Ｂ＋３＋３＋２＋２１８１８２Ｂ２Ｂ＋３＋３＋８＋８44

２Ｏｒ２Ｏｒ＋３＋３＋２＋２１７１７２２ＣｙＣｙ＋３＋３＋８＋８33

２Ｍ２Ｍ＋６＋６＋２＋２１６１６２Ｃｙ２Ｃｙ＋６＋６＋８＋８22

２ＹＧ２ＹＧ＋９＋９＋２＋２１５１５ＳＳ＋６＋６＋１７＋１７11

HueL*ScoreOrderHueL*ScoreOrder

22ndnd Stage; ExperimentalStage; Experimental

nn 21 samples21 samples（（Out of 102 samples)Out of 102 samples)nn 15 observers15 observers（（including 4 experts)including 4 experts)nn 5 categories (1:bad 5 categories (1:bad ～～ 5:5:Excellent) Excellent) nn Paired comparison and Triplet Paired comparison and Triplet

comparisoncomparisonnn Repeated 5 times Repeated 5 times nn ScheffeScheffe’’ss methodmethod

Examples Examples

Triplet comparisonTriplet comparison

Observer:MIObserver:MI

Paired comparisonPaired comparison Triplet comparisonTriplet comparison

1

2

3

4

5

4

3

2

1

5

Times Times

S2c2o2r2m2b2y2g2yg4c4o4r4m4g4b4y4yg2k2w6w01-2-1201-2-12１回目２回目３回目４回目５回目S2c2o2r2m2b2y2g2yg4c4o4r4m4g4b4y4yg2k2w6w SS2c2c2o2o2r2r2m2m2b2b2y2y2g2g2yg2yg4c4c4o4o4r4r4m4m4g4g4b4b4y4y4yg4yg2k2k2w2w6w6w01-2-1201-2-12１回目２回目３回目４回目５回目

Bad Excellent Bad Excellent

Observer:MTObserver:MT

01-2-1201-2-12１回目２回目３回目４回目５回目S2c2o2r2m2b2y2g2yg4c4o4r4m4g4b4y4yg2k2w6wS2c2o2r2m2b2y2g2yg4c4o4r4m4g4b4y4yg2k2w6w 01-2-1201-2-12１回目２回目３回目４回目５回目S2c2o2r2m2b2y2g2yg4c4o4r4m4g4b4y4yg2k2w6wS2c2o2r2m2b2y2g2yg4c4o4r4m4g4b4y4yg2k2w6w

Paired comparisonPaired comparison Triplet comparisonTriplet comparison

1

2

3

4

5

4

3

2

1

5

Times Times

Bad Excellent Bad Excellent

Observer:TKObserver:TK

Paired comparisonPaired comparison Triplet comparisonTriplet comparison

1

2

3

4

5

4

3

2

1

5

Times Times

SS2c2c2o2o2r2r2m2m2b2b2y2y2g2g2yg2yg4c4c4o4o4r4r4m4m4g4g4b4b4y4y4yg4yg2k2k2w2w6w6w01-2-1201-2-12１回目２回目３回目４回目５回２回目３回目４回目01-2-12５回目01-2-12１回目S2c2m2b2g4c4m4g4b6w2o2r2y2yg4o4r4y4yg2k2wS2c2o2r2m2b2y2g2yg4c4o4r4m4g4b4y4yg2k2w6w

Bad Excellent Bad Excellent

Psychophysical scalePsychophysical scale

Samples

Triplet

Paired

0

-2

+2

+1

-1

Interval scaleInterval scale

ObserverObserver’’s variance s variance

0

0.1

0.2

0.3

0.4

0.5

0.6

MI

KS

NT

CT

ST

MT

MN

MH

MH

EY

TY

YK

KK

YH

YS

Me

an

Observer

Va

ria

nce

TripletPairt

Scalability Scalability

Paired comparison Triplet comparison

Scale Range 1.51 2.29Standard Deviation 0.24 0.33

SR/SD 6.19 7.00

Correlation Correlation

y = 1.5768xR2 = 0.978

-1.5

-1

-0.5

0

0.5

1

1.5

-1.5 -1 -0.5 0 0.5 1 1.5

Paired comparison

Trip

let c

ompa

rison

CorrelationCorrelation

y = 0.9954xR2 = 0.9962-1.0

-0.5

0.0

0.5

1.0

-1.0 -0.5 0.0 0.5 1.0

Paired comparison

Tri

ple

t co

mp

ari

so

n

Thurston’s Case Ⅴ

Preferred skin colors Preferred skin colors

b*a*L*s2y2o2r2m2b2c2yg2g4y4o4r4m4yg4g4c4b2k2ws6w一般人専門家

O RG

C

B

M

YG

Y

O

+4

+2

a*

b*

Experimental time Experimental time

15 s

5 s

12 s

12 s

One set of One set of combinationcombination

For a pairFor a pair

Triplet comparison

Paired comparison

Required time for assessment

(In case of 20 samples)

Experimental time Experimental time

0

1

2

3

4

5

6

7

0 10 20 30 40 50 60

The number of samples

Paired comparison

Triplet comparison

Hours

ConclusionConclusion

TheThe triplet comparison reduced the assessment triplet comparison reduced the assessment time by more than 50% compared to the paired time by more than 50% compared to the paired comparison.comparison.

TheThe new psychophysical experimental method can new psychophysical experimental method can provide a useful procedure to estimate image provide a useful procedure to estimate image quality from the following points of view;quality from the following points of view;

・　Scalability of the method　　 　・　Repeatability of the scale

・　Observer stress

Status of ISO 20462 :Status of ISO 20462 :Psychophysical experimental method to Psychophysical experimental method to

estimate image qualityestimate image quality

nn MultiMulti--part standard (ISO/TC42/WG18)part standard (ISO/TC42/WG18)nn Part 1 Part 1 –– General description of psychophysical General description of psychophysical

evaluation By Dr. evaluation By Dr. Keelan Keelan (Kodak)(Kodak)nn N4925_WD20462N4925_WD20462--1_E_.PDF1_E_.PDFnn Part 2 Part 2 –– Triplet comparison method By UrabeTriplet comparison method By Urabenn N4929_WD20462N4929_WD20462--2_E_.PDF2_E_.PDFnn Part 3 Part 3 –– Quality ruler method By Dr. Quality ruler method By Dr. KeelanKeelannn N4926_WD20462N4926_WD20462--3_E_.PDF3_E_.PDF