3D-MAD: A Full Reference Stereoscopic Image Quality...

1

3D-MAD: A Full Reference Stereoscopic ImageQuality Estimator based on Binocular Lightness and

Contrast PerceptionYi Zhang, Damon M. Chandler

School of Electrical and Computer EngineeringOklahoma State University

Stillwater, OK 74075Email: {yi.zhang, damon.chandler}@okstate.edu

Abstract—Algorithms for stereoscopic image quality assess-ment (IQA) aim to estimate qualities of 3D images in a mannerthat agrees with human judgments. Modern stereoscopic IQAalgorithms often apply 2D IQA algorithms on stereoscopic views,disparity maps, and/or cyclopean images, to yield an overallquality estimate based on properties of human visual system(HVS). This paper presents an extension of our previous 2DMAD [1] algorithm to a 3D version (3D-MAD) to evaluate3D image quality. The 3D-MAD operates via two main stageswhich estimate perceived quality degradation due to (1) distortionof the monocular views and (2) distortion of the cyclopeanview. In the first stage, the conventional MAD algorithm isapplied on the two monocular views, and then the combinedbinocular quality is computed as the weighted sum of thetwo estimates, where the weights are related to the averagedblock-based contrast value. In the second stage, intermediatemaps corresponding to the lightness distance and pixel-basedcontrast are generated based on a multipathway contrast gain-control model. Then, the cyclopean view quality is estimatedby measuring statistical-difference-based features obtained fromthe pristine stereopair and the distorted stereopair, respectively.Finally, the estimates obtained from the two stages are combinedto yield an overall quality score of the stereoscopic image. Testson various 3D image quality databases demonstrate that ouralgorithm significantly improves upon many other state-of-the-art 2D/3D IQA algorithms.

Index Terms—Full reference, stereoscopic image quality, con-trast gain-control, binocular combination

I. INTRODUCTION

The rapid development of digital image processing tech-nology has encouraged tremendous research in the field of2D image quality assessment (IQA), and noticeable progresshas been made on developing various full reference, reducedreference, and no reference 2D IQA algorithms (see [2]–[4]for reviews). Stereoscopic IQA, however, has been relativelyless explored, partially due to the difficulty in viewing com-mercially available 3D content, as well as issues surroundingvisual discomfort and fatigue after long-term stereoscopicviewing [5]. However, in recent years, 3D media has dra-matically increased its popularity and availability for humanconsumption, and applications may range from entertainment(e.g., 3D television, 3D cinema, and 3D game) to morespecialized purposes (e.g., 3D remote education, 3D robotnavigation, and 3D medical imagery). As a result, there is

a need for IQA algorithms that can assess the quality ofstereoscopic/3D imagery.

Different from the conventional 2D image viewing, a stereo-scopic image contains two slightly different views, each ofwhich is projected separately onto the human retina. When astereoscopic image is viewed by a human subject, the HVSdoes not examine the left view and right view individually,but instead, goes through a very complex binocular fusionand rivalry process between the two views to obtain a merged3D mental view. The occurrence of binocular single visiondepends not only on the disparities of individual stimuluselements, but also on the geometrical relation of differentparts of the pattern presented to each eye [6]. Therefore, thequality of a 3D image is affected not only by the degradationlevel of each individual left and right view, but also bythe experience of binocular visual perception. The 3D QAalgorithm, however, cannot access this merged single view.Thus, assessing the perceived quality of a stereoscopic imagecan be quite challenging if only the two stereoscopic viewsare available.

The 3D QA task can be easier when the left and right viewsare equally distorted (also called the symmetrically distortedstimuli). One technique is to simply apply 2D QA methods onboth stereoscopic views independently and then combine thetwo scores to yield an overall quality estimate (e.g., [7]–[10]).The other technique is to employ both the stereopairs and thedepth/disparity information (e.g., [11], [12]), assuming that theHVS also uses binocular depth/disparity information to judgethe 3D image quality. However, two important points needto be considered. First, since the ground truth depth/disparitymaps are not always available, these models can operatebased only on the estimated maps. Thus, the accuracy of thedepth/disparity estimate algorithm employed may substantiallyaffect the performance. Second, the depth/disparity informa-tion may have no correlation with the 3D perceived quality.This latter point has been confirmed by Kaptein et al. [13]who ran subjective experiments using blurred images with thesame objects at different depths; Kaptein et al. found that thedepth level in a 3D display does not influence the perceivedimage quality. Also in [14], Goldmann and Ebrahimi havedemonstrated the similar shortcoming of applying 2D QAmethods on 3D QA task.

Preprint of paper published in IEEE Transactions on Image Processing 2015 © 2015 IEEE

2

Despite these difficulties, the two aforementioned tech-niques can still achieve fairly good performances on predictingquality of these symmetrically distorted stereoscopic images.However, if the two stereoscopic views contain differentamounts and/or types of distortions (also called the asymmet-rically distorted stimuli), these algorithms perform much lesseffectively; the asymmetrically distorted stimuli can make thestereoscopic IQA problem much more challenging. The mainreason is that the merged mental view changes dependingon the different natural distributions of the distortion. Todemonstrate, Figure 1 shows two distorted stereoscopic imagesthat contain similar high-quality images for the left view,and low-quality images for the right view. As reported bysubjects, for the white noise image, they perceive a mergedview that looks more like the right view image, while forthe Gaussian blurred image, the perceived merged view looksmore like the left view image. Thus, effectively modeling theHVS’s binocular perceptual mechanisms when asymmetricallydistorted views are presented to the two eyes is essential toenhance the performance of a 3D QA algorithm. Accordingto [15], binocular vision is a complex visual process thatbinocular fusion and binocular rivalry coexist at the samepoint in space and time. To study these two well-knownproperties in binocular vision, most vision studies (e.g., [16]–[26]) have used unnatural stimuli (or simulated data) withhighly controllable parameters, and their results can indeedoffer some basic insights. However, it remains unclear howto apply the results of such controlled studies to the 3D QAproblem, as they have never tested on real natural images.

For QA of stereoscopic images that contain asymmetricallydistorted stimuli, a recent technique has been employed, whichattempts to model the 3D QA behavior of the HVS based onbinocular fusion and suppression properties. Approaches basedon this technique often apply 2D QA methods on the so-calledcyclopean view, a single mental image of a scene created inthe brain by combining the two images received from the twoeyes. These approaches assume that the HVS perceives 2Dtypes of degradation after they are combined in the cyclopeanview, and thus it is meaningful to measure 2D artifacts on thecyclopean view as an estimate of the 3D quality degradationlevel. In [28], the cyclopean image was defined as the averageof the left image and the disparity-compensated right image.More recently in [29], the cyclopean image was synthesized byusing a linear model proposed in [30], in which the coefficientsof the linear model are computed based on the local Gaborfilter energy. Although a better prediction performance wasobserved compared with the two aforementioned techniques,which partially demonstrates the effectiveness of the cyclopean3D QA model, issues concerning how the cyclopean image isformulated in the brain, and how the HVS judges quality basedon this cyclopean image, remain to be further explored.

Beyond these aforementioned techniques, no strong baselineor core principle has been reported to point out the natureof the stereoscopic IQA problem, and these techniques areonly roughly distinguished from each other based on thetype and/or the amount of information they extract from 3Dimages. In fact, some recently developed 2D IQA algorithmsare so powerful that impressive predictive performance can be

achieved on most stereoscopic image databases that containsymmetrically distorted stimuli when these algorithms are onlyapplied on the two stereoscopic views and then averaged (seeTable III). Thus, we believe that (1) a core principle of 3D QAalgorithm design should be to more accurately predict qualityof asymmetrically distorted stereoscopic images while stillmaintaining a high performance on symmetrically distortedones; and (2) a grand challenge confronted by current 3DQA research is to better model the binocular QA behavior ofthe HVS when asymmetrically distorted stimuli are presented.Unfortunately, as noted by Moorthy et al. in [27], noneof the current 3D QA algorithms have achieved predictionperformance better than (or in most cases, even as good as)2D QA algorithms applied to the individual stereopairs.

To make progress of the current 3D QA research, in thispaper, we extend our previous 2D MAD algorithm [1] toaddress the 3D IQA problem by incorporating two mainstrategies. First, we propose a contrast-weighting strategy thataims at balancing the roles of each stereoscopic view indetermining the overall 3D image quality. There is evidencethat the monocular stimulus with higher contrast often receivesenhanced predominance and shorter duration of suppressionin human visual perception [31]–[33]. If the contrasts ofthe stimuli presented to both eyes are equally high, theneach stimulus remains suppressed for shorter periods of timeand, consequently, the two stimuli alternate in dominancemore rapidly [30]. Accordingly, we believe that the qualityof a monocular scene with higher contrast can play a moredominant role in determining the HVS’s judgment of 3Dimage quality. Second, we propose a cyclopean-feature-imagestrategy which combines a recent model of binocular rivalrywith quality-relevant features and machine learning to providesignificantly improved predictions of 3D quality. Specifically,we propose two different types of quality-related features(lightness distance and pixel-based contrast) and build threetypes of corresponding cyclopean feature images 1 (cyclopeanimages of the global/local lightness distance and pixel-basedcontrast) by using the multipathway contrast gain-controlmodel (MCM) [34], [35] to better account for the binocularfusion and rivalry processes in the HVS. We then quantifydistortions contained in these computed cyclopean featureimages as a measurement of the 3D image quality by usingthe statistical-difference-based features, which are combinedvia a support vector machine learning approach. As we willdemonstrate, by employing the two aforementioned strategies,it is possible to more accurately estimate the quality of anasymmetrically distorted stereoscopic image.

Owing to the above-mentioned analysis, the proposed 3D-MAD algorithm operates via two main stages. The first stageof the algorithm estimates perceived quality degradation dueto 2D monocular scene artifacts. This stage operates by firstdirectly applying the MAD algorithm on two separated stereo-scopic views (left and right views) coming from the referenceand distorted images, respectively. The only difference is thatthe MAD parameters were obtained after training on the

1Here, we use the term “cyclopean feature image” to distinguish from theterm “cyclopean image” that deals only with the image luminance.


3

Left view Right view Reference

LC = 0.095 RC = 0.103 Distorted (White noise) DMOS=64.40 MAD=50.47 2D-MAD=2.29

MADL = 0; LC = 0.095; LW = 0.995 MADR = 100.93; RC = 0.173; RW = 2.816 Distorted (Gaussian blur) DMOS=42.05 MAD=59.56 2D-MAD=1.34

MADL = 0; LC = 0.096; LW = 1.017 MADR = 119.12; RC = 0.043; RW = 0.177

Fig. 1. Two demonstrative asymmetrically distorted stereoscopic images from the LIVE 3D image database [27] have high quality on the left view andlow quality on the right view. Note that C̄L and C̄R represent the averaged block-based contrast value for the left and right views, respectively; WL andWR denote their corresponding weights computed through Eqs. (4) and (5). “MAD” denotes a quality score computed by averaging MADL and MADR.“2D-MAD” is computed through Eq. (6).

2D LIVE image database [36]. Then, the left- and right-image quality scores are linearly combined using weightsthat are computed as the average of the normalized block-based contrast value of each stereopair. The second stage ofthe algorithm estimates perceived quality degradation due tocyclopean view artifacts. This stage operates by measuringdissimilarities between the reference and distorted cyclopeanlightness distance and pixel-based contrast maps based onlocal statistical differences. All of these cyclopean featureimages are disparity-compensated and are synthesized by us-ing stereopairs following a multipathway contrast gain-controlmodel. Finally, the quality measurements obtained from thetwo stages are combined to yield an overall estimate of the per-ceived 3D quality degradation. As we will demonstrate, thesetwo stages together allow 3D-MAD to achieve good predictiveperformance on both the symmetrically and asymmetricallydistorted stereoscopic images across various databases.

The main contributions of this work are as follows. First,we tackle the aforementioned challenge confronted by current3D QA research by presenting two strategies to enhance thequality predictive performance on asymmetrically distortedstereoscopic images. One is the contrast-weighting strategyand the other one is the cyclopean-feature-image strategy, bothof which allow the proposed method to take into accountdifferent contributions of the left and right views towards

the overall 3D quality prediction. As shown in Table III,the former strategy improves upon the conventional averaged-approach [28] (when comparing 2D-MAD results with MAD),the latter strategy improves upon the previous linear cyclopeanimage model [30] (when comparing Cyc-MAD results withChen’s method [29]), and the combined strategies allows3D-MAD to achieve better performance on most 3D imagedatabases. Also, the training process employed in SectionIII-C3 has demonstrated a way to predict 3D image qualityby referring to the distortion information of 2D images. Thisfact also indicates that if appropriate features and models areutilized, the 2D image and 3D image can have some propertiesin common in the feature space. Finally, to some extent, theproposed 3D-MAD algorithm will benefit the current 3D QAresearch work. To the best of our knowledge and based onthe test databases used in our work, 3D-MAD is currently thefirst 3D full reference (FR) IQA algorithm that can achievecompetitive or even better performance than 2D FR IQAmethods applied on stereopairs.

This paper is organized as follows: Section II provides abrief review of current 2D and 3D QA algorithms. In SectionIII, we describe details of the 3D-MAD algorithm. In SectionIV, we analyze and discuss the performance of the proposedalgorithm on various 3D image quality databases. Generalconclusions are presented in Section V.


4

II. PREVIOUS WORK

In this section, we provide a brief review of current 3D QAalgorithms. As mentioned in Section I, a stereoscopic imagemay contain either symmetrically or asymmetrically distortedstimuli. The complex QA behavior of the HVS in viewingsymmetrically and asymmetrically distorted stereopairs hasgiven rise to a variety of simple or sophisticated 3D QAtechniques. Based on the type and/or the amount of infor-mation extracted from stereoscopic images, these techniquesapproximately fall into one of the three categories: (1) thosethat apply 2D QA methods on each view of the stereoscopicimage separately and then combine the two scores into anoverall quality score; (2) those that take into account the depthinformation and apply 2D QA methods on both stereopairsand the estimated disparity map; and (3) those that take intoaccount the binocular visual properties of HVS and apply 2DIQA methods on cyclopean images and/or stereopairs.

A. Stereopair-based IQA

As stated in Section I, the most straightforward way toestimate the quality of a 3D image is to directly apply the 2DQA algorithms on each individual view, and then combine thescores into one quality measurement. For instance, Campisi etal. [10] applied four IQA algorithms (SSIM [37], UQI [38], C4[39], and RRIQA [40]) on stereopairs and combined qualityscores via three approaches: “average” approach, “main eye”approach, and “visual acuity” approach. Gorley et al. [9]proposed a “Stereo Band Limited Contrast” metric to estimate3D image quality based on matched points of stereopairsdelivered by SIFT [41] and RANSAC [42]. Yang et al. [43]estimated 3D image quality based on the average PSNR ofstereopairs, and also based on the absolute difference in pixelvalues between the left and right view. Also, in [7], [8], threewidely accepted 2D IQA algorithms (PSNR [44], SSIM [37],and VQM [45]) were investigated on their correlation withperceived quality of 3D video.

B. Algorithms Based on Stereopair and Depth Information

Another technique commonly used in 3D QA is to apply2D QA methods on estimated disparity maps and to possi-bly combine this information with 2D QA estimates of thestereopairs. For instance, Benoit et al. [12] applied 2D QAmethods (SSIM [37], C4 [39]) on the left and right viewsindependently, and the 3D image quality was computed bycombining the two quality scores with the estimated disparitymap distortion in a either local or global manner. You etal. [11] investigated the capabilities of some 2D IQA algo-rithms for stereoscopic IQA and found that applying SSIMon stereopairs combined with a mean absolute difference tocompute the disparity map distortion always yields the bestperformance within all possible combinations considered inthe paper. Hwang et al. [46] designed a visual attention anddepth-assisted stereo image quality model, which consists ofthree main components: a stereo attention predictor, a depthvariation predictor, and a stereo distortion predictor. Xing etal. [47] proposed a perceptual quality estimator which operates

based on three main factors (crosstalk level, camera baseline,and scene content) contributing to crosstalk perception inevaluating quality levels of stereoscopic presentations; thequality score of this approach is computed as the average ofthe SSIM-weighted disparity map.

There are also some researchers who have employed stereodepth information for stereo video quality assessment (3DVQA). Although these works address video quality assess-ment, they still demonstrate a correlation between depthinformation and visual quality. For example, Ha et al. [48]used the disparity information (estimated via a horizontalblock matching algorithm [49]) to evaluate stereoscopic videoquality by taking into account four factors: temporal variance,disparity variation in intra-frames, disparity variation in inter-frames, and disparity distribution of frame boundary areas.Tikanmaki et al. [50] proposed a 3D VQA algorithm byapplying PSNR [44] and VSSIM [51] on color and depthsequences. Hewage et al. [8] proposed a reduced-referencequality assessment approach for 3D depth map transmissionusing extracted edge information and PSNR [44] to assessvideo quality. Boev et al. [52] proposed a compound FR 3DVQA algorithm composed of a monoscopic quality componentand stereoscopic quality component. The former assesses themonoscopic perceived distortions and the latter assesses theperceived degradation of the perceptual disparity maps as wellas of the cyclopean images.

C. Algorithms Based on Binocular Fusion and Suppression

A third type of 3D QA technique has been recently de-veloped, which incorporates information regarding variousbinocular artifacts, visual masking effects, and monocularvisual properties, all of which may affect the quality of a 3Dimage. In [53], Ko et al. introduced a binocular perceptionmodel called the structural distortion parameter (SDP) tocontrol the three components of SSIM to obtain an overallquality index. Similar work can be seen in [54]. In [55],Wang et al. presented a 3D IQA algorithm based on binocularspatial sensitivity (BSS), which considers binocular fusionand rivalry properties. The BSS module is implemented incorresponding and non-corresponding regions according to thedisparity map generated by disparity matching. In [56], Qi etal. suggested that binocular vision is sensitive to the structureof low-level features and the discrepancy between two viewsof the stereoscopic image pair. Thus, the phase congruency andsaliency map are employed to compose a feature map and alocal matching function is employed to weight the discrepancybetween the two feature maps to estimate image quality.

Other 3D QA algorithms take into account binocular in-tegration, which combines two views into one cyclopeanview; the quality of the stereoscopic image is determinedby analyzing this integrated image. For example, Bensalmaet al. [57] developed a “Binocular Energy Quality Metric,”which first simulates the binocular visual signal by modelingsimple and complex cells; quality is then estimated based onthe difference of the associated binocular energy. Maaloufet al. [28] applied a multi-spectral wavelet decomposition tothe cyclopean image, and then used the contrast sensitivity


5

coefficients of the cyclopean image together with disparitymaps for quality evaluation. Chen et al. [29] applied MS-SSIM [58] on the cyclopean image which was generated byusing a linear model, where the model weights are computedfrom the normalized Gabor filter magnitude responses. Lin etal. [59] utilized the binocular combination and the binocularfrequency integration as the bases for measuring the quality ofa stereoscopic image. Shao et al. [60] measured the binocularenergy response similarity between the original and distorted3D image by using components of binocular energy andbinocular just noticeable difference (BJND), and then the twoevaluation results are nonlinearly combined to estimate the3D image quality. Also, in [61], Shao et al. classified thestereoscopic images into non-corresponding, binocular fusion,and binocular suppression regions, and then evaluate theseregions separately via the BJND to estimate the quality.

In the following section, we describe our dual-strategy-based 3D IQA algorithm, 3D-MAD, which employs adaptiveleft/right weighting, cyclopean feature images, and machinelearning to estimate stereoscopic image quality.

III. ALGORITHM

In this section, we first briefly describe the background ofthe stereoscopic IQA problem, upon which the motivation ofour work is presented. Then, we provide details for each stagein the 3D-MAD algorithm.

A. Background and Motivation

The stereoscopic/3D IQA task refers to assess quality ofstereoscopic images in a manner that is consistent with humanjudgments. Thus, how to effectively model the QA behaviorof the HVS in binocular vision can be essential to enhancethe performance of a 3D IQA algorithm. It is widely knownthat binocular fusion and binocular rivalry are the two mainvision processes in the HVS when people are viewing 3Dscenes. According to [15], binocular rivalry inevitably occurswhenever visual stimuli are presented, even if the two stimuliare identical. However, regarding to the precedence of the twoprocesses, Blake and Boothroyd [62] stated that

“The binocular visual system first seeks to es-tablish correspondence between image features con-tained in the two monocular views. Failure to es-tablish such correspondence leads to binocular sup-pression, an effective means for eliminating diplopiaand/or confusion. When correspondence is estab-lished, however, binocular fusion takes precedenceover the suppression. Only those monocular featureswith no interocular counterpart participate in therivalry process. ” [62]

Because of the complex fusion and rivalry processes inbinocular vision, early 3D QA work that attempted to applyvarious 2D QA approaches on stereopairs and/or the disparitymap, are often less effective in predicting the stereoscopicimage quality, especially when asymmetrically distorted stim-uli are presented on the two views. Later, although Maalouf[28] and Chen [29] have employed the cyclopean imageas a representative for the merged 3D mental view, and

Chen’s work [29] has indeed improved the quality predictiveperformance on asymmetrically distorted stereopairs to someextent, the performance on symmetrically distorted stereopairsis actually not high (see Table III). Thus, as we have arguedin Section I, a core principle of 3D QA algorithm designshould be to improve the algorithm performance on predictingquality of both the symmetrically and asymmetrically distortedstereopairs. Under this principle, we further argue that thegrand challenge confronted by current 3D QA research isto more accurately model the QA behavior of the HVSin binocular vision especially when asymmetrically distortedstimuli are presented, which is also the most fundamentaldriven force of this work.

To address this challenge, we adopt in our work two mainstrategies for better modeling the binocular perception of im-age quality in the HVS. One is the contrast-weighting strategyand the other one is the cyclopean-feature-image strategy. Theformer strategy is motivated by both the theoretical analysisin [30]–[33] and the practical 3D-viewing experiences holdby the subjects. As demonstrated in Figure 1, the differentialmean opinion scores (DMOS values) indicate that the stereo-scopic image containing the Gaussian blurred right view hasbetter perceived quality than the stereoscopic image containingthe white noise in the right view. The reason is that, for thewhite noise image, the HVS judges quality based mainly onthe right view which has higher contrast because of the noise;while for the Gaussian blurred image, the left view plays amore important role, since the right view has relatively lowercontrast due to the blurring. Thus, the proposed method ofcombining quality estimates of both views via contrast-basedweighting can predict the qualities of these two images quitewell, whereas the conventional approach that only averagesthe two quality scores cannot.

The latter strategy, motivated by the recent multipathwaycontrast gain-control model [34], [35] and the cyclopean-image-based 3D QA approach proposed in [28], [29], isemployed to more efficiently model the binocular fusion andrivalry processes of the HVS. As mentioned in Section I, theHVS judges stereoscopic image quality based on a merged 3Dmental view created in the brain while the 3D QA algorithmcannot access this merged view, and this fact makes the 3DQA task much more challenging. The cyclopean image canbe used as a replacement of this merged view for objectivequality estimate. However, the complex QA behavior of theHVS in binocular vision makes the simple linear model [29]to be less effective in cyclopean image computation. Thus,we adopt the multipathway contrast gain-control model, whichcomputes the perceived contrast and phase of the cyclopeanimages in separate pathways and incorporates a interocularcontrast gain-control mechanism to simulate the fusion andrivalry processes in the HVS. To extend MCM to work for3D QA on natural images, we propose a feature-based MCM[Eq. (12)] which takes into account both the cyclopean featureimages and the different dominant roles of the two eyesin 3D vision. As shown in Figure 2, the three cyclopeanfeature images [Figure 2(b)] computed from the referenceand distorted stereopairs [Figure 2(a)] exhibit considerabledissimilarities between the reference and distorted stereoscopic


6

TABLE IPERFORMANCE COMPARISON OF USING FEATURES VS. ABSOLUTE

LUMINANCE TO PREDICT STEREOSCOPIC IMAGE QUALITY. NOTE THATFOR BOTH OPTIONS, WE USE THE SAME MCM PARAMETERS TO COMPUTE

THE CYCLOPEAN IMAGES (DENOTED BY “CYC-LUMINANCE”) AND

CYCLOPEAN FEATURE IMAGES (DENOTED BY “CYC-MAD”), AND THE

SAME 2D LIVE IMAGE DATABASE [36] FOR MODEL TRAINING.

LIVE3D Phase I LIVE3D Phase IIApproach CC SROCC CC SROCCCyc-MAD 0.944 0.937 0.921 0.915Cyc-luminance 0.944 0.937 0.894 0.885

images, and some dissimilar regions are clearly visible in theclose-ups.

There are two reasons to employ these features [see Eqs.(7)-(9)] instead of the absolute luminance values to estimateimage quality. First, the multipathway contrast gain-controlmodel only provides a model for how stimulus contrast iscombined in binocular vision; thus, it cannot be used directlywith a (luminance) image. Also, it is important to note thatthe MCM was designed only for modeling unnatural stimulior simulated data with highly controllable parameters, and notfor natural images. Thus, to extend MCM to work for 3D QAon natural images, we utilize the lightness distance and pixel-based contrast feature maps as various representations of theperceived contrast.

Second, it is evident that no current models can fully modelbinocular vision. Hence, it is more reasonable to use multiplefeatures to comprehensively measure the distortions, just asthe fact that human use multiple cues to judge image quality.This has been demonstrated in Table I, in which we comparedthe quality predictive performance of using the proposed threetypes of features vs. the absolute luminance. (In Table I, weused the same MCM parameters to compute the cyclopeanimage and the cyclopean feature images, and also use thesame 2D LIVE image database [36] for model training. For faircomparison, we do not consider the 2D-MAD quality estimateon stereopairs, and only use the quality degradation measure-ments of the cyclopean images/feature images as the overallquality estimate of the stereoscopic image.) As shown in TableI, the two options can achieve almost equal performance whentesting on the LIVE 3D phase I database [27] which containsthe symmetrically distorted stimuli. However, when testing onthe LIVE 3D phase II database [27] which partially containsthe asymmetrically distorted stimuli, the proposed strategy thatemploys the three types of features can achieve much betterperformance than that which employs the luminance only.This result also demonstrates the effectiveness of the proposedfeatures and models in predicting quality of asymmetricallydistorted stereoscopic images.

Motivated by all mentioned above, the proposed 3D-MADalgorithm operates via two main stages to estimate qualityof stereoscopic image: (1) 2D-MAD-based quality estimateon stereopairs, and (2) MCM-based quality estimate on cy-clopean feature images. Then, the two quality estimates arecombined through a geometric mean to yield a single valuethat represents the overall perceived quality degradation of thestereoscopic image. The following subsections provide details

MADDS2

Distorted left view

Original left view

MAD Algorithm

Block-based contrast

Distorted right view

Original right view

MAD Algorithm


LW

RW

MADL

MADR

Fig. 3. Block diagram of the first stage of our algorithm, which applies 2DMAD on stereopairs.

for each stage.

B. 2D-MAD on Stereopairs

In the 2D-MAD-based QA stage, the conventional MADalgorithm [1] is applied on the stereopairs (i.e., the left andright view images) to estimate the perceived distortion corre-sponding to each monocular view, but with different trainedparameters. Then, the overall 2D-MAD quality is computed asa weighted sum of both stereopair distortion measures, wherethe weights are computed based on the normalized block-basedcontrast. The block diagram of this stage is shown in Figure3.

The MAD algorithm consists of two separate strategies:(1) a detection-based strategy, which computes the perceiveddistortion due to visual detection (denoted by ddetect) and (2)an appearance-based strategy, which computes the perceiveddistortion due to visual appearance dissimilarity (denoted bydappear). The final MAD quality measurement is a weightedgeometric mean given by

MAD = (ddetect)α × (dappear)

1−α, (1)

where α ∈ [0, 1] serves to adaptively combine the twostrategies based on the overall level of distortion:

α =1

1 + β1 × (ddetect)β2

. (2)

In original MAD, the two parameters (β1 = 0.467 and β2 =0.130) and the five scale-specific weights (ws = 0.5, 0.75, 1,5, and 6) were all obtained after training on the A57 imagedatabase [63] (see [1] for a complete description of the MADalgorithm). However, the A57 image database only contains57 images (3 reference images and 54 distorted images) whichis a small dataset that might be insufficient for training. Thus,to obtain more reasonable estimations of these parameters aswell as to further improve the algorithm performance, in thiswork, we utilize the 2D LIVE image database [36] and thetrained parameters are β1 = 0.369, β2 = 0.153, ws = 0.75,1.25, 0.5, 4, and 7 (s ∈ {1, 2, 3, 4, 5}), respectively.

With these newly-trained parameters, we apply MAD toeach view of the stereoscopic image by assuming that 3Dquality is influenced by 2D content, and that the local maskingwhich comes from two visual paths is also observed. As weadvocated in Section I, the perceived quality of a 3D image


Administrator

Line

Administrator

Line

7

Left view Right view

Ref

eren

ce

D

isto

rted

(a)

Cyclopean global lightness distance Cyclopean local lightness distance Cyclopean pixel-based contrast

Ref

eren

ce

Dis

torte

d

Clo

se-u

ps

(b)

Fig. 2. Demonstrative cyclopean feature images corresponding to the lightness distance and pixel-based contrast (b) computed from the reference and distortedstereopairs (a) in the the LIVE 3D image database [27]. The close-ups show some dissimilar regions between the cyclopean feature images.

is determined largely by the quality of the monocular viewwith higher contrast. If the contrasts perceived by both eyesare the same, then we assume that the quality of both visualsignals are of equal significance. Therefore, we weight eachMAD score by using block-based contrast maps.

Specifically, a block-based contrast map for an image iscomputed in the lightness domain by first dividing the imageinto blocks of 16×16 pixels (with 75% overlap between neigh-boring blocks), and then measuring the root mean squaredcontrast of each block.

C(b) = σ̃(b)/μ(b) (3)

where μ(b) represents the average luminance value of blockb, and σ̃(b) represents the minimum standard deviation amongthe four 8×8 subblocks within b (see Appendix A in [1]).

For both the reference and distorted stereopairs, we firstcompute their block-based contrast maps denoted as C ref

L ,

CrefR , Cdst

L , and CdstR , respectively. Here, “ref” denotes the

reference stereoscopic image; “dst” denotes the distortedstereoscopic image; “L” and “R” denote the left and rightview of each stereopair. The normalized weights of distortedstereopairs are then computed via

WL = (C̄dstL /C̄ref

L )γ (4)

WR = (C̄dstR /C̄ref

R )γ (5)

where C̄ is the mean value of C, and γ = 2 is a factor thatis aimed to emphasize higher contrast values (see the onlinesupplement at http://vision.okstate.edu/3dmad/ for experimentresults based on different γ values.). Note that the computedblock-based contrast may vary even for the reference stere-opairs, thus we normalized the contrast of each distorted viewby dividing by the contrast of each corresponding referenceview to ensure that the weight difference for a stereopair isproduced by the distortion only, and not by the image content.


8

Lightness distance statistical difference

Pixel-based contrast statistical difference

3fS

21, ff SS

MADCycS

SVM Model

Multipathway contrast gain-control model

Multipathway contrast gain-control model

Cyclopean Image

Lightness distance map

Pixel-based contrast map

Cyclopean Image

Lightness distance map

Pixel-based contrast map

Original Stereoscopic Image

Disparity map

Lightness distance

Pixel-based contrast


Distorted Stereoscopic Image

Lightness distance

Pixel-based contrast


Fig. 4. Block diagram of MCM-based cyclopean IQA.

Finally, the 2D-MAD score is given by

S2D-MAD =WL · exp(MADL

100 ) +WR · exp(MADR

100 )

WL +WR, (6)

where MADL and MADR denote the MAD quality scores ofthe left and right views of the distorted stereoscopic image,respectively. The devision 100 is chosen simply to bring thetwo values (MADL and MADR) into an approximate range of[0, 1].

C. MCM-based Cyclopean IQA

In the second stage, the appearance-based strategy (whichhas been used in [1]) is applied to compute the statisticaldifference between the reference and distorted stereoscopicimages by using the cyclopean lightness distance and pixel-based contrast maps. These cyclopean feature images aresynthesized by using stereopairs and an estimated disparitymap based on a multipathway contrast gain-control model[34], [35]. Figure 4 shows a block diagram of this stage.

1) Feature maps: Different from previous 3D IQA algo-rithms that directly analyze image intensity/luminance value,we propose to use the lightness distance and pixel-basedcontrast as our algorithm’s raw inputs for cyclopean-basedquality assessment. In [64], we demonstrated that lightnessdistance can be a useful feature for detecting main subjects inphotos. Here, we use maps of both local and global versionsof that feature for estimating quality. The global lightnessdistance measures how much the lightness of each pixel differsfrom the average lightness of the whole image. The locallightness distance measures how much the lightness of eachpixel differs from the average lightness of a local area aroundthat pixel. As a third feature map, we use the pixel-basedcontrast, which measures how contrast changes within a localarea; larger pixel-based contrast values often indicate edges orsharper regions in an image. These three features are computedvia

f1 (x, y) =∣∣L∗ (x, y)− L̄∗

I

∣∣ (7)

f2 (x, y) =∣∣L∗ (x, y)− L̄∗

B (x, y)∣∣ (8)

f3 (x, y) =L∗ (x, y)

L̄∗B (x, y) +K

(9)

where L∗ denotes the lightness component in the CommissionInternationale de i’ Eclairage (CIE) 1976 (L∗, a∗, b∗) color

space (CIELAB); L̄∗I denotes the average lightness value for

the whole image I; L̄∗B (x, y) denotes the average lightness

value of a 9×9 block centered around pixel (x, y); and K =0.001 is a small constant that prevents division by zero.

2) MCM-based cyclopean feature maps: Given the light-ness distance and pixel-based contrast maps for each viewof the reference and distorted stereoscopic image, we builda disparity-compensated cyclopean maps corresponding tothese two features. First, we compute a disparity map of thereference stereopairs (denoted by D) by using the segment-based stereo matching approach proposed in [65]. Then, thecyclopean feature image is synthesized based on the MCM[34], [35], which assumes that each eye exerts contrast gaincontrol not only on the other eye’s visual signal, but also onthe incoming gain control signal from the other eye, with botheffects in proportion to the eye’s own total contrast energy.

Specifically, MCM employs three gain-control mechanisms:(1) attenuation of signal contrast in the non-dominant eye;(2) stronger direct inhibition from the dominant eye; and(3) stronger indirect inhibition from the dominant eye to thegain control signal coming from the non-dominant eye. LetCL and CR denote the scene contrast presented to the leftand right eye, respectively. By assuming that the left eye isthe dominant eye, then after contrast gain control, the signalstrengths perceived by the two eyes are modeled as:

C ′L = CL

1

1 + εR1+βεL

, C ′R = ηCR

1

1 + αεL1+εR

(10)

where εL = ρCγ1

L , εR = ρ(ηCR)γ1 . ρ is the contrast gain-

control efficiency, η is used to model contrast attenuation inright eye; α and β are used to model the stronger inhibitionto the right eye from the left eye, and γ1 is the transducernonlinearity in the gain-control pathway. According to [35],the perceived contrast of the cyclopean image is given by

C =

[(CL

1

1 + εR1+βεL

)γ2

+

(ηCR

1

1 + αεL1+εR

)γ2] 1

γ2

,

(11)where γ2 is the transducer nonlinearity for the power summa-tion in binocular contrast combination.

Motivated by [35], we employ an MCM-based technique tocompute the cyclopean feature images, and this approach ismainly based on two assumptions. First, we assume that botheyes will dominate equally when the reference stereopairs areviewed. Second, when the distorted stereopairs are presentedto human eyes, the one that receives signal with higher contrastwill dominate in the binocular vision. Based on the twoassumptions, we compute the perceived cyclopean lightnessdistance and pixel-based contrast images for each pixel (x, y)by using Eq. (12), where Cfi(i = 1, 2, 3) denotes the com-puted cyclopean lightness distance and pixel-based contrastimage; dx,y = D(x, y) denotes an estimated disparity index inD; εL(x, y) = ρ[ηLCL(x, y)]

γ1 , εR(x, y) = ρ[ηRCR(x, y)]γ1 ;

CL and CR denote the block-based contrast value computedvia Eq. (3) for the left and right view of a stereo image,respectively.

We set the following parameter values by referring to theexperiment results provided in [66]: γ1 = 1.5; ρ = 10. For the


9

Cfi (x, y) =

[(ηLfi,L(x− dx,y, y)

1

1+εR(x−dx,y,y)

1+βεL(x−dx,y,y)

)γ2

+

(ηRfi,R(x, y)

1

1+αεL(x,y)

1+εR(x,y)

)γ2] 1

γ2

[(ηL

1+εR(x−dx,y,y)

1+βεL(x−dx,y,y)

)γ2

+

(ηR

1+αεL(x,y)

1+εR(x,y)

)γ2] 1

γ2

, (i = 1, 2, 3) (12)

reference stereopairs, we set ηL = ηR = 1, which indicatesthat there is no contrast attenuation coming from the other(non-dominant) eye. For the distorted stereopairs, the dominanteye is first determined by comparing the averaged block-basedcontrast of the two views (i.e., C̄L and C̄R). If the left eyedominates (i.e., C̄L > C̄R), then we set ηL = 1 and ηR =0.9. Otherwise, we set ηL = 0.9 and ηR = 1 to indicate thedominant role of the right eye. Note that the value of 0.9 isalso selected by referring to [66]. Additionally, for simplicitywe set γ2 = 0.5 and α = β = 1, which correspond to supra-linear summation and equal inhibition between the two eyes.Experiment results in our research show that changing of γ 2

does not affect the algorithm performance significantly (seethe online supplement at http://vision.okstate.edu/3dmad/).

3) Compute statistical difference: Based on the cyclopeanfeature images computed from the lightness distance andpixel-based contrast maps, we estimate quality by using localstatistical differences between the cyclopean feature imagesfor the reference and distorted stereopairs by the local standarddeviation, skewness, and kurtosis. This approach of comparinglocal statistics was shown in [1] to be highly effective for quan-tifying the appearances of superthreshold distortions. Here,we propose an alternative approach to map these statisticaldifferences to image quality.

Specifically, in [1], quality is assessed based on a statisticaldifference map, which is computed by linearly combining thethree statistics of the log-Gabor filters subbands at five scales(s ∈ {1, 2, 3, 4, 5}) and four orientations (o ∈ {1, 2, 3, 4}):

η(b) =5∑

s=1

4∑o=1

ws[|σrefs,o (b)− σdst

s,o (b)|+ 2|ςrefs,o (b)

− ςdsts,o (b)|+ |κrefs,o (b)− κdst

s,o (b)|],(13)

where σs,o(b), ςs,o(b), and κs,o(b) denote, respectively, thestandard deviation, skewness, and kurtosis computed for a16×16 block b (with 75% overlap between blocks) in log-Gabor subband corresponding to scale s and orientation o.The scale-specific weights (ws = 0.5, 0.75, 1, 5, and 6) wereobtained through training on A57 database [63], which alsoaccount for the HVS’s performance for coarse scales over finescales. Finally, the statistical difference map is collapsed into asingle scalar that represents the perceived quality of the image.

Instead of using the Eq. (13), we propose to train the threestatistical difference values by using support vector machine(SVM) learning. Specifically, we decompose the cyclopeanfeature images of the reference and distorted stereopairs byusing log-Gabor filters at five scales and four orientations, andthen compute three statistical difference maps for each scale.

This first step is given by

ηφs(b) =

4∑o=1

∣∣φrefs,o (b)− φdst

s,o (b)∣∣ , (14)

where φ represents the three statistics (σ, ς , and κ).Next, we collapse each statistical difference map into a

single scalar, which is given by

φ̄s =

[1

Bφs

∑b

ηφs(b)2

] 12

, (15)

where Bφs denotes the total number of blocks within onestatistical difference map (φ) at a specific scale s (s ∈{1, 2, 3, 4, 5}). Thus, for one distorted cyclopean feature imagewe compute a vector of 15 statistical-difference-based featuresdenoted by

v = [σ̄1, σ̄2, σ̄3, σ̄4, σ̄5, ς̄1, ς̄2, ς̄3, ς̄4, ς̄5, κ̄1, κ̄2, κ̄3, κ̄4, κ̄5].(16)

Given the feature vectors extracted from the cyclopeanfeature images, we then map each vector into a quality scoreby using the trained SVM models. Specifically, we trained ourregression models on the 2D LIVE image quality database[36], which contains 779 distorted images with five typesof distortions: Gaussian blurring, additive white noise, JPEGcompression, JPEG2000 compression, and fast fading. Foreach 2D image, we extracted the same type of features.The only difference is that these statistical-difference-basedfeatures were extracted from feature maps corresponding toone single reference/distorted image, and not the cyclopeanimages. We used the LIBSVM package [67] to implement thetraining. To improve predictive performance, optimal radialbasis function (RBF) kernel parameters were used for theregression.

Note that the SVM-based approach is applied only tothe lightness distance (local and global) cyclopean images,resulting into two computed feature vectors, vf1 and vf2 ; thetwo corresponding quality scores are denoted by S f1 and Sf2 ,respectively. The reason for using only the lightness distancemaps is based on several observations. First, the lightnessdistance map of the two views can be combined (via theMCM model) to effectively predict 3D image quality whena symmetrically-distorted 3D image is viewed. By training ona 2D image database, it is assumed that the given 3D imagehas symmetrically distorted stimuli, and therefore its quality isequally determined by the quality of each stereoscopic view.Thus, training the lightness distance statistical features onLIVE [36] works quite well for those images that containsymmetrically distorted stimuli, but it is less effective onasymmetrically distorted images, because the overall quality


10

TABLE IIPERFORMANCE COMPARISON OF USING DIFFERENT FEATURES/FEATURE

COMBINATIONS TO PREDICT STEREOSCOPIC IMAGE QUALITY. NOTE THATFOR THIS COMPARISON, WE DO NOT CONSIDER THE 2D-MAD QUALITY

ESTIMATE ON STEREOPAIRS, AND ONLY USE THE QUALITY DEGRADATION

MEASUREMENTS OF THE CORRESPONDING CYCLOPEAN FEATURE IMAGES

AS THE FINAL QUALITY ESTIMATE OF THE STEREOSCOPIC IMAGE.

LIVE3D Phase I LIVE3D Phase IICyclopean features CC SROCC CC SROCCGlobal lightness distance (f1) 0.944 0.938 0.888 0.877Local lightness distance (f2) 0.937 0.927 0.893 0.889Lightness distance (f1 + f2) 0.948 0.939 0.899 0.891Pixel-based contrast (f3) 0.909 0.903 0.886 0.879All (f1 + f2 + f3) 0.944 0.937 0.921 0.915

of these images are more determined by the visual signal thathas higher contrast (as shown in Table II). Thus, as mentionedpreviously, we additionally employ the pixel-based contraststatistical features to improve the performance of 3D-MADon asymmetrically distorted images. In this case, we havefound that the 2D LIVE image database is less effective attraining these contrast features, possibly due to the differentcharacteristics that the HVS uses when judging the quality ofa 2D image vs. an asymmetrically distorted 3D image.

In this work, we adopt a similar strategy as in [1] to solvethis problem. Specifically, we first decompose the cyclopeanpixel-based contrast images by using the middle radial fre-quency (third scale) log-Gabor filters, and then compute thestatistical difference map via:

ηf3(b) =

4∑o=1

[|σref3,o (b)− σdst

3,o (b)|+ 2|ςref3,o (b)

− ςdst3,o (b)|+ |κref3,o (b)− κdst

3,o(b)|],(17)

where σ, ς , and κ denote the three statistics. The third scaleof the log-Gabor filter subbands are utilized because we foundthat distortions in the cyclopean pixel-based contrast imagesare more representative and distinguishable in the middle-levelfrequency range. The final scalar value of contrast distortion(denoted by Sf3) is then given by

Sf3 =

[1

B

∑b

ηf3(b)2

] 12

, (18)

where B denotes the total number of blocks within the pixel-based contrast statistical difference map.

Finally, based on all three perceived distortion estimates(Sf1 , Sf2 , Sf3), we compute the MCM-based cyclopean fea-ture image quality (denoted by SCyc-MAD) as

SCyc-MAD =Sf1 + Sf2

10+ Sf3 . (19)

Note that Sf1 and Sf2 are obtained through training on the 2DLIVE image database (that has DMOS values approximatelybetween 0 and 100) and then testing using the cyclopeanfeature images. Thus, both values approximately range from0 to 100. The Sf3 value is obtained directly from analyzingthe cyclopean pixel-based contrast images and approximatelyranges from 0 to 10. Therefore, the division by 10 was chosensimply to bring the three values (Sf1 , Sf2 , and Sf3 ) onapproximately the same scale.

D. 3D-MAD Quality Estimate

Given both the 2D-MAD-based estimate on stereopairs andthe MCM-based estimate on three cyclopean feature images,the final stage of 3D-MAD is to combine them to yield aoverall perceived distortion estimate (denoted by S3D-MAD)of the stereoscopic image. Specifically, S3D-MAD is computedas a product of S2D-MAD and SCyc-MAD , which is given by

S3D-MAD = S2D-MAD × SCyc-MAD. (20)

Smaller values of S3D-MAD denote predictions of betterstereoscopic image quality. A value of S3D-MAD = 0 indicatesthat the distorted stereoscopic image is equal in quality to thereference image.

IV. RESULTS AND ANALYSIS

In this section, we analyze 3D-MAD’s ability to predictstereoscopic image quality. For this task, we test 3D-MAD onthree publicly available 3D image databases: (1) the LIVE 3Dimage database [27], (2) the MCL-3D image database [68],and (3) the IRCCyN/IVC 3D image database [12].

A. Stereoscopic Image Quality Databases

The LIVE 3D image quality database [27], from TheUniversity of Texas at Austin, USA, was constructed in twophases. Phase I contains 20 original images and 365 sym-metrically distorted stereopairs corresponding to five distortiontypes: Gaussian blurring (Gblur), additive white noise (WN),JPEG compression, JPEG2000 compression (JP2K), and sim-ulated packet-loss of transmitted JPEG2000-compressed im-ages, which is also known as fast fading (FF). Phase II containseight original images and 360 distorted stereopairs, with thesame five distortion types. For each distortion type, everyoriginal stereopair was processed to create three symmetricdistorted stereopairs and six asymmetric distorted stereopairs.Both phases contain the ground truth depth/disparity infor-mation for every reference image and the associated DMOSvalues for distorted images.

The MCL-3D image quality database [68], from the Uni-versity of Southern California, USA, contains 693 symmetri-cally distorted stereoscopic image pairs and associated meanopinion score (MOS) values. One third of the database areof resolution 1024×728 and two thirds are of resolution1920×1080. Nine image-plus-depth sources were first se-lected, and a depth image based rendering technique was usedto render stereoscopic image pairs. Four levels of distortionswere applied to either the texture image or the depth imagebefore stereoscopic image rendering, which include Gaussianblur, additive white noise, down-sampling blur (SBLUR),JPEG compression, JPEG2000 compression and transmissionerror (TERROR). The distortion caused by imperfect renderingwas also examined.

The IRCCyN/IVC 3D image database [12], from the Uni-versity of Nantes, France, contains six original images, 15symmetrically distorted versions of each original image, andsubjective ratings of quality for each distorted image (DMOSvalue) obtained by averaging 14 subjective scores (three were


11

discarded). Three types of distortions are included: JPEG com-pression, JPEG2000 compression, and blurring; each distortiontype has five different degradation levels. Note that there aretwo types of blurred images: Gaussian filtered images and re-sampled images (see [12]). The database contains 96 stereo-scopic images, but without depth and disparity information.

B. Algorithms and Performance Measures

We compared 3D-MAD with seven other 3D IQA methods:a SSIM-based stereo IQA model proposed by Benoit et al.[12], a SDP-based method proposed by Ko et al. [53], aBSS weighted 3D IQA method proposed by Wang et al. [55],the cyclopean MS-SSIM proposed by Chen et al. [29], theFrequency-Integrated PSNR (FI-PSNR) proposed by Lin etal. [59], the BJND-based method proposed by Shao et al.[61], and a 3D IQA approach proposed by You et al. [11], inwhich the quality of each stereoscopic image was computedbased on various ways of combining 2D quality estimates onstereopairs and disparity maps. Note that for the methods byYou et al. and by Benoit et al., the combinations leading thebest performances are presented. Specifically, for You et al.’smethod, the quality of each stereopair was estimated by SSIM[37] and the quality of the disparity map was estimated viamean absolute difference; the overall stereoscopic image qual-ity was then computed via a “global combination”, in whichthe optimum parameters were determined by the Levenberg-Marquardt algorithm. For Benoit et al.’s method, a local SSIMmap weighted by the Euclidean-distance-based local disparitydistortion measure was employed to estimate the quality ofeach stereoscopic view, and the final quality estimate was theaverage of the two views. For both algorithms, the same stereomatching algorithm [69] was used to create disparity maps.

We also listed the results of four 2D IQA algorithms forreference: SSIM [37], MS-SSIM [58], VIF [70] and MAD [1](the original 2D version). For these 2D IQA algorithms, thepredicted quality of a stereoscopic image was taken to be theaverage quality predicted from the left and right views.

Before evaluating the performance of a particular quality as-sessment method on a particular database, we applied a logistictransform to bring the prediction values on the same scalesas the DMOS values. The logistic transform recommended byVideo Quality Experts Group [71] is a four-parameter sigmoidgiven by

f(x) =τ1 − τ2

1 + exp(− x−τ3|τ4| )

+ τ2, (21)

where x denotes the raw predicted score, and where τ1, τ2,τ3, and τ4 are free parameters selected to provide the bestfit of the predicted scores to the MOS/DMOS values. Threecriteria were used to measure the prediction monotonicityand prediction accuracy of each algorithm: (1) the SpearmanRank-Order Correlation Coefficient (SROCC), (2) the PearsonLinear Correlation Coefficient (CC), and (3) the Root MeanSquare Error (RMSE) after non-linear regression. Note that thelogistic transform in Eq. (21) will only affect CC and RMSE,not SROCC.

TABLE IVPERFORMANCE OF 3D-MAD AND OTHER IQA ALGORITHMS ON

SYMMETRICALLY AND ASYMMETRICALLY DISTORTED STIMULI IN THELIVE PHASE II DATABASE. ITALICIZED ENTRIES DENOTE 2D

ALGORITHMS. RESULTS OF THE BEST-PERFORMING 3D IQA ALGORITHM

ARE BOLDED.

Symmetric AsymmetricMetric CC SROCC RMSE CC SROCC RMSESSIM [37] 0.839 0.825 6.950 0.760 0.736 6.658MS-SSIM [58] 0.927 0.915 5.990 0.694 0.677 7.589VIF [70] 0.920 0.914 5.291 0.771 0.731 6.513MAD [1] 0.936 0.934 4.687 0.783 0.756 6.379You [11] 0.911 0.898 7.128 0.659 0.604 8.009Benoit [12] 0.921 0.910 5.712 0.746 0.732 6.976Wang [55] 0.868 0.861 6.364 0.768 0.736 6.570Ko [53] 0.896 0.886 6.558 0.679 0.669 7.703Chen [29] 0.937 0.925 4.429 0.875 0.854 4.927Lin [59] 0.816 0.811 8.467 0.588 0.549 8.510Shao [61] 0.931 0.936 5.187 0.762 0.739 6.6443D-MAD 0.954 0.947 3.900 0.903 0.895 4.371

C. Overall Performance

The overall testing results on the LIVE, MCL, and IRC-CyN/IVC 3D image databases are shown in Table III in termsof CC, SROCC, and RMSE. Also included are the testingresults of the first and the second stage of 3D-MAD (denotedby “2D-MAD” and “Cyc-MAD”, respectively) for compari-son. Italicized entries denote 2D IQA algorithms. The resultsof the best-performing 3D IQA algorithm in each case arebolded. As recommended in [68], we tested 648 stereoscopicimages in the MCL-3D database, which cover six types ofdistortions applied on the texture image, the depth image, andboth, respectively. Also note that the MCL-3D image pairs areof large size, which may require different parameter settingsin the disparity estimation algorithm (compared with that onLIVE and IRCCyN/IVC). Thus, we rescaled 2 the images to1/2 (for 1024×728 images) or 1/3 (for 1920×1080 images)of the original size before testing in order to gain a consistentparameter setting, as well as to save computational time.

From Table III, we can see that 3D-MAD outperforms allthe other 2D/3D IQA algorithms in terms of all three per-formance criteria on all databases. Specifically, on the LIVEphase I and MCL-3D databases, which contain symmetricallydistorted stimuli, all these 2D/3D IQA algorithms performquite well, and there seems to be less performance differencebetween 2D and 3D methods. However, these 2D-based IQAalgorithms generally perform worse on the LIVE phase IIdatabase, which partially contains the asymmetrically distortedstimuli. The reason is that these algorithms are utilized toestimate the quality of each stereopair without taking intoaccount the depth/disparity information. To demonstrate thislatter assertion, Table IV shows the CC, SROCC, and RMSEvalues of all these algorithms tested on separated subsetsof symmetrically and asymmetrically distorted images in theLIVE phase II database. Observe from Table IV that 3D-MAD improves upon the performance of the original MADalgorithm, most notably on asymmetrically distorted images.

In comparison, the 3D-based IQA algorithms take into ac-count the stereo depth/disparity information, and/or binocular

2In this paper, we used bicubic interpolation to rescale the images; theoutput pixel value is a weighted average of pixels in the nearest 4 × 4neighborhood.


12

TABLE IIIOVERALL PERFORMANCES OF 3D-MAD AND OTHER 2D/3D IQA ALGORITHMS ON THE LIVE, MCL, AND IRCCYN/IVC 3D IMAGE DATABASES.

ITALICIZED ENTRIES DENOTE 2D ALGORITHMS. RESULTS OF THE BEST-PERFORMING 3D IQA ALGORITHM ARE BOLDED.

SSIM MS-SSIM VIF MAD You Benoit Wang Ko Chen Lin Shao 2D-MAD Cyc-MAD 3D-MADCC LIVE (phase I) 0.872 0.926 0.925 0.942 0.895 0.915 0.888 0.910 0.917 0.864 0.932 0.945 0.944 0.951

LIVE (phase II) 0.801 0.778 0.840 0.854 0.729 0.812 0.817 0.760 0.906 0.658 0.836 0.901 0.921 0.927MCL 0.883 0.915 0.856 0.931 0.837 0.871 0.890 0.893 0.881 0.870 0.848 0.942 0.912 0.934

IRCCyN/IVC 0.736 0.802 0.798 0.802 0.745 0.820 0.725 0.742 0.689 0.695 0.716 0.824 0.878 0.864Average 0.851 0.877 0.866 0.907 0.819 0.864 0.862 0.855 0.884 0.806 0.858 0.925 0.920 0.932

SROCC LIVE (phase I) 0.876 0.922 0.920 0.939 0.896 0.911 0.890 0.907 0.916 0.856 0.927 0.942 0.937 0.944LIVE (phase II) 0.792 0.772 0.817 0.842 0.681 0.806 0.805 0.756 0.901 0.638 0.819 0.890 0.915 0.924

MCL 0.893 0.925 0.858 0.931 0.851 0.874 0.901 0.903 0.884 0.869 0.853 0.942 0.917 0.939IRCCyN/IVC 0.691 0.716 0.706 0.772 0.743 0.791 0.678 0.662 0.673 0.636 0.648 0.811 0.866 0.853

Average 0.852 0.874 0.854 0.902 0.814 0.862 0.861 0.853 0.883 0.794 0.851 0.921 0.919 0.931RMSE LIVE (phase I) 8.033 6.193 6.228 5.498 7.312 6.633 7.536 6.804 6.550 8.242 5.941 5.365 5.406 5.052

LIVE (phase II) 6.757 7.096 6.132 5.869 7.727 6.582 6.502 7.341 4.767 8.496 6.196 4.889 4.393 4.220MCL 1.221 1.051 1.345 0.953 1.424 1.277 1.185 1.172 1.230 1.285 1.380 0.874 1.065 0.930

IRCCyN/IVC 14.936 13.172 13.299 13.176 14.719 12.639 15.191 14.792 15.998 15.860 15.408 12.503 10.578 11.118Average 5.126 4.567 4.476 4.049 5.262 4.618 4.940 4.933 4.336 5.692 4.566 3.698 3.552 3.395

rivalry properties, and thus these algorithms should achieverelatively better performance than 2D-based IQA methods.However, performance of You’s and Benoit’s methods dependlargely on the accuracy of the stereo matching algorithm tocompute the disparity maps, and they are also influenced bythe fact that 2D QA for the disparity maps may not alwayscoincide with human judgments of quality on stereoscopicimages. Although Wang, Ko, and Lin used properties ofbinocular vision, their methods are still based on weightingtwo quality scores obtained by applying 2D QA methods (e.g.,SSIM and PSNR) on both views. Shao’s method is also basedon combining the quality scores obtained from the left andright views, but the combination strategy varies according todifferent image regions (non-corresponding region, binocularsuppression region, and binocular fusion region). Since moreefficient features are employed, this method performs quitewell on the LIVE phase I database, but the performanceis not that high for the other databases. Chen’s methodemploys binocular rivalry and estimates quality based on acyclopean image, and thus achieves better performance thanthe other six algorithms on the LIVE phase II database. Onthe IRCCyN/IVC 3D image database, Benoit’s method and3D-MAD perform better.

Compared with 2D-MAD and Cyc-MAD results, we ob-serve that the combined stages either improve or balancethe performance of each individual stage across different 3Dimage databases. For example, the combined stages can helpincrease the overall performance on the LIVE 3D imagedatabase, but only achieve a performance balance on theMCL and IRCCyN/IVC 3D databases. For MCL, 2D-MADperforms better than Cyc-MAD, which indicates that qualitiesof 3D images in this database are more determined by thequality predictions of stereopairs. For IRCCyN/IVC, such aperformance balance can be attributed to the fact that 2D-MADand all other 2D/3D QA algorithms considered here performless effectively on this database (see Section IV-E for a detailedexplanation), which eventually decreases the overall 3D-MADperformance. This result also partially demonstrates the effec-tiveness of the proposed features and binocular combinationmodels in estimating stereoscopic image quality. Despite that,3D-MAD still performs competitively well, and both stages arerequired to achieve a reasonable performance balance when allthree databases are considered.

-20

-10

0

10

20

30

40

50

60

70

80

-20 -10 0 10 20 30 40 50 60 70 80

DM

OS

Linearized 3D-MAD

20

30

40

50

60

70

80

20 30 40 50 60 70 80

DM

OS

Linearized 3D-MAD

0

2

4

6

8

10

12

14

0 3 6 9 12 15

MO

S

Linearized 3D-MAD

0

10

20

30

40

50

60

70

80

0 10 20 30 40 50 60 70 80

DM

OS

Linearized 3D-MAD

LIVE (phase I) LIVE (phase II)

MCL IRCCyN/IVC

CC=0.951 CC=0.927

CC=0.934 CC=0.864

Fig. 5. Scatter plots of objective scores predicted by 3D-MAD algorithmafter logistic transform versus subjective scores on different image databases.

The last rows of the CC, SROCC, and RMSE results inTable III show the average CC, SROCC, and RMSE, wherethe averages are weighted by the number of distorted imagestested in each database. Also, shown in Figure 5 are scatter-plots of logistic-transformed 3D-MAD quality predictions vs.subjective ratings (MOS/DMOS) on different databases. Inall graphs, the y-axis denotes the subjective ratings of theperceived distortions and the x-axis denotes the predictedquality value transformed via Eq. (21). Despite the presenceof some outliers, the plots are generally heteroscedastic. Insummary, when looking at the overall performance acrossdatabases, 3D-MAD has a better average performance thanother 2D/3D IQA methods.

D. Statistical Significance

A statistical significance test was performed by using anF-test to quantify whether the numerical difference betweenthe IQA algorithms’ performances are statistically significant.


13

The test statistic is the ratio of two algorithms’ residualvariances (errors in predictions), denoted by F = σ 2

A/σ2B . A

smaller residual variance indicates a better prediction. Valuesof F > Fcritical (or F < 1/Fcritical) indicate that at a givenconfidence level, method A has significantly larger (or smaller)residuals than method B, where Fcritical is computed basedon the number of residuals and the confidence level. Notethat if residuals are not Gaussian, then the significance test isoften inconclusive. In this paper, a formal test using Jarque-Bera (JB) statistic [72] is used to measure the Gaussianity ofthe residuals. A smaller value of the JB statistic denotes lessdeviation from Gaussianity, and vice versa.

Table V shows the overall statistical performance of each3D IQA algorithm on the LIVE, MCL, and IRCCyN/IVC3D image databases. Each entry is the ratio of the residualvariance of the algorithm in the row to the algorithm in thecolumn. Bold entries denote that the algorithm in the rowhas a statistically smaller residual variance than the algorithmin the column with confidence greater than 95%. Italicizedentries denote that the algorithm in the row has statisticallygreater residual variance with the same confidence. Plain textentries denote that there is statistically no difference betweenthe residuals of the two predictions. Also contained in TableV are the JB statistic measures of Gaussianity. Italicized JBentries denote that the residuals can be deemed Gaussian with95% confidence. Larger values of the JB statistic denote largerdeviations from Gaussianity.

As shown in Table V, 3D-MAD is statistically the best-performing algorithm on the LIVE phase I, phase II, and MCL-3D databases. On the IRCCyN/IVC database, 3D-MAD hasstatistically the same performance as Benoit’s method, whichmight due to a small number of images being tested, but is stillstatistically better than the others. Also, note that on the LIVEphase I, phase II, and MCL-3D databases, 3D-MAD has moreGaussian residuals than every other algorithm as denoted bythe JB statistic. However, on the IRCCyN/IVC database, 3D-MAD has a relatively higher JB statistic due to several outliers(see Figure 5), and thus the fact that it can still achieve a lowerresidual variance with these outliers is noteworthy.

E. Performance on Individual Distortion Types

We also report the performance of 3D-MAD and other2D/3D IQA algorithms on subsets of the LIVE, MCL, andIRCCyN/IVC 3D image databases corresponding to eachindividual distortion type. Test results of five distortion typesin LIVE, six in MCL, and three in IRCCyN/IVC are pre-sented in Table VI in terms of SROCC values. The CC andRMSE values follow similar trends (see the online supplementat http://vision.okstate.edu/3dmad/). For the CC and RMSEevaluation, the same logistic transform was used and theperformance for each distortion type was evaluated based onextracting the corresponding transformed scores previouslycomputed when the whole database was considered. Boldentries denote the best-performing 3D IQA algorithm for eachdistortion type on each database. Italicized entries denote the2D IQA algorithms.

As shown in Table VI, 3D-MAD provides better predictionson most distortion types in comparison to other 3D IQA

TABLE VIIRUNTIME REQUIREMENTS (SECONDS/IMAGE) FOR 3D-MAD AND OTHER

3D IQA METHODS ON DIFFERENT IMAGE SIZES.

128×128 256×256 512×512 768×768 1024×1024You [11] 0.54±0.03 1.72±0.04 7.25±0.25 14.37±0.48 28.53±0.46Benoit [12] 0.55±0.04 1.72±0.04 7.31±0.12 14.54±0.12 28.91±0.38Wang [55] 0.32±0.01 1.76±0.02 9.55±0.10 22.74±0.18 51.38±0.50Ko [53] 0.03±0.00 0.06±0.00 0.18±0.00 0.34±0.00 0.61±0.02Chen [29] 3.86±0.12 10.29±0.17 37.82±0.24 80.72±0.36 145.57±0.73Lin [59] 0.06±0.02 0.15±0.01 0.68±0.01 1.61±0.03 2.92±0.06Shao [61] 4.91±0.51 20.19±2.48 96.21±8.71 257.81±25.80 514.64±77.793D-MAD-I 0.25±0.01 0.83±0.02 4.02±0.03 9.12±0.03 17.86±0.133D-MAD-II 0.63±0.01 2.69±0.04 13.04±0.08 31.16±0.18 58.01±0.303D-MAD 0.88±0.02 3.52±0.04 17.06±0.09 40.29±0.19 75.89±0.31

algorithms. Compared with the 2D IQA algorithm results(especially the original MAD), however, we observe that 3D-MAD is not always performing better, and even slightly worseon certain distortion types (e.g., JPEG and JPEG2000 com-pression). This interesting observation, together with resultsin Tables III and IV, might indicate that the simple strategy ofaveraging quality estimates of two stereoscopic views worksefficiently on evaluating degradation levels of certain distortiontypes symmetrically presented on each stereopair. However,this strategy has difficulty in dealing with asymmetricallydistorted stimuli, and also difficulty in bringing all qualitymeasures on the same scale when viewing the database as awhole. Also, note that all 2D/3D IQA algorithms consideredhere seem to have relatively worse quality predictions onblurred images in the IRCCyN/IVC database. The reason isthat these IQA algorithms are insensitive to the resampledimages, which is also considered as a blur distortion in theIRCCyN/IVC database. In summary, when looking at theperformance on individual distortion types, 3D-MAD is stillthe best overall choice to estimate stereoscopic image quality.

F. Computational Analysis

To determine the computational runtime of each stage in the3D-MAD algorithm (denoted by 3D-MAD-I and 3D-MAD-II,respectively), we measured the runtime required to computethe qualities of 45 images with different sizes (128×128,256×256, 512×512, 768×768, and 1024×1024). We alsocompared the overall computational time of 3D-MAD withother 3D IQA algorithms on these images. Note that all ofthese tests were performed by running unoptimized MATLABcode on a modern desktop computer (AMD Phenom II X4965 Processor at 3.39 GHz, 4 GB RAM, Windows 7 Pro 64-bit). The results are shown in Table VII in terms of averageseconds per image followed by the standard deviation.

As shown in Table VII, the second stage of 3D-MADconsumes a much longer time than the first stage. This is dueto the fact that in the second stage, the disparity map of thereference stereopairs is computed and six cyclopean featureimages are synthesized. Compared with other 3D IQA algo-rithms, 3D-MAD is faster than Chen’s and Shao’s approaches,but is slower than the others, which can be attributed tothe more complex analysis on stereoscopic images. However,considering 3D-MAD’s improved predictive performance, webelieve that the time cost is justified.


14

TABLE VSTATISTICAL SIGNIFICANCE RELATIONSHIPS (RATIO OF RESIDUAL VARIANCE) BETWEEN 3D IQA ALGORITHMS ON THE LIVE, MCL, AND

IRCCYN/IVC 3D IMAGE DATABASES. SEE TEXT FOR DETAILS.

You Benoit Wang Ko Chen Lin Shao 3D-MAD You Benoit Wang Ko Chen Lin Shao 3D-MADLIVE (phase I) LIVE (phase II)

You – 1.215 0.941 1.155 1.246 0.787 1.515 2.095 – 1.378 1.412 1.108 2.627 0.827 1.555 3.352Benoit 0.823 – 0.775 0.950 1.025 0.648 1.246 1.723 0.726 – 1.025 0.804 1.906 0.600 1.128 2.432Wang 1.062 1.291 – 1.227 1.324 0.836 1.609 2.225 0.708 0.976 – 0.784 1.860 0.586 1.101 2.373Ko 0.866 1.052 0.815 – 1.079 0.681 1.312 1.814 0.903 1.244 1.275 – 2.372 0.747 1.404 3.026Chen 0.802 0.975 0.755 0.927 – 0.632 1.216 1.681 0.381 0.525 0.538 0.422 – 0.315 0.592 1.276Lin 1.271 1.544 1.196 1.467 1.583 – 1.925 2.661 1.209 1.666 1.707 1.339 3.176 – 1.880 4.053Shao 0.660 0.802 0.621 0.762 0.823 0.519 – 1.383 0.643 0.886 0.908 0.712 1.690 0.532 – 2.1563D-MAD 0.477 0.580 0.449 0.551 0.595 0.376 0.723 – 0.298 0.411 0.421 0.330 0.784 0.247 0.464 –JBSTAT 3.3 8.2 19.4 6.5 45.6 1.9 2.0 1.2 9.2 45.2 8.3 30.5 31.5 8.8 6.0 0.4

MCL IRCCyN/IVCYou – 1.243 1.444 1.477 1.342 1.229 1.066 2.346 – 1.356 0.939 0.990 0.847 0.861 0.913 1.753Benoit 0.804 – 1.162 1.188 1.079 0.989 0.857 1.887 0.737 – 0.692 0.730 0.624 0.635 0.673 1.292Wang 0.692 0.861 – 1.022 0.929 0.851 0.738 1.624 1.065 1.445 – 1.055 0.902 0.917 0.972 1.867Ko 0.677 0.842 0.978 – 0.909 0.832 0.722 1.589 1.010 1.370 0.948 – 0.855 0.870 0.922 1.770Chen 0.745 0.926 1.076 1.100 – 0.916 0.794 1.748 1.181 1.602 1.109 1.170 – 1.017 1.078 2.070Lin 0.814 1.012 1.175 1.201 1.092 – 0.867 1.909 1.161 1.575 1.090 1.150 0.983 – 1.060 2.035Shao 0.938 1.166 1.355 1.385 1.259 1.153 – 2.201 1.096 1.486 1.029 1.085 0.928 0.944 – 1.9213D-MAD 0.426 0.530 0.616 0.629 0.572 0.524 0.454 – 0.571 0.774 0.536 0.565 0.483 0.491 0.521 –JBSTAT 6.8 7.7 21.1 7.3 28.0 3.8 10.3 5.4 2.0 47.2 6.1 19.2 3.2 7.9 10.4 26.7

TABLE VISROCC VALUES OF 3D-MAD AND 2D/3D IQA ALGORITHMS ON DIFFERENT TYPES OF DISTORTION ON THE LIVE, MCL, AND IRCCYN/IVC 3D

DATABASES. ITALICIZED ENTRIES DENOTE 2D IQA ALGORITHMS. RESULTS OF THE BEST-PERFORMING 3D IQA ALGORITHM ARE BOLDED.

SSIM MS-SSIM VIF MAD You Benoit Wang Ko Chen Lin Shao 3D-MADLIVE (phase I) JP2K 0.857 0.898 0.902 0.925 0.884 0.887 0.870 0.891 0.896 0.839 0.883 0.916

JPEG 0.436 0.599 0.582 0.736 0.547 0.565 0.445 0.527 0.558 0.207 0.599 0.700WN 0.938 0.942 0.932 0.950 0.929 0.939 0.939 0.933 0.948 0.928 0.930 0.950

GBLUR 0.879 0.928 0.931 0.954 0.910 0.911 0.918 0.941 0.926 0.935 0.910 0.942FF 0.586 0.735 0.804 0.772 0.629 0.683 0.654 0.756 0.688 0.658 0.793 0.833

LIVE (phase II) JP2K 0.703 0.817 0.826 0.869 0.834 0.842 0.727 0.902 0.833 0.719 0.788 0.895JPEG 0.679 0.827 0.778 0.839 0.755 0.839 0.694 0.728 0.840 0.613 0.745 0.866

WN 0.920 0.947 0.820 0.885 0.878 0.926 0.934 0.900 0.955 0.907 0.807 0.952GBLUR 0.836 0.801 0.950 0.924 0.275 0.766 0.882 0.836 0.910 0.711 0.939 0.942

FF 0.835 0.830 0.934 0.918 0.740 0.862 0.865 0.811 0.889 0.701 0.935 0.922MCL JP2K 0.900 0.928 0.933 0.930 0.919 0.927 0.904 0.897 0.902 0.919 0.862 0.930

JPEG 0.885 0.895 0.895 0.885 0.803 0.823 0.888 0.898 0.857 0.882 0.888 0.899WN 0.912 0.928 0.944 0.952 0.855 0.886 0.915 0.919 0.889 0.859 0.890 0.950

GBLUR 0.912 0.936 0.952 0.953 0.881 0.916 0.913 0.920 0.893 0.895 0.873 0.970SBLUR 0.905 0.938 0.954 0.972 0.891 0.944 0.918 0.930 0.920 0.954 0.859 0.972

TERROR 0.891 0.908 0.789 0.892 0.871 0.842 0.874 0.890 0.903 0.815 0.889 0.903IRCCyN/IVC JP2K 0.875 0.941 0.924 0.965 0.820 0.947 0.910 0.857 0.823 0.885 0.802 0.972

JPEG 0.832 0.968 0.956 0.915 0.842 0.916 0.855 0.863 0.775 0.782 0.896 0.918BLUR 0.408 0.431 0.321 0.535 0.556 0.496 0.470 0.417 0.459 0.467 0.321 0.789

V. CONCLUSION

This paper presented an algorithm to evaluate stereoscopicimage quality, called 3D-MAD, which extends our previousMAD algorithm from 2D to 3D. The proposed algorithmoperates via two main stages. In the first stage, the origi-nal MAD algorithm is employed to assess quality of eachstereoscopic view (left view and right view) of a stereoscopicimage, and then the two obtained quality measurements arelinearly combined through two averaged block-based contrastvalues (computed from each channel) to quantify the stereopairimage quality degradation. In the second stage, disparity-compensated cyclopean feature images are synthesized fromthe lightness distance maps and the pixel-based contrast mapby using a multipathway contrast gain-control model. Accord-ingly, three cyclopean feature images are generated from boththe reference and distorted stereopairs, respectively. Thesedistorted cyclopean feature images are then compared againstthe reference cyclopean feature images, and the computedstatistical difference maps/features are employed to predict thecyclopean feature image quality degradation. Finally, qualityestimates from both the stereopair images and cyclopean

feature images are combined to yield an overall 3D imagequality prediction. We acknowledge that there are many pa-rameters available in the underlying models, which could bea drawback to the implementation of the proposed method.However, most of these parameters are predetermined (fixed)by the prior work, and there are only four empirical valueswe have selected. Experimental results on various databasesdemonstrated that 3D-MAD can achieve better performancethan many other 2D/3D IQA algorithms.

REFERENCES

[1] E. C. Larson and D. M. Chandler, “Most apparent distortion: full-reference image quality assessment and the role of strategy,” Journal ofElectronic Imaging, vol. 19, no. 1, pp. 011006, 2010.

[2] W. Lin and C-C Jay Kuo, “Perceptual visual quality metrics: A survey,”Journal of Visual Communication and Image Representation, vol. 22,no. 4, pp. 297–312, 2011.

[3] H. R. Sheikh, M. F. Sabir, and A. C. Bovik, “A statistical evaluationof recent full reference image quality assessment algorithms,” IEEETransactions on Image Processing, vol. 15, no. 11, pp. 3440–3451, Nov.2006.

[4] D. M. Chandler, “Seven challenges in image quality assessment: past,present, and future research,” ISRN Signal Processing, vol. 2013, ArticleID 905685, 53 pages, 2013.


15

[5] M. Lambooij, W. IJsselsteijn, M. Fortuin, and I. Heynderickx, “Visualdiscomfort and visual fatigue of stereoscopic display; a review,” Journalof Imaging Science and Technology, vol. 53, no. 3, pp. 030201/114,2009.

[6] O. J. Braddick, “Binocular single vision and perceptual processing,”Proceedings of the Royal Society of London. Series B, BiologicalSciences, vol. 204, no. 1157, pp. 503–512, 1979.

[7] S. L. P. Yasakethu, C. T. E. R. Hewage, W. A. C. Fernando, and A. M.Kondoz, “Quality analysis for 3D video using 2D video quality models,”IEEE Transactions on Consumer Electronics, vol. 54, no. 4, pp. 1169–1176, 2008.

[8] C. T. E. R. Hewage, S. T. Worrall, S. Dogan, and A. M. Kondoz,“Prediction of stereoscopic video quality using objective quality modelsof 2-D video,” Electronics Letters, vol. 44, no. 16, pp. 963–965, 2008.

[9] P. Gorley and N. Holliman, “Stereoscopic image quality metrics andcompression,” in Proceedings of SPIE, Feb. 2008, vol. 6803.

[10] P. Campisi, P. L. Callet, and E. Marini, “Stereoscopic images qualityassessment,” in 15th European Signal Processing Conference, Pozan,Poland, Sep. 3-7 2007.

[11] J.-Y. You, L.-Y. Xing, A. Perkis, and X. Wang, “Perceptual qualityassessment for stereoscopic images based on 2D image quality metricsand disparity analysis,” in Proceedings of International Workshop VideoProcess Quality Metrics for Consumer Electronics, 2010.

[12] A. Benoit, P. L. Callet, P. Campisi, and R. Cousseau, “Qualityassessment of stereoscopic images,” EURASIP Journal on Image andVideo Processing, vol. 2008, pp. 1–13, 2009.

[13] R. G. Kaptein, A. Kuijsters, Marc T. M. Lambooij, W. A. IJsselsteijn,and I. Heynderickx, “Performance evaluation of 3D-TV systems,” inProceedings of SPIE: Image Quality and System Performance V, Jan.2008, vol. 6808.

[14] L. Goldmann and T. Ebrahimi, “3D quality is more than just the sumof 2D and depth,” in IEEE International workshop on hot topics in 3D,2010.

[15] Jeremy M. Wolfe, “Stereopsis and binocular rivalry,” PsychologicalReview, vol. 93, no. 3, pp. 269–282, 1986.

[16] Willem J. M. Levelt, “The alternation process in binocular rivalry,”British Journal of Psychology, vol. 57, pp. 225–238, 1966.

[17] R. Fox and F. Rasche, “Binocular rivalry and reciprocal inhibition,”Perception and Psychophysics, vol. 5, pp. 215–217, 1969.

[18] M. Hollins, “The effect of contrast on the completeness of binocularrivalry suppression,” Perception and Psychophysics, vol. 27, pp. 550–556, 1980.

[19] Timothy J. Andrews and D. Purves, “Similarities in normal andbinocularly rivalrous viewing,” in Proceedings of the National Academyof Science of the USA, 1997, vol. 94, pp. 9905–9908.

[20] Robert P O’Shealf, Randolph Blake, and Jeremy M Wolfe, “Binocularrivalry and fusion under scotopic luminances,” Perception, vol. 23, pp.771–784, 1994.

[21] Vincent A Nguyen, Alan W Freeman, and David Alais, “Increasingdepth of binocular rivalry suppression along two visual pathways,”Vision research, vol. 43, no. 19, 2003.

[22] S. Takase, S. Yukumatsu, and K. Bingushi, “Local binocular fusion isinvolved in global binocular rivalry,” Vision research, vol. 48, no. 17,pp. 1798–1803, 2008.

[23] M. Vergeer and R. van Lier, “Feature-based activation and suppressionduring binocular rivalry,” Vision research, vol. 50, no. 8, pp. 743–749,2010.

[24] J. Brascamp, H. Sohnc, S.-H. Lee, and R. Blake, “A monocular contri-bution to stimulus rivalry,” in Proceedings of the National Academy ofScience of the USA, 2013, vol. 110, pp. 8337–8344.

[25] A. Bartels and N. K. Logothetis, “Binocular rivalry: A time dependenceof eye and stimulus contributions,” Journal of Vision, vol. 10, pp. 3,1–14, 2010.

[26] Georgeson MA and Wallis SA, “Binocular fusion, suppression anddiplopia for blurred edges,” Ophthalmic and Physiological Optics, vol.34, no. 2, pp. 163–185, 2014.

[27] A. K. Moorthy, C.-C. Su, A. Mittal, and A. C. Bovik, “Subjectiveevaluation of stereoscopic image quality,” Signal Processing: ImageCommunication, vol. 28, pp. 870–873, 2012.

[28] A. Maalouf and M. C. Larabi, “CYCLOP: A stereo color image qualityassessment metric,” in IEEE International Conference on Acoustics,Speech and Singal Processing, 2011, pp. 1161–1164.

[29] M.-J. Chen, C.-C. Su, D.-K. Kwon, L. K. Cormack, and A. C. Bovik,“Full-reference quality assessment of stereopairs accounting for rivalry,”Signal Processing: Image Communication, vol. 28, no. 9, pp. 1143–1155, 2013.

[30] Willem J. M. Levelt, “On binocular rivalry,” in Soesterberg (TheNetherlands): Institute for Perception RVO-TNO. 1965, National De-fence Research Organization TNO.

[31] R. Blake, “A primer on binocular rivalry, including current controver-sies,” Brain and mind, vol. 2, no. 1, pp. 5–38, 2001.

[32] T. J. Mueller and R. Blake, “A fresh look at the temporal dynamics ofbinocular rivalry,” Biological Cybernetics, vol. 61, pp. 223–232, 1989.

[33] C. J. Bossink, P. F. Stalmeier, and C. M. De Weert, “A test of leveltssecond proposition for binocular rivalry,” Vision Research, vol. 33, pp.1413–1419, 1993.

[34] C.-B. Huang, J.-W. Zhou, Y.-F. Zhou, and Z.-L. Lu, “Contrast and phasecombination in binocular vision,” PLoS One, vol. 5, pp. e15075, 2010.

[35] C.-B. Huang, J.-W. Zhou, Z.-L. Lu, and Y.-F. Zhou, “Deficient binocularcombination reveals mechanisms of anisometropic amblyopia: Signalattenuation and interocular inhibition,” Journal of Vision, vol. 11, pp.4, 1–17, 2011.

[36] H. R. Sheikh, Z. Wang, A. C. Bovik, and L. K. Cormack,“Image and video quality assessment research at LIVE,” Online,http://live.ece.utexas.edu/research/quality/.

[37] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Imagequality assessment: From error visibility to structural similarity,” IEEETransactions on Image Processing, vol. 13, no. 4, pp. 600–612, Apr.2004.

[38] Z. Wang and A. Bovik, “A universal image quality index,” IEEE SignalProcessing Letters, vol. 9, pp. 81–84, March 2002.

[39] M. Carnec, P. L. Callet, and D. Barba, “An image quality assessmentmethod based on perception of structural information,” in Proceedingsof International Conference on Image Processing, 2003, vol. 3, pp. III–185.

[40] Z. Wang and E. P. Simoncelli, “Reduced-reference image qualityassessment using a wavelet-domain natural image statistic model,” inProceedings of SPIE Human Vision and Electronic Imaging, Jan. 2005,vol. 5666.

[41] D. G. Lowe, “Object recognition from local scale-invariant features,” inProceedings of the seventh IEEE International conference on Computervision, 1999, vol. 2, pp. 1150–1157.

[42] M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigmfor model fitting with applications to image analysis and automatedcartography,” Communications of the ACM, vol. 24, no. 6, pp. 381–395,1981.

[43] J.-C. Yang, C.-P. Hou, Z.-Y. Zhang Y. Zhou, and J.-C. Guo, “Objectivequality assessment method of stereo images,” in 3DTV Conference: TheTrue Vision-Capture, Transmission and Display of 3D Video, 2009, pp.1–4.

[44] ANSI T1.TR.74-2001, Objective Video Quality Measurement Using aPeak-Signal-to-Noise-Ratio (PSNR) Full Reference Technique, 2001.

[45] M. H. Pinson and S. Wolf, “A new standardized method for objectivelymeasuring video quality,” IEEE Transactions on Broadcasting, vol. 50,pp. 312– 322, 2004.

[46] J. J. Hwang and H. R. Wu, “Stereo image quality assessment using visualattention and distortion predictors.,” KSII Transactions on Internet &Information Systems, vol. 5, no. 9, 2011.

[47] L.-Y. Xing, J.-Y. You, T. Ebrahimi, and A. Perkis, “A perceptual qualitymetric for stereoscopic crosstalk perception,” in 17th IEEE InternationalConference on Image Processing, 2010, pp. 4033–4036.

[48] K. Ha and M. Kim, “A perceptual quality assessment metric usingtemporal complexity and disparity information for stereoscopic video,”in 18th IEEE International Conference on Image Processing, 2011, pp.2525–2528.

[49] Z. M. P. Sazzad, S. Yamanaka, and Y. Horita, “Continuous stereoscopicvideo quality evaluation,” in Proceedings of SPIE Stereoscopic Displaysand Applications XXI, 2010, pp. 75241E–75241E.

[50] A. Tikanmaki, A. Gotchev, A. Smolic, and K. Miller, “Quality assess-ment of 3d video in rate allocation experiments,” in IEEE InternationalSymposium on Consumer Electronics, 2008, pp. 1–4.

[51] Z. Wang, L. Lu, and A. C. Bovik, “Video quality assessment based onstructural distortion measurement,” Signal Processing: Image Commu-nication, vol. 19, no. 2, 2004.

[52] A. Boev, A. Gotchev, A. Aksay K. Egiazarian, and G. B. Akar,“Towards compound stereo-video quality metric: a specific encoder-based framework,” in IEEE Southwest Symposium on Image Analysisand Interpretation, 2006, pp. 218–222.

[53] H. Ko, C.-S. Kim, S. Y. Choi, and C.-C. Jay Kuo, “3D image qualityindex using SDP-based binocular perception model,” in 11th IEEEWorkshop on 3D Image/Video Technologies and Applications, 2013.


16

[54] S. Ryu, D. H. Kim, and K. Sohn, “Stereoscopic image quality metricbased on binocular perception model,” in IEEE International Conferenceon Image Processing, 2012, pp. 609–612.

[55] X. Wang, S. Kwong, and Y. Zhang, “Considering binocular spatial sensi-tivity in stereoscopic image quality assessment,” Visual Communicationsand Image Processing, pp. 1–4, 2011.

[56] F. Qi, T.-T. Jiang, S.-W. Ma, and D.-B. Zhao, “Quality of experienceassessment for stereoscopic images,” in IEEE International Symposiumon Circuits and Systems, 2012, pp. 1712–1715.

[57] R. Bensalma and M. Larabi, “A perceptual metric for stereoscopic imagequality assessment based on the binocular energy,” MultidimensionalSystems and Signal Processing, vol. 24, no. 2, pp. 281–316, 2013.

[58] Z. Wang, E. P. Simoncelli, and A. C. Bovik, “Multiscale structuralsimilarity for image quality assessment,” in Conference Record of theThirty-Seventh Asilomar Conference on Signals, Systems and Comput-ers, Nov. 2003, vol. 2, pp. 1398–1402.

[59] Y.-H. Lin and J.-L. Wu, “Quality assessment of stereoscopic 3D imagecompression by binocular integration behaviors,” IEEE Transactions onImage Processing, vol. 23, no. 4, pp. 1527–1542, 2014.

[60] F. Shao, G. Jiang, M. Yu, F. Li, Z. Peng, and R. Fu, “Binocular energyresponse based quality assessment of stereoscopic images,” DigitalSingal Processing, vol. 29, pp. 45–53, 2014.

[61] F. Shao, W.-S. Lin, S.-B. Gu, G.-Y. Jiang, and T. Srikanthan, “Perceptualfull-reference quality assessment of stereoscopic images by consideringbinocular visual characteristics,” IEEE Transactions on Image Process-ing, vol. 22, no. 5, pp. 1940–1953, 2013.

[62] R. Blake and K. Boothroyd, “The precedence of binocular fusion overbinocular rivalry,” Perception and Psychophysics, vol. 37, no. 2, pp.114–124, 1985.

[63] D. M. Chandler and S. S. Hemami, “VSNR: A wavelet-based visualsignal-to-noise ratio for natural images,” IEEE Transactions on ImageProcessing, vol. 16, no. 9, pp. 2284–2298, Sep. 2007.

[64] C. Vu and D. M. Chandler, “Main subject detection via adaptive featurerefinement,” Journal of Electronic Imaging, vol. 20, no. 1, pp. 013011,2011.

[65] A. Klaus, M. Sormann, and K. Karner, “Segment-based stereo matchingusing belief propagation and a self-adapting dissimilarity measure,” inProceedings of the 18th International Conference on Pattern Recogni-tion, 2006, vol. 3, pp. 15–18.

[66] F. Hou, C.-B. Huang, J. Liang, Y.-F. Zhou, and Z.-L. Lu, “Contrast gain-control in stereo depth and cyclopean contrast perception,” Journal ofvision, vol. 13, no. 8, pp. 3, 2013.

[67] C.-C. Chang and C.-J. Lin, “LIBSVM: a library for support vectormachines,” ACM Transactions on Intelligent System Technology, vol. 2,no. 3, pp. 27:1–27:27, 2001.

[68] R. Song, H. Ko, and C. C. Jay Kuo, “MCL-3D: a databasefor stereoscopic image quality assessment using 2d-image-plus-depthsource,” Journal of Visual Communication and Image Representation,arXiv:1405.1403 [cs.CV], 2014.

[69] P. F. Felzenszwalb and D. P. Huttenlocher, “Efficient belief propagationfor early vision,” International journal of computer vision, vol. 70, no.1, pp. 41–54, 2006.

[70] H. R. Sheikh and A. C. Bovik, “Image information and visual quality,”IEEE Transactions on Image Processing, vol. 15, no. 2, pp. 430–444,Feb. 2006.

[71] VQEG, “Final report from the video quality experts group on thevalidation of objective models of video quality assessment, phaseII,” August 2003, http://www.its.bldrdoc.gov/vqeg/projects/frtv-phase-ii/frtv-phaseii.aspx.

[72] A. K. Bera and C. M. Jarque, “Efficient tests for normality, homoscedas-ticity and serial independence of regression residuals,” ElectronicsLetters, vol. 6, pp. 255–259, 1980.


3D-MAD: A Full Reference Stereoscopic Image Quality...

Documents

Transcript of 3D-MAD: A Full Reference Stereoscopic Image Quality...