Research Article Nonintrusive Method Based on Neural...

Research ArticleNonintrusive Method Based on Neural Networks forVideo Quality of Experience Assessment

Diego Joseacute Luis Botia Valderrama and Natalia Gaviria Goacutemez

Engineering Department Universidad de Antioquia Medellın Colombia

Correspondence should be addressed to Diego Jose Luis Botia Valderrama diegobotiagmailcom

Received 11 August 2015 Revised 4 December 2015 Accepted 17 December 2015

Academic Editor Stefanos Kollias

Copyright copy 2016 D J L Botia Valderrama and N Gaviria Gomez This is an open access article distributed under the CreativeCommons Attribution License which permits unrestricted use distribution and reproduction in any medium provided theoriginal work is properly cited

The measurement and evaluation of the QoE (Quality of Experience) have become one of the main focuses in the telecommunica-tions to provide services with the expected quality for their users However factors like the network parameters and codification canaffect the quality of video limiting the correlation between the objective and subjective metricsThe above increases the complexityto evaluate the real quality of video perceived by users In this paper a model based on artificial neural networks such as BPNNs(Backpropagation Neural Networks) and the RNNs (RandomNeural Networks) is applied to evaluate the subjective quality metricsMOS (Mean Opinion Score) and the PSNR (Peak Signal Noise Ratio) SSIM (Structural Similarity Index Metric) VQM (VideoQuality Metric) and QIBF (Quality Index Based Frame) The proposed model allows establishing the QoS (Quality of Service)based in the strategy Diffserv The metrics were analyzed through Pearsonrsquos and Spearmanrsquos correlation coefficients RMSE (RootMean Square Error) and outliers rate Correlation values greater than 90 were obtained for all the evaluated metrics

1 Introduction

The assessment of quality in digital video systems is a topicof great interest to the telecommunications companies thathope to increase the quality to their users The QoE (Qualityof Experience) is the degree of userrsquos satisfaction with anykind of multimedia service This concept has been defined indifferent ways by various authors Liu et al [1] suggested thatQoE involves two aspects

(i) The monitoring of the userrsquos experience online

(ii) The service control to ensure that QoS (Quality ofService) can satisfy the userrsquos requirements

QoE is an extension of QoS since the former providesinformation about the services delivery from point of viewof the end users QoE refers to personal preferences of usersand so seeks to assess the subjective perception of the receivedservice [2 3] However this perception is influenced by thenetwork performance in terms of QoS and video encodingparameters Different methodologies proposed in the liter-ature aim at estimating subjective Quality of Experience

through the assessment of different metrics of video qualitygenerally using objective methods

In the implementation of digital television platforms (egIPTV and DVB) some important restrictions can affect themanagement and proper operation of the network Some ofthese are

(i) the large amount of bandwidth the user shouldcontract

(ii) the limitation of internal buffers in routers and STB(Set Top Box) which can generate problems suchas packet loss that are critical on video or audiotransmission

(iii) the type of video compression format which willreduce the channel use without affecting the quality

(iv) other items that should be installed and properlyconfigured such as the last mile link used and theadmission control class used

Different researches have proposed quality assessmentmodels for video streams and measurement strategies in

Hindawi Publishing CorporationAdvances in MultimediaVolume 2016 Article ID 1730814 17 pageshttpdxdoiorg10115520161730814

2 Advances in Multimedia

order to identify the optimal values for each metric guaran-teeing the experience of the viewer

According to Winkler [4] there are projects for QoEassessment VQEG (Video Quality Experts Group) QoSM(Quality of ServiceMetrics) fromATIS IPTV InteroperabilityForum and specificmetrics such as those oriented to packetsa bitstream hybrids and images metrics [5ndash8] but these aremore complex and the correlation methods have not yetbeen applied for real time services assessment Other workspropose a new relationship between KPIs (Key PerformanceIndexes)withQoE assessment onmobile environments but itcan be applied in other scenarios as fixed broadbandnetworksfocused particularly on telecommunications providers [9]

One of the main problems on the estimation of thevideo subjective quality is the lack of proper estimationor correlation models These must guarantee results withaccuracy but in many cases heavily depend on objectivemetrics [10ndash12] Also the correlation models in QoE metricsare not accurate and reliable

According to [13] three strategies were developed toperform the estimation of video quality The first one is toapply a subjective assessment with a selected group of peoplethe main drawback of the evaluations is the cost and timeThe second one is the objective quality metrics assessmentfor video in which the principal disadvantage is the lowcorrelation with regard to subjective quality metrics [14]in addition such metrics obviate the network and contentparameters The last one is to use machine learning methodsto analyse the objective and subjective assessment but themajor drawback is the difficult for configuring and testingfurthermore some methods may fail whether a suitabledesign and optimal parameter selection are not performed

According to Kuipers et al [15] the minimum thresholdof accepted quality is a MOS (Mean Opinion Score) value of35 The subjective tests (such as MOS) are widely useful inassessing the users satisfaction due to its accuracy howeverthe application of them is still very complex due to thehigh consumption of time and money therefore they areimpractical for tasks of testing in network devices and acontrolled environment is required is required (in some casesits implementation are complex)

Also if we want to develop trafficmanagement techniquesin real time it is necessary to find a relationship amongthem and the objective metrics measurable by the networkequipment

Objective methods are based on algorithms for theassessment of video quality making them less complexand furthermore can be performed on controlled simula-tion environments The objective metrics are supported inmathematical models that approximate themselves to theHuman Vision System (HVS) behavior and therefore try toestimate as accurately as possible the true QoE Howeverthe perception of each viewer is highly influenced by thequality of the data network expressed by the QoS parameters[26] A lot of proposals for assessing QoE metrics have beencreated to define the userrsquos experience ITU-T has carried outstandards for some of them [27 28]

Due to complex factors such as the HVS different kindsof solutions such as the implementation of machine learning

techniques have been proposed Some of themost usedmeth-ods are Artificial Neural Network (ANN) fuzzy logic basedon rules neural-fuzzy networks (eg ANFIS) support vectormachines (SVM) Gaussian processes and genetic algorithmamong others Nonintrusive QoE estimation methods forvideo are mainly based on application layer and networkparameters

Models as artificial neural networks have been littlestudied for estimation and prediction of video quality For theevaluation of nonintrusive methods we decided to employtwo methods of machine learning BPNN feedforward andRNN (Random Neural Network) We assess the output ofeach system estimated MOS versus expected MOS In eachcase the fit of the correlation determined by Pearson andSpearman rank order correlation coefficients RMSE (RootMean Square Error) and Outliers Ratio were calculated Theexpected MOS was calculated from the VQM (Video QualityMetric) SSIM (Structural Similarity Index Metric) PSNR(Peak Signal Noise Ratio) and QIBF (Quality Index BasedFrame) metrics

This paper presents the key objective and subjectivequality metrics and introduces a nonintrusive QoE assess-ment methodology based on machine learning techniquesshowing its main features and functionality Afterwards thedeveloped testbed is explained both results obtained as theiranalyses are presented Finally the main conclusions andfurther works will be shown

2 Related Works

21 Methods for Quality of Experience Assessment In spiteof all proposals for objective metrics they are always notclose to the human perception due to the fact that theperception is highly influenced by the performance of thenetwork defined in terms of QoS parameters According to[29 30] the objective metrics are computational models thatpredict the image quality perceived by any person and can beclassified as intrusive and nonintrusive methods as shown inFigure 1

The Subjective Pseudo Quality Assessment model orPSQA (Pseudo Subjective Quality Assessment) is an exampleof nonintrusive method (NR) This model uses an RNN(Random Neural Network) [31ndash33] to learn and recognizethe relationship between video and the characteristics of thenetwork with the quality perceived by users

Initially for the training process of the RNN a databaseis required which contains different sequences to assess thedistortions generated by several QoS and coding parametersAfterwards the training of the RNNwith any video sequenceis evaluated in order to validate the MOS measure

22 QoE Metrics

221 PSNR and MSE Metrics The PSNR (Peak Signal NoiseRatio) and MSE (Mean Square Error) metrics are the mostfrequently applied ones [11] They assess the quality of thereceived video sequence and thus can be mapped on asubjective scale PSNR aiming to compare pixel-by-pixel andframe-by-frame the quality of the received image with the

Advances in Multimedia 3

QoE assessment

Subjective methods

Objective methods

Intrusive methods

Nonintrusivemethods

MOSDMOSDSISDSCQSSSCQEACRACR-HR

PSNRQ metricVQMSSIMMDIandso forth

Based on ANN ANFIS [24]RNNPSQA (Pseudo Subjective Quality Assessment) [19]NARX (nonlinear autoregressivenetwork exogenous) [66]

Are more accurate but they have highconsumption time and high expensiveIn addition they are not scalable andimpractical for testing with networkequipment

Figure 1 Methods for Quality of Experience assessment

source image It is the most known FR (Full Reference)metric If we consider frames with a size of119872times119873 pixels and8 bitssample the PSNR can be calculated using (1) accordingto [34 35] as followsPSNR = 20

sdot log10(

255

radic(1 (119872 times 119873))sum119872minus1

119894=0sum119873minus1

119895=0

1003817100381710038171003817119884119904 (119894 119895) minus 119884119889 (119894 119895)1003817100381710038171003817

2

)

(1)

where 119884119904(119894 119895) denotes a pixel in the (119894 119895) position of the

original frame and 119884119889(119894 119895) refers to the pixel located at (119894 119895)

position of the frame reconstructed in the receiver sideAround 255 elements are the maximum value that the pixelcan take (255 for 8-bit images)The denominator is known asMSE or Mean Square Error which is the mean square of thedifferences among the grey level values of the pixels into thepictures or sequences 119884

119904and 119884

119889

Since several studies have used this mapping PSNR islimited by the image content and it is not able to identifyartifacts due to packet loss In addition the measure doesnot always correlate with the real userrsquos perception dueto the fact that a pixel-by-pixel comparison is carried outwithout performing an analysis of the image structuralelements (eg contours or specific distortions introducedby either encoders or transmitting devices on the networkor spatial and temporal artifacts) Therefore some metricsare proposed to generate the extraction and analysis of thefeatures and artifacts into video sequences such as SSIMmetric [36]

222 SSIM Metric Assuming that the human visual per-ception is highly adapted for extracting structures froma scene SSIM (Structural Similarity Index) calculates themean variance and covariance between the transmittedand received frames [37] To apply SSIM three components(luminance contrast and structural similarities) are mea-sured and combined into one value called SSIM index rangingbetween minus1 and 1 where a negative or 0 value indicates zero

correlation with the original image and 1 means that it is thesame image [35]

According to Wang et al [38] SSIM gives a goodapproximation of the image distortion due to changes inthe measurement of structural information On the videosequences this metric considers a wide range of scenescomplexity in terms of movement spatial details and colorAccording to Wang et al [37] this metric uses a structuraldistortion measure instead of the error The above is focusedon the human vision system to extract structural informationfrom the visual field and ignored the extraction of errorsTherefore a calculation of the structural distortion shouldgive a better correlation with subjective metrics In [39 40] asimple and effective algorithm was proposed to calculate theSSIM index

Let 119909 be the original signal 119909 = 119909119894| 119894 = 1 2 119873 and

let 119910 be the distorted signal 119910 = 119910119894| 119894 = 1 2 119873 the

similarity structure index is given by

SSIM =

(2119909119910 + 1198621) (2120590119909119910+ 1198622)

[(119909)2+ (119910)2

+ 1198621] (1205902119909+ 1205902119910+ 1198622)

(2)

where is themean of 119909 is themean of 119910 and are the variancesof 119909 and 119910 is the covariance of 119909 and 119910 and 1198621 and 1198622 areconstant values The SSIM value can be defined as

SSIM (119909 119910) = [119897 (119909 119910)]120572

[119888 (119909 119910)]120573

[119904 (119909 119910)]120574

(3)

where 119897(119909 119910) 119888(119909 119910) and 119904(119909 119910) are comparison functions ofthe luminance contrast and structure components and theparameters 120572 120573 and 120574 gt 0 are constants SSIM(119909 119910) is thequality assessment index For more details see [39]

223 VQM Metric The NTIA VQM metric (Video QualityMetric) [39] considers two image inputs the original and pro-cessed video in which the quality levels are verified throughthe human vision system and some subjective aspects Thismetric divides the image into sequences of space-temporal


Table 1 Comparative table of the main QoE metrics

Metric Standard Class Estimate Type

PSNR Objective Pixel-by-pixel comparison between reference image and compressedimage FR

VQM(Video Quality Metric) NTIA Objective

Sequences are divided into temporal-space blocks Calculate theorientation of each block using spatial luminance gradient Scoretends to 0 (best)Assess blurring global noise block distortion and color distortion

RR

SSIM(Structural SimilarityIndex)

ObjectiveFind the mean variance and covariance within a frame and combinethem in a distortion map Use luminance contrast and structuresimilarityUse decimal value from 0 to 1 (high correlation with original picture)

FR

MOS ITU-T P800 Subjective Subjective scale from 1 (poor) to 5 (good) NA

blocksmeasuring elements like blurring general noise blockdistortion color distortion and mix among them into asingle metric The score closer to 0 is considered the bestpossible value According to Wang [41] this metric shows agood correlation with subjective methods and has also beenadopted by ANSI as a standard for the assessment of videoquality

In Table 1 the comparative table of the main QoEmetricsis summed up which is used in this work

224 New Mapping QoE Metrics Zinner et al [42] pro-posed a framework for the assessment of the QoE by usingstreaming video systems On the other hand Botia et al [16]proposed a new mapping among the PSNR SSIM VQMand MOS metrics shown in Table 2 In our simulations wecalculated the average of each MOS for all video sequenceswith the value of each FR metric from Table 2 and then weobtained the expected MOS

225 QIBF Metric Serral-Garcia et al [43] propose aframework called PBQAF (Profile Based QoE AssessmentFramework) This framework defines three states for frames(correct altered and lost) through the analysis of the payloadMoreover a mapping is performed to generate and associatethe quality index This metric is calculated from payloadof the received packets associated with a PLR (Packet LossRate) in particular Equation (4) shows the quality functionto generate the mapping function119872

119876 (119891) = 119872(PLR (119891)) (4)

with PLR(119891) being the packet loss rate of the frame 119891 Themapping function is given by

119872(119909) = 1 minus 119909 (5)

where 119909 shows the ratio of lost frames therefore when a highpacket loss rate is measured the quality index will be lowerand tends to be 0 Thus the final quality of the video streaminto a set of quality values 119902(119891119909) for particular frames is givenby

119876119891= 119902 (119891

1) 119902 (119891

119899) (6)

where 1198911119899

is the set of all frames in the video stream FromBotia et al [44] a mapping between QIBF (Quality Index

Table 2 Mapping among QoE metrics (SSIM PSNR VQM andMOS) [16]

MOS PSNR (dB) SSIM VQM5 (excellent) ge37 ge093 lt114 (good) ge31ndashlt37 ge085ndashlt093 ge11ndashlt393 (fair) ge25ndashlt31 ge075ndashlt085 ge39ndashlt652 (poor) ge20ndashlt25 ge055ndashlt075 ge65ndashlt91 (bad) lt20 lt055 ge9

Table 3 QIBF versus MOS mapping [16]

MOS QIBF value5 (excellent) ge0854 (good) ge065ndashlt 0853 (fair) ge045ndashlt 0652 (poor) ge025ndashlt 0451 (bad) lt025

Based Frame) metric and MOS metric is proposed for eachframe class 119868119875119861 that is each frame is equivalent to oneclass In (7) the QIBF is given by

QIBF (seq nt) =119865

sum

119891=1

1 minus NFL (119891)3

minus 120572 (7)

where 119865 is the maximum number of frames seq is thesequence of actual video to be evaluated nt refers to the classnetwork applied over the transmitted sequence (BestEffort orDiffserv) NFL(119891) is the number of frames (119868119875119861) lost deter-mined by the set of MPEG images and 120572 is an adjustmentfactor set to 005 obtained from several developed tests

The factor 13 in (7) is defined through 3 classes offrames used in the tests As NFL(119891) isin [0 100] the valueof NFL is divided by 100 in order to define NFL(119891) isin

[0 1] and therefore to calculate QIBF(seq nt) isin [0 1] Thedemonstration is shown in Appendix A

Table 3 shows the proposed mapping between qualityindex and MOS metric where QIBF(119891) isin [0 1] and asobserved in Table 3 for an index value QIBF(119891) gt 065 agood value for theMOSmetric is obtained indicating a videosequence with a minimum of artifacts


(a) News TVE (b) Winter tree

(c) Mass (d) Highway

Figure 2 Screenshots from video sequences assessed

3 Simulation Testbed

For our test a testbed is built up to use a simulation softwaretool NS-2 and the framework Evalvid for assessing a set ofvideo sequences The results obtained from FRRR metricsare evaluated to find the possible correlation with subjectivemetrics using machine learning techniques

In the simulation we used a selection of different videoraw uncompressed sequences in format YUV with colormode or sampling 4 2 0 encoded with the ffmpeg andmainconcept software tools to adapt them to different bitrates andGOP (Group of Pictures) lengths Four video sequences wereinitially assessed with different levels of movement encodedin theMPEG-4 format which were adapted to be transmittedby a simulated IP network

Figure 2 illustrates some screenshots of the evaluatedvideos (News of the Spanish Public TV Mass Highwayand Winter Tree) converted and encoded at a resolutionof 720 times 480 pixels (standard definition) under the NTSCstandard with frame rate of 30 fps For each video streamseveral parameters were combined as length of the GOP(10 15 and 30) the bit rates recommended by DSL Forum[45] (15 2 25 and 3Mbps) and packet loss rate forboth networks (BestEffort and Diffserv using the congestioncontrol algorithm WRED) which produced 385 differentvideo sequences for testing [16]

The generated video traces were adapted to be sent to thedata network through the encapsulation of each packet withan MTU (Maximum Transfer Unit) of 1024 bits using theRTP protocol (Real Time Transport Protocol) withMP4tracesoftware tool Considering the simulation tool NS-2 andEvalvid framework [46] the sender and receiver trace filesthat were created to calculate the sent and received lostframes and packets delays and jitters The above facilitatesthe analysis of each video sequence for both implementedscenarios (BestEffort and Diffserv data networks)

The Evalvid framework also supports PSNR and MOSmetrics and has a modular structure making it easily adapt-able to any simulation environment MSU VQMT softwaretool [47] allows getting the Y-PSNR SSIM and VQMmetricsvalues through the original reference video and receivedvideo with distortion

The simulation scenario is composed of a video sender(server for video on demand) and 9 cross-traffic sourceswhich consist ofCBR andOn-Off traffic sourcesThenetworkis based on dumbell topology [48] (see Figure 3) In ourtest we send several video packets over a network withcongestion and will be allowed to test the defined QoSscheme (Diffserv with WRED) The MPEG-4 video flow iscomplete with backgroundOn-Off traffic flows which has anexponential distribution with an average packet size of 1500bytes burst time of 50ms idle time of 001ms and sendingrate of 1Mbps [16 44] The access network is represented bya video receiver (simulating a last mile with ADSL2) witha bandwidth link of 10Mbps and several receiving nodes(sink) for cross traffic with a bandwidth of 10Mbps for eachone Transmission distortionswere simulated at different PLR(Packet Loss Rate)The traffic behavior andQoEmetrics weretested with several error rates over a link established betweenthe core and edge routers using a loss model with uniformdistribution at rates of 0 1 5 and 10 and delay of5ms Tables 4 and 5 show the main parameters used in thesimulation and encoding

4 Implementation of Nonintrusive Methodsto Estimate Video Quality by Objective andSubjective Metrics

For the evaluation of nonintrusive methods we decided touse two machine learning methods (BPNN feedforward andRandom Neural Network) These methods have been usedin different environments described in the next section To


Network

Video trace file

Video sender

Routeredge 1

Routercore 1

Routeraccess

Evaluate trace

Video receiver 1

Receiver n

Cross trafficsource (CBR y exponential)

Receiver 2

Receiver 3

Sender 2

Sender 3

Sender n

Bottleneck link

Dumbbell topology

PLR = 0 1 5 10

300Mbps001ms

10Mbps011ms

10Mbps001ms

10Mbps001ms

100Mbps001ms

20Mbps001ms

10Mbps001ms

10Mbps001ms

10Mbps001ms

10Mbps5ms

Figure 3 Simulation scenery with NS-2 and Evalvid

Table 4 Encoder parameters

GOP length 10 15 and 30 framesFrame rate 30 fpsBitrate 15 2 25 and 3MpbsGOP sequence IPBBPBBPBBP Resolution NTSC 720 times 480 pVideo color mode YUV (4 2 0)

Table 5 Simulation parameters

Bandwidth link 10100MbpsLink delay 001msndash5msBuffer size 50

Tail behavior DropTailWRED random earlydetection

Packet size 1052 bytes (8 bytes UDP and 20bytes IP)

Max fragment size 1024 bytesPLR loss model with uniformdistribution

0 1 5 and 10 and delay of5ms

assess the output of each system we considered the followingestimates MOS versus expected MOS where the latter iscomputed through the mapping of PSNR VQM SSIM andQIBF metrics (see Tables 1 and 2)

In each case the correlation adjustment was calculateddetermined by the Pearson correlation coefficient and RMSEto estimate the error The general methodology is presentedin Figure 4 [26 49]

Initially the video sequences in RAW-format wereobtained and they were codified in MPEG-4AVC-formatEach sequence was sent through an IP-simulated networkThe above is based on ldquoBestEffortrdquo and ldquoDiffservrdquo where eachFRRR metric is calculated and their respective values aremapped In this manner average MOS value is obtained byusing Tables 2 and 3 Considering the codification values thekind of QoS network and the obtained FRRR metrics allowbuilding the input database for training two neural networks

MATLAB software suite was used for BPNN using theNeural Networks Toolbox For RNNs case we used the QoE-NNR software tool [50] From 385 different sequences atdifferent configured parameters around 70 were used astraining data and the remaining 30 as test and validationdata Finally obtaining the estimated MOS value the corre-lation is calculated with respect to the average MOS valueThe given results of machines learning will be presented anddiscussed

Table 7 shows a brief state of the art where several papersassessQoEusing neural networks as PNN (ProbabilisticNeu-ral Network) BPNN (Backpropagation Neural Network)RNN (Random Neural Network) and ANFIS (AdaptiveNeurofuzzy Inference System) Works using a few videosequences do not show resolutions or codec information andapply low resolutions (for mobile devices) Codec formatsmore used are H264 and MPEG-4 AVC We found thatPSNR SSIM and VQM metrics are frequently applied andfew works use 2 metrics or more In our case we were using5 different metrics and 4 video sequences with motion levelsin each scene Finally the machine learning based on neuralnetworks (RNN BPNN and ANFIs) is used in these worksThe results show a good performance of theMOS estimation


Video raw sequences

Encoder

BestEffort Diffservnetwork

Decoder

Display

BitrateGOP lengthQoS classPLRPSNRSSIMVQMQIBF

Calculate FRRR metrics

Mapping MOS subjectivescore

Original data set

Datatraining

Datatest

(1) BPNNfeedforward

(2) Randomneuralnetwork

EstimatedMOS

Correlation

Figure 4 Proposed methodology to estimate the MOS metric using machine learning techniques

However they are defined by few input parameters onlyassessing 1 or 2 video sequences in low resolution(s)They donot compare with other kinds of ANNs and in most casesthe Pearson correlation coefficient was lt090

41 Case 1 Implementation of a Feedforward Artificial NeuralNetwork with Backpropagation The ANNs are a paradigmfor processing information inspired by the human neuralsystem Usually ANNs are composed of a large number ofhighly interconnected processing elements called neuronswhich work together to solve problems [13] The base is thecreation of a neural network also called MLP (MultilayerPerceptron) usually divided into 3 or119873-layers

The first layer contains neurons connected to the inputvector data the second layer is called the hidden layerand incudes a set of synapses and a number of weights119882119894119895and some activating functions defined for exciting or

inhibiting each neuron generating a response According toRubino et al [51] if the number of hidden neurons is lowit can have large training and generalization errors due tothe underfitting Otherwise if there are many neurons inthe hidden layer low training errors may occur but highgeneralization errorsmay appear causing the undesired effectof overtraining (overfitting) and high variance The thirdlayer is the output layer directly connected to the hidden layerwhere data vectors of each estimated output will be obtainedMultiple layers of neurons with nonlinear transfer functions(such as tangential-sigmoid) allow the ANNs learning thelinear and nonlinear relationships between the inputs (PLRGOP bitrate QoS class QIBF PSNR SSIM and VQM)and the desired output vectors (MOS) The structure ofBackpropagation Neural Network is shown in Figure 5

ANNs type feedforward are the most commonly usedones to perform estimation of subjective metrics Several

works propose different models based on ANN [52ndash58]These approaches are generally applied on mobile systemsor with low resolutions (QCIF or CIF) furthermore theseworks obviate the network parameters or video objectivemetrics In most cases one or twometrics as input to the net-work are used and the results fromPearsonrsquos autocorrelationsalmost always are below 090

According to Ding et al [52] the neural network can beused to obtain mapping functions between objective qualityassessment and subjective quality assessment indexes Thisaffirmation allows the understanding of the usefulness of theANN to analyse the estimated MOS and the proposed modelpresented in this research

In that case the network training was carried out throughseveral parameters which will turn into input variablesAs stated the objective parameters may affect the videoquality After training an evaluation with a set of test data(96 sequences) will be performed in order to reach thecorresponding network validation The idea is to reach thelowest error and to be able to correlate the estimatedMOS byANN versus the average MOS computed from the objectivemetrics defined by Tables 1 and 2

For the case of study we want to build the estimatedMOSfunction defined by

MOS Est = 119891 (1199091 1199092 119909

119899) (8)

where 1199091 1199092 119909

119899 are the input parameters established

by bitrate packet loss rate GOP length QoS class (1 forBestEffort and 2 for Diffserv) SSIM PSNR VQM and QIBF

Like a human being the ANN system needs a learningphase and another one for validation and testing to establishwhen the neural network generalized its learning for any dataset For this process the network is trained through a training


LW1 1

LW1 2

LW1 8

MOS Est

BI1

BI2

BI8

IW1 1

IW1 2

IW1 3

IW2 1IW2 2

IW2 3

IW7 8

IW7 7

Tangsig

Tangsig

Tangsig

Linear

IW initialization of random weights LW load weightsBI bias

Input layer Hidden layer Output layer

Long GOP

PLR

BR

QoSclass

PSNR

SSIM

VQM

QIBF

sumsum

sum

sum

[1ndash5]

120596t+1i = 120596t

i + 120578(yi_yi)

yi target expected (output)yi output network120578 learning rate

Figure 5 Proposed architecture for estimating MOS through BPNN feedforward

algorithm and the lowest possible error is computed throughthe cost function MSE (Mean Square Error) In that casewe used an iterative gradient descent algorithm or otherlearning algorithms to achieve convergence to a target value(target) that is to calculate the minimum training error ofthe network

According to Ries et al [59] in the multilayer networkMLP with a wide variety of nonlinear continuous activationfunctions on the hidden layers one of these layers whichcontain a largely arbitrary number of neurons is sufficient tosatisfy the property called universal approach This alloweddefining the neural network with a single hidden layer (with20 neurons) satisfying the outputs (estimated MOS anddesired MOS) calculated from the mapping of objectivemetrics One of the major drawbacks is to find the rightnumber of hidden neurons due to factors such as thequantity of inputoutput neurons the number of trainingcases the amount of noise in the output and input theused architecture the learning algorithms and the kind ofactivation functions in the neurons on the hidden layer

An empirical methodology was performed starting with16 neurons in the hidden layer equal to twice of input neuronsand a new neuron to reach the objective function with the

training algorithm Levenberg-Marquardt was added in eachtraining and testing cycle This is a widely used algorithm tosolve the problem of least squares

The proposed BPNN feedforward architecture is shownin Figure 6 In the input and output layer the neurons have alinear activation function (purelin) In the hidden layer aftervarious tests the lowest training error was obtained with 20neurons with function tangent-sigmoid activation (tansig)

After performing several iterations and resetting theweights in the training stage the best performance is obtainedas shown in Figure 7

In Figure 8 the validation stage is illustrated As showna good fit between the output data of the network and thedesired MOS is obtained An analysis of the linear regressionbetween them is performed The relationship is establishedbetween the estimated value by the BPNN (119910 variable) anddesired data (119909 variable) The representation is given by thefollowing classical linear equation

119910 = 119898119909 + 119887 (9)

According to Pearson correlation parameter between theestimated MOS by BPNN and the expected MOS a linearfit of 9672 and a RMSE of 01977 were obtained Figure 9


W

b

+

W

b

+

20 1

18

Input

Hidden layer Output layer

Output

Figure 6 Neural Network feedforward 3-layer architecture

Mea

n Sq

uare

Err

or (M

SE)

100

10minus5

10minus10

7 epochs0 21 43 65 7

TrainValidation

TestBest

Best validation performance is 0030507 at epoch 3

Figure 7 MSE obtained in the training stage

0 10 20 30 40 50 60 70 80 90 1001

152

253

354

455

55

Number of sequences

Estim

ated

and

desir

ed M

OS

Output obtained for estimated MOS ANN versus desired MOS

Output estimated MOSOutput target MOS

Figure 8 Results of the output estimatedMOS versus desiredMOS

shows the output of the correlation obtained The resultsestablish that the feedforward neural network allowed a goodgeneralization with data used for validation and full linearrelationship

42 Implementation of a Random Neural Network (RNN)through PSQA Methodology This kind of network captureswith great accuracy and robustness mapping functions

15 2 25 3 35 4 45 51

152

253

354

455

Desired MOSEs

timat

ed M

OS

Estimated MOS ANN versus desired MOS

Estimated MOS ANN versus desired MOSFit 1

Figure 9 Estimated ANNMOS versus expected MOS

where several parameters are involved According to Casaset al [60] such networks have been used on multiple engi-neering fields highlighting and solving NP-complete opti-mization problems generating textures in images problemsvideo and image compression algorithms and classificationproblems for perceived quality of voice and video over IPwhich make them ideal for its application in QoE assessAppendix B explains in detail the RNNs and themain param-eters used in these networks The RNNs are a cross betweenneural networks and queuing networks By definition RNNsare sets of ANNs formed by a range of interconnectedneurons These neurons exchanged instantly signals whichtravel from one neuron to another and send signals from andto the environment Each neuron is associatedwith an integerrandom variable associated with a potential The potential ofa neuron 119894 at time 119905 is defined by 119902

119894(119905) If the potential of the

neuron 119894 is positive the neuron is excited and randomly sendssignals to other neurons or to the environment according toPoisson process of rate 119903

119894 The signals can be positive (+) or

negative (minus) The RNN has a three-tier architecture Thusthe set of neurons 119877 isin 1 119873 is split into 3 subsets theinput neuron set the set of hidden neurons and the outputneurons An output

(119896)

is generated when the input is (119896)

therefore

(119896)

= MOSs for 119896 = 1 119870 the set of weightsfor step 119896

According to Casas et al [60] it is necessary to apply amethodology to assess theQuality of Experience based on theuse of network parameters (probability of packet loss delay


Table 6 General overview of the correlations obtained for the study cases

Correlation PCC SROCC RMSE ORY-PSNR versus MOS BPNN 09303 09713 02882 05833VQM versus MOS BPNN 09412 09707 02647 01979SSIM versus MOS BPNN 09446 09733 0257 01483QIBF versus MOS BPNN 0937 09711 02739 01562Y-PSNR versus MOSpsqa 09698 09873 01796 05833VQM versus MOSpsqa 09645 09900 01948 01666SSIM versus MOSpsqa 09497 09889 02319 01354QIBF versus MOSpsqa 09254 09853 02824 01354Note PCC (Pearson correlation coefficient) for prediction accuracySROCC (Spearman rank order correlation coefficient) for monotonicityRMSE (Root Mean Square Error) for correlation quality assessmentOR (Outlier Ratio) for consistence

jitter etc) and video parameters (encoding bitrate framerate GOP length etc) which will become input parametersBased on these criteria a mapping function between theseparameters and subjective quality value defined by the MOSmetric can be generated To perform this task the authorproposes the methodology PSQA (Pseudo Subjective QualityAssessment) which uses the RNNs to learn the mappingbetween the parameters and perceived quality [51]

This methodology is characterized by its accuracy itallows generating automatic evaluations in real time whichis efficient and can be applied with many kinds of mediacodec and under different parameters and network con-ditions Also it can be extended for comparison withobjective metrics generating more accurate correlations[61]

The RNN is considered as a supervised learningmachinewhich uses multimedia and network characteristics and theexpected MOS values If in the training stage a relationshipof the input parameters through the objective metrics andthe expected output is found it is possible to estimate andorpredict the subjective values with a higher level of accuracyDue to the RNNs features they are perfect to generate goodassessments for a wide variation of all parameters that affectthe quality [51] Therefore it is an accurate model fastand with low computational cost To develop the proposedstudy case RNN feedforward 3-layer architecture proposedby Mohamed and Rubino [61] is implemented QoE-RNNsoftware tool was used to estimate theMOS value through theuse of a RNNThis tool has LGPL license and was developedin the programming language-C [50]

The whole process is presented in Figure 10 where videosequences transmitted over the network are assessed bycomparing the target average MOS Thus depending on theQoS strategy implemented the MOS values associated toeach objectivemetrics (PSNRVQM SSIM andQIBF) are setand are defined by MOSobj and so the average MOS (MOS

119901)

that is the target value of the RNN is calculated MOS119901is

expressed as shown in the following equation [26]

MOS119901=

119873

sum

119894=1

MOSobj119873

(10)

whereMOSobj refers to theMOS for each objectivemetric and119873 is the total of samples

The input parameters are chosen and with the MOSpsqaobtained from the network the corresponding correlationanalysis is performed

The number of boundary iterations is 2000 and thenetwork topology is defined by 9 input neurons 10 neuronsin the hidden layer and one output neuron In differentconducted tests the best fit is achieved with 10 neurons in thehidden layer

Using MATLAB an analysis of the linear correlationbetween the expected MOS and MOSpsqa is performed Agood linear fit is found by the Pearson coefficient 1198772 at 09812and RMSE at 01412 In Figure 11 this correlation is shown

The general summary of all correlations obtained fromeach study case is presented in Table 6

The performance of perceptual quality metrics dependson its correlation with the results of objective metrics Thusthe accuracy in the estimation of subjective metrics withrespect to issues such as prediction accuracy monotonicityand consistency is evaluatedThis guarantees a high reliabilityin the subjective assessment of video quality over a range ofvideo test sequences with different artifacts The assessmentmethods are proposed by the Video Quality Experts Group(VQEG) [62] Four statistical measurements are applied toevaluate the video quality metrics performance Pearsoncorrelation coefficient (PCC) Spearmanrsquos rank order corre-lation coefficient (SROCC) Mean Square Error (RMSE) andOutlier Ratio (OR) [63 64]

As shown in the results of Table 6 the correlationsbetween MOSpsqa metrics were certainly good with objectivemetrics with high Spearman coefficient proving high lineartrend in all cases The generalization was very high and wenoted the consistency accuracy and monotonicity results Inthe simulations performedwe observed that theVQM SSIMand QIBF metrics are highly correlated and are close to theusersrsquo perception (established by the MOS metric)

For all cases the correlation reached values higher than90 and the correlation between the objective and subjectivemetrics in each case presents an excellent linear behaviorAccordingly the metrics as VQM SSIM and QIBF arelargely related to the subjectivity established by MOS Also


Table7Paperswith

machine

learning

metho

dsfora

ssessin

gQoE

Paper

video

sequ

ences

Resolutio

nCod

ecSSIM

VQM

PSRN

MSE

BRFR

PLR119876value(decodificables

fram

esrate)

QP

Playou

tinterrup

tions

Delay

JitterBW

MLB

SGOP

Machine

learning

Assess

MOSDMOS

Correlatio

nperfo

rmance

PCC

SROCC

RMSE

OR

Hee

tal

[17]2012

1QCI

FMPE

G4

XX

XX

X

Prob

abilistic

Neural

Network

(PNN)

Yes

09286

No

No

No

Kiplietal

[18]2012

ND

Nodata

Nodata

XX

XBP

NN

Yes

0891

No

No

No

Sing

het

al[19]

2012

4HD-720p

H264

XX

XX

RNN

Yes

No

No

037

No

Liae

tal

[20]2011

4Nodata

Nodata

BPNN

No

No

No

No

No

Khanatal

[21]2010

6QCI

FH264

XX

XANFIS

Yes

08717

No

02812

No

Duetal

[22]200

91

SDNodata

XX

XX

XX

BPNN

Yes

No

No

No

No

Piam

ratet

al[23]

2009

1Nodata

H264

XX

XPS

QAwith

RNN

Yes

No

No

No

No

Khanetal

[24]2008

3CI

FMPE

G4

XX

XX

XANFIS

Yes

R=08056

forM

OS

predicted

versus

MOS

Measure

andR=

09229

for

119876

predicted

versus119876

measure

No

RMSE

=01846

for

MOS

predicted

versus

MOS

measure

andRM

SE=006234

for119876

predicted

versus119876

measure

No

Rubino

etal[25]

2004

Only

VoIP

Nodata

Nodata

XX

RNN

No

No

No

No

No

BRbitrateFR

framerateP

LRP

acketL

ossR

ateQP

QuantizationParameterB

Wb

andw

idthM

LBSMeanLo

ssBu

rstS

izeGOP

Group

ofPicturesP

CCP

earson

correlationcoeffi

cientSR

OCC

Spearman

rank

orderc

orrelatio

ncoeffi

cientRM

SER

ootM

eanSquare

ErrorandOR

OutlierR

atio


Video test sequencewithout distortion

NetworkBestEffort

Diffserv Encode Decoded

Video test sequencewith impairments

MOS_PSNRMOS_VQMMOS_SSIMMOS_QIBF

Input parameters

RNN training script

RNN test script

QoE-RNN software tool

Objective-subjective metricsCorrelation analysis

MOSobj

MOSpsqaMOSp =

N

sumi=1

MOSobj

N

Figure 10 Employed methodology for the implementation of PSQA with the RNN

15 2 25 3 35 4 45 51

152

253

354

455

Desired MOS

Estim

ated

MO

S

Fit 1

MOSpsqa versus desired MOS using RNN

MOSpsqa versus MOSp

Figure 11 Correlation between MOSpsqa and MOS expectedthrough the RNN

it was observed that the results of the BPNN feedforwardindicate a good generalization for estimated MOS againstevery assessed objective metrics although it was slightlyhigher for applying the RNN The RNN with PSQA providesa better correlation with the predicted values for MOSThe strategy generated by the nonintrusive model based onmachine learning methods was shown to be accurate andhighly flexible It allows the estimation of subjective MOSvalues by relating them with FR (Full Reference) and RR(Reduced Reference) chosen for the research

5 Conclusion

In this work we obtained excellent correlation valuesbetween the objective and subjective QoE metrics throughthe use of nonintrusive methods The accuracy consistency

and monotonicity were validated with the analysis of Pear-sonrsquos and Spearmanrsquos correlations outliers rate and RMSE(Roots Mean Square Error) One of the main limitations ofthe objective and subjective methods is the lack of completemethodologies to analyze the accuracy of QoE Thereforethis problem was addressed in order to propose a newmethodology that allows finding new correlations betweenobjective and subjective metrics To improve the analysis themachine learning techniques were proposed through back-propagation artificial neural networks and Random NeuralNetworks to enhance the approach of the estimation ofhuman perception In different performed simulations weobserved that VQM SSIM and QIBF metrics are highlycorrelated and are close to the user perception (determinedby the MOS metric) Unlike previous works we developeda general correlation model which uses network and codingparameters applied to several video sequences Analyzingthe results from learning machines the BPNNs and RNNsgenerated high correlations with objective metrics obtainingPCC values higher than 90 and low error rates Theconsistency of correlations between the metrics throughoutliers was calculated with low values We conclude that theapplication of nonintrusive methods allows us to generatemore accurate approaches to human perception In additiontelecommunications providers can use this methodology forestimating the QoE of users and improve their data networkarchitectures andor global settings on their platforms opti-mize the QoS and employ better encoding mechanisms Thedevelopment of new models for the assessment of QoE is atop research topic according to the state of the art The topicsthat are being currently working include

(i) development of new objective metrics which are easyto apply and can be mapped accurately to the trueperception of the viewer


(ii) the close relationship of all QoS factors which affectthe QoE

(iii) new transmission strategies over highly congesteddata networks that can have a greater impact ontransmission environments for streaming video overthe internet which affect the user experience on themultimedia content over the next generation mobiledevices

(iv) the application of new kinds of video codecs specif-ically aimed at HD (H265MPEG-DASH DynamicAdaptive Streaming over HTTP) [65]

(v) the application of different methodologies based onmachine learning techniques especially those relatedto artificial neural networks neurofuzzy networkssupport vector machines and genetic algorithms

Appendices

A Proof of QIFB Metric

Let QIBF(seq nt) be a QIBF metric [0 1] (7) fulfills thefollowing axioms

(P1) [Maximum] If QIBF(seq nt) = 1 for all 119891 = 1 2 3with 1 being equivalent to 119868 2 equivalent to 119875 and 3equivalent to119861 theMOS index is excellentMOS = 5

(P2) [Minimum] If QIBF(seq nt) = 0 for all 119891 = 1 2 3with 1 being equivalent to 119868 2 equivalent to 119875 and 3equivalent to 119861 the MOS index is bad MOS = 1

(P3) [Resolution] QIBF1(seq nt) lt QIBF

2(seq nt) for all

119891 = 1 2 3 with 1 being equivalent to 119868 2 equivalentto 119875 and 3 equivalent to 119861 if NFL

1(1) gt NFL

2(1)

NFL1(2) gt NFL

2(2) and NFL

1(3) gt NFL

2(3)

(P4) [Symmetry] QIBF1(seq nt) = QIBF

2(seq nt) for all


1(1) = NFL

2(1) =

NFL1(2) = NFL

2(2) = NFL

1(3) = NFL

2(3)

Proof (P1) Considering NFL(1) = NFL(2) = NFL(3) =

0 for all 119891 = 1 2 3 and supposing that 120572 = 0 thenQIBF(seq nt) = 1 If 120572 = 005 as real adjustment thenQIBF(seq nt) = 095 but this value can be approached at1 in order to get a better evaluation of MOS Therefore asQIBF(seq nt) = 1 the MOS score is around 5 as maximumvalue

(P2) If NFL(1) = NFL(2) = NFL(3) = 1 (maximumlost) for all 119891 = 1 2 3 and supposing that 120572 = 0 thenQIBF(seq nt) = 0 If 120572 = 005 as real adjustment thenQIBF(seq nt) = 005 but this value can be approached at0 in order to get a better evaluation of MOS Therefore asQIBF(seq nt) = 0 the MOS score is around 1 as minimumvalue

(P3) Assuming NFL1(1) gt NFL

1(2) gt NFL

1(3) and

NFL2(1) gt NFL

2(2) gt NFL

2(3) these relations are rewritten

asNFL1(1) gt NFL

2(1) gt NFL

1(2) gt NFL

2(2)

gt NFL1(3) gt NFL

2(3)

(A1)

Taking into account QIBF1(seq nt) and QIBF

2(seq nt)

QIBF1(seq nt) =

119865

sum

119891=1

1 minusNFL1(119891)

3minus 120572

QIBF2(seq nt) =

119865

sum

119891=1

1 minusNFL2(119891)

3minus 120572

(A2)

Then

QIBF2(seq nt) minusQIBF

1(seq nt)

= [

[

119865

sum

119891=1

1 minusNFL2(119891)

3minus 120572]

]

minus [

[

119865

sum

119891=1

1 minusNFL1(119891)

3minus 120572]

]


1(seq nt)

=

119865

sum

119891=1

1 minusNFL2(119891)

3minus

119865

sum

119891=1

1 minus NFL1(119891)

3

(A3)

If NFL1(1) gt NFL

2(1) gt NFL

1(2) gt NFL

2(2) gt

NFL1(3) gt NFL

2(3) it is obtained that

QIBF2(seq nt) gt QIBF

1(seq nt) larrrarr

119865

sum

119891=1


3gt

119865

sum

119891=1

1 minusNFL1(119891)

3

(A4)

Therefore it is found that QIBF1(seq nt) lt

QIBF2(seq nt) in which if NFL

21 2 3 is monotonically

decreased the loss is also decreased and NFL11 2 3 is

monotonically decreased if the loss is also increased ThusQIBF1(seq nt) lt QIBF

2(seq nt) is a sufficient condition

(P4) If NFL1(1) = NFL

2(1) = NFL

1(2) = NFL

2(2) =

NFL1(3) = NFL

2(3) it is obvious that QIBF

1(seq nt) =

QIBF2(seq nt) where the quantity of loss is the same

B Random Neural Network

The RNNs are a cross between neural networks and queuingnetworks By definition RNNs are sets of ANNs comprisinga series of interconnected neurons These neurons exchangesignals which instantly travel from one neuron to anotherand send signals from and to the environment Each neuronis associated with an integer random variable and these areassociated with a potentialThe potential of a neuron 119894 at time119905 is defined by 119902

119894(119905) If the potential of the neuron 119894 is positive

the neuron is excited and randomly it sends signals to otherneurons or to the environment according to Poissonrsquos processwith rate 119903

119894 The signals may be positive (+) or negative (minus)

Thus the probability that the signal sent from neuron 119894 toneuron 119895 is positive is denoted by 119875+

119894119895 and the probability that

the signal is negative is denoted by


The probability that the signal goes to the environmentis denoted by 119889

119894 If 119873 is the number of neurons for all 119894 =

1 119873 then 119889119894is expressed as shown in

119889119894+

119873

sum

119895=1

(119901+

119894119895+ 119901plusmn

119894119895) = 1 (B1)

Therefore when a neuron receives a positive signal fromanother neuron or from the environment its potential isincreased by 1 If a negative potential signal is received itdecreases by oneWhen a neuron sends a positive or negativesignal its potential decreases in a unit

The flow of positive signals arriving from the environ-ment to the neuron 119894 is Poissonrsquos process with a 120582+

119894or 120582minus119894rate

It is thus possible to have 120582+119894= 0 and 120582minus

119894= 0 for any neuron 119894

To have an active network (B2) is needed

119873

sum

119894=1

(120582+

119894) gt 0 (B2)

If 119892119894is defined as the equilibrium probability for neuron 119894

in excitation state then (B3) is considered

119892119894= lim119905rarrinfin

119875 (119902119894(119905) gt 0) (B3)

In (B3) if Poissonrsquos process for the potential of theneurons

997888997888rarr119902(119905) = 119902

1(119905) 119902

119873(119905) is ergodic the network is

defined as a stable and satisfies the conditions for a nonlinearsystem

In RNNs the purpose of the learning process is to obtainthe values of 119877

119894and probabilities 119875+

119894119895and 119875

minus

119894119895 The above

allows obtaining the weights of the connections between theneurons 119894 119895 as shown in

120596+

119894119895= 119877119894119875+

119894119895

120596minus

119894119895= 119877119894119875minus

119894119895

(B4)

From (B4) the set of weights on the network topology isinitialized with arbitrary positive values and119870 iterations thatare performed tomodify the weights For 119896 = 1 119870 the setof weights for the step 119896 is calculated from the set of weightsat step 119896minus1 Let119877(119896minus1) be the network obtained after step 119896minus1defined by weights 120596+(119896minus1)

119894119895and 120596minus(119896minus1)

119894119895 then the set of inputs

rates (positive external signals) on 119877(119896minus1) for 119883(119896)119894

will get anetwork that allows generating an output

(119896)

when the inputis (119896)

therefore (119896)

= MOSsThe RNN has a three-tier architecture Thus the set of

neurons 119877 isin 1 119873 is split into three subsets the set ofinput neuron the set of hidden neurons and the set of outputneurons The input neurons receive positive signals from theoutside For each node 119894 119889

119894= 0 For the output nodes 120582+

119894= 0

119889119894gt 0 The intermediate nodes are not directly connected to

the environment for any hidden neuron 119894 120582+119894= 120582minus

119894= 119889119894= 0

There are several video sequences with different parame-ters for the case study where a set of training data and anotherset of test data are selected In (B5) 119878 denotes the set of

training sequences where each sequence is defined by 120572119899and

1198781015840 refers to the set of validation sequences Moreover 119875 is theset of parameters 120582 that affects each sequence where

119878 = 1205721 1205722 120572

119878

1198781015840= 1205721015840

1 1205721015840

2 120572

1015840

119878

119875 = 1205821 1205822 120582

119899

(B5)

The value of the parameter 120582119901in the sequence 120572

119878is

defined by 119881119901119904

where 119881 = (119881119901119904) where 119904 is a matrix

119904 = 1 2 119878 Each sequence 120572119878receives a score MOSs isin

[1 5] Moreover for the sequences 1205721015840119878isin 1198781015840 a function

119891(1198811119878 1198812119878 119881

119901119904) asymp MOSs is obtained and the training

process is completed Otherwise we can try with more dataor change some parameters of the RNN and proceed to builda new function 119891

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

Acknowledgment

This research was developed as part of themacroproject ldquoSys-tem of Experimental Interactive Televisionrdquo for the Researchand Innovation Center (Regional Alliance of Applied ICT-Artica) with code 1115-470-22055 and project no RC584funded by Colciencias and MinTIC

References

[1] L-Y Liu W-A Zhou and J-D Song ldquoThe research of qual-ity of experience evaluation method in pervasive computingenvironmentrdquo in Proceedings of the 1st International Symposiumon Pervasive Computing and Applications pp 178ndash182 IEEEUrumqi China August 2006

[2] S Mohseni ldquoDriving Quality of Experience in mobile contentvalue chainrdquo in Proceedings of the 4th IEEE InternationalConference on Digital Ecosystems and Technologies (DEST rsquo10)pp 320ndash325 Dubai United Arab Emirates April 2010

[3] A Devlic P Kamaraju P Lungaro Z Segall and KTollmar ldquoTowards QoE-aware adaptive video streamingrdquoin Proceedings of the IEEEACM International Symposiumon Quality of Service Portland Ore USA February 2015httppeoplekthsesimdevlicpublicationsIWQoS15pdf

[4] S Winkler ldquoStandardizing quality measurement for videoservicesrdquo IEEE COMSOC MMTC E-Letter vol 4 no 9 2009

[5] S Winkler ldquoVideo quality measurement standardsmdashcurrentstatus and trendsrdquo in Proceedings of the 7th International Con-ference on Information Communications and Signal Processing(ICICS rsquo09) Macau China December 2009

[6] K Seshadrinathan and A C Bovik ldquoMotion tuned spatio-temporal quality assessment of natural videosrdquo IEEE Transac-tions on Image Processing vol 19 no 2 pp 335ndash350 2010

[7] H R Sheikh and A C Bovik ldquoImage information and visualqualityrdquo IEEE Transactions on Image Processing vol 15 no 2pp 430ndash444 2006


[8] A K Moorthy and A C Bovik ldquoA motion compensatedapproach to video quality assessmentrdquo in Proceedings of the 43rdAsilomar Conference on Signals Systems and Computers pp872ndash875 Pacific Grove Calif USA November 2009

[9] K Radhakrishnan and L Hadi ldquoA study on QoS of VoIPnetworks a random neural network (RNN) approachrdquo inProceedings of the Spring SimulationMulticonference (SpringSimrsquo10) Society for Computer Simulation International OrlandoFla USA April 2010

[10] S Winkler and P Mohandas ldquoThe evolution of video qualitymeasurement from psnr to hybrid metricsrdquo IEEE Transactionson Broadcasting vol 54 no 3 pp 660ndash668 2008

[11] F Boavida E Cerqueira R Chodorek et al ldquoBenchmarking thequality of experience of video streaming andmultimedia searchservices the content network of excellencerdquo in Proceedingsof the 23rd National Symposium of Telecommunications andTeleinformatics (KSTiT rsquo08) Institute of Telecommunicationand Electrical Technologies of University of Technology andLife Sciences Bydgoszcz Poland September 2008

[12] P Casas A Sackl R Schatz L Janowski J Turk and R IrmerldquoOn the quest for new KPIs in mobile networks the impactof throughput fluctuations on QoErdquo in Proceedings of the IEEEInternational Conference on Communication Workshop (ICCWrsquo15) pp 1705ndash1710 London UK June 2015

[13] M Ries and J R Kubanek ldquoVideo Quality Estimation formobile streaming applications with neural networksrdquo in Pro-ceedings of the Measurement of Speech and Audio Quality inNetworks Workshop (MESAQIN rsquo06) Prague Czech RepublicJune 2006

[14] O Nemethova M Ries E Siffel and M Rupp ldquoQualityassessment for H264 coded low rate and low resolution videosequencesrdquo in Proceedings of the IASTED International Confer-ence on Communications Internet and Information Technology(CIIT rsquo04) pp 136ndash140 St Thomas Virgin Islands November2004

[15] D Kuipers R Kooij D Vleeshauwer and K BrunnstromldquoTechniques for measuring quality of experiencerdquo inWiredWireless Internet Communications vol 6074 of LectureNotes in Computer Science pp 216ndash227 Springer BerlinGermany 2010

[16] D Botia N Gaviria D Jimenez and J M Menendez ldquoAnapproach to correlation of QoE metrics applied to VoD serviceon IPTV using a diffserv networkrdquo in Proceedings of the 4thIEEE Latin-American Conference on Communications (IEEELATINCOM rsquo12) Cuenca Ecuador November 2012

[17] Y He C Wang H Long and K Zheng ldquoPNN-based QoEmeasuring model for video applications over LTE systemrdquoin Proceedings of the 7th International ICST Conference onCommunications and Networking in China (CHINACOM rsquo12)pp 58ndash62 Kunming China August 2012

[18] K Kipli M Muhammad S Masra N Zamhari K Lias and DAzra ldquoPerformance of Levenberg-Marquardt backpropagationfor full reference hybrid image quality metricsrdquo in Proceedingsof International Conference of Muti-Conference of Engineers andComputer Scientists (IMECS rsquo12) Hong Kong March 2012

[19] K D Singh Y Hadjadj-Aoul and G Rubino ldquoQuality ofexperience estimation for adaptive HttpTcp video streamingusing H264AVCrdquo in Proceedings of the 9th Anual IEEEConsumer Communications and Networking Conf Multimediaand Entertaiment Networking and Services (CCNC rsquo12) pp 127ndash131 Las Vegas Nev USA January 2012

[20] Q Lia J Yangb L He and S Fan ldquoReduced-reference videoquality assessment based on bp neural network model forpacket networksrdquo Energy Procedia vol 13 pp 8056ndash8062 2011Proceedings of the International Conference onEnergy Systemsand Electric Power (ESEP rsquo11)

[21] A Khan L Sun E Ifeachor J-O Fajardo F Liberal and HKoumaras ldquoVideo quality prediction models based on videocontent dynamics for H264 video over UMTS networksrdquoInternational Journal of Digital Multimedia Broadcasting vol2010 Article ID 608138 17 pages 2010

[22] H Du C Guo Y Liu and Y Liu ldquoResearch on relationshipbetween QoE and QoS based on BP neural networkrdquo inProceedings of the IEEE International Conference on NetworkInfrastructure and Digital Content (IC-NIDC rsquo09) pp 312ndash315Beijing China November 2009

[23] K Piamrat C Viho A Ksentini and J-M Bonnin ldquoQualityof experience measurements for video streaming over wirelessnetworksrdquo in Proceedings of the 6th International Conference onInformation Technology New Generations (ITNG rsquo09) pp 1184ndash1189 IEEE Las Vegas Nev USA April 2009

[24] A Khan L Sun and E Ifeachor ldquoAnANFIS-based hybrid videoquality prediction model for video streaming over wirelessnetworksrdquo in Proceedings of the 2nd International Conference onNext Generation Mobile Applications Services and Technologies(NGMAST rsquo08) pp 357ndash362 Cardiff Wales September 2008

[25] G Rubino andM Varela ldquoA new approach for the prediction ofend-tomdashend performance of multimedia streamsrdquo in Proceed-ings of the International Conference on Quantitative Evaluationof Systems (QUEST rsquo04) pp 110ndash119 IEEE CS Press Universityof Twente Enschede The Netherlands September 2004

[26] P Goudarzi ldquoA no-reference low-complexity QoE measure-ment algorithm for H264 video transmission systemsrdquo ScientiaIranica Transactions Computer Science amp Engineering andElectrical Engineering vol 20 no 3 pp 721ndash729 2013

[27] ITU-T ldquoSubjective video quality assessment methods for mul-timedia applicationsrdquo Recommendation P910 InternationalTelecommunication Union Geneva Switzerland 1999

[28] ITU-T Recommendation QoE Requirements in Consideration ofService Billing for IPTV Service UIT-T FG-IPTV InternationalTelecommunication Union Geneva Switzerland 2006

[29] P Casas P Belzarena I Irigaray and D Guerra ldquoA userperspective for end-to-end quality of service evaluation inmultimedia networksrdquo in Proceedings of the 4th InternationalIFIPACM Latin American Conference on Networking (LANCrsquo07) San Jose Costa Rica 2007

[30] P Casas P Belzarena and S Vaton ldquoEnd-2-end evaluationof IP multimedia services a user perceived quality of serviceapproachrdquo in Proceedings of the 18th ITC Specialist Seminaron Quality of Experience (ITEC rsquo08) Karlskrona Sweden May2008

[31] R Immich P Borges E Cerqueira and M Curado ldquoQoE-driven video delivery improvement using packet loss predic-tionrdquo International Journal of Parallel Emergent andDistributedSystemsmdashSmart Communications in Network Technologies vol30 no 6 pp 478ndash493 2015

[32] M Kailash and D Padma ldquoAn overview of quality of servicemeasurement and optimization for voice over internet protocolimplementationrdquo International Journal of Advances in Engineer-ing amp Technology vol 8 no 1 pp 2053ndash2063 2015

[33] S Timotheou ldquoThe random neural network a surveyrdquo TheComputer Journal vol 53 no 3 pp 251ndash267 2010


[34] K Kilkki ldquoNext generation internet and QoErdquo in Presentationat EuroFGI IA76 Workshop on Socio-Economic Issues of NGISantander Spain June 2007

[35] ITU FG-IPTVDOC-0814 Quality of Experience Requirementsfor IPTV Services 2007

[36] E Cerqueira L Veloso M Curado and E Monteiro QualityLevel Control for Multi-User Sessions in Future Generation Net-works University of Coimbra INESC Porto Coimbra Portugal2008

[37] ZWang A C Bovik H R Sheikh and E P Simoncelli ldquoImagequality assessment from error visibility to structural similarityrdquoIEEE Transactions on Image Processing vol 13 no 4 pp 600ndash612 2004

[38] D Wang W Ding Y Man and L Cui ldquoA joint image qualityassessment method based on global phase coherence andstructural similarityrdquo in Proceedings of the 3rd InternationalCongress on Image and Signal Processing (CISP rsquo10) vol 5 pp2307ndash2311 IEEE Yantai China October 2010

[39] MVranjes S Rimac-Drlje andD Zagar ldquoObjective video qual-ity metricsrdquo in Proceedings of the 49th International SymposiumELMAR Faculty of Electrical Engineering University of OsijekZadar Croatia September 2007

[40] Z Wang and A C Bovik ldquoA universal image quality indexrdquoIEEE Signal Processing Letters vol 9 no 3 pp 81ndash84 2002

[41] YWang Survey of Objective Video QualityMeasurements EMCCorporation Hopkinton Mass USA 2004 ftpftpcswpiedupubtechreportspdf06-02pdf

[42] T Zinner O Abboud O Hohlfeld T Hossfeld and P Train-Gia ldquoTowards QoE management for scalable video streamingrdquoin Proceedings of the 21st ITC Specialist Seminar on MultimediaApplications Traffic Performance and QoE Miyazaki JapanMarch 2010

[43] R Serral-Garcia Y Lu M Yannuzzi X Masip-Bruin and FKuipers Packet Loss Based Quality of Experience of MultimediaVideo Flows Crente de Recerca drsquoArquitectures Avancadesde Xarxes (Politencic University of Cataluna) Spain DelftUniversity of Technology Delft The Netherlands 2009 httppersonalsacupcedurserralresearchtechreportspsnr mospdf

[44] D Botia N Gaviria J Botia D Jimenez and J M MenendezldquoImproved the quality of experience assessment with qualityindex based frames over IPTV networkrdquo in Proceedings of the2nd International Conference on Information and Communi-cation Technologies and Applications (ICTA rsquo12) Orlando FlaUSA 2012

[45] DSL Forum ldquoTriple play services quality of experiencerequierements and machanismrdquo Working Text WT-126 version05 2006

[46] J Klaue B Rathke and A Wolisz ldquoEvalVidmdasha framework forvideo transmission and quality evaluationrdquo in Proceedings of the13th International Conference onModelling Techniques and Toolsfor Computer Performance Evaluation pp 255ndash272 Urbana IllUSA September 2003

[47] D Vatolin ldquoMSU Video Metric Quality Toolrdquo MSU Graph-ics and media Lab 2012 httpcompressionruvideoqualitymeasurevideo measurement tool enhtml Rusia

[48] P Subramanya K Vinayaka H Gururaj and B Ramesh ldquoPer-formance evaluation of high speed TCP variants in dumbbellnetworkrdquo IOSR Journal of Computer Engineering vol 16 no 2pp 49ndash53 2014

[49] D Botia N Gaviria D Jimenez and J M Menendez ldquoStrate-gies for improving the QoE assessment over iTV platforms

based on QoS metricsrdquo in Proceedings of the IEEE InternationalSymposium onMultimedia (ISM rsquo12) pp 483ndash484 Irvine CalifUSA December 2012

[50] QoE-RNN Tool 2011 httpcodegooglecompqoe-rnn[51] G Rubino M Varela and J-M Bonnin ldquoControlling multime-

dia QoS in the future home network using the PSQA metricrdquoThe Computer Journal vol 49 no 2 pp 137ndash155 2006

[52] W Ding Y Tong Q Zhang and D Yang ldquoImage and videoquality assessment using neural network and SVMrdquo TsinghuaScience and Technology vol 13 no 1 pp 112ndash116 2008

[53] A Bouzerdoum A Havstad and A Beghdadi ldquoImage qualityassessment using a neural network approachrdquo in Proceedings ofthe 4th IEEE International Symposium on Signal Processing andInformation Technology pp 330ndash333 Rome Italy December2004

[54] J Choe K Lee and C Lee ldquoNo-reference video qualitymeasurement using neural networksrdquo in Proceedings of the 16thInternational Conference on Digital Signal Processing (DSP rsquo09)pp 1ndash4 IEEE Santorini-Hellas Greece July 2009

[55] X Zhang L Wu Y Fang and H Jiang ldquoA study of FR videoquality assessment of real time video streamrdquo InternationalJournal of Advanced Computer Science and Applications vol 3no 6 2012

[56] C-H Kung W-S Yang C-Y Huan and C-M Kung ldquoInvesti-gation of the image quality assessment using neural networksand structure similartyrdquo in Proceedings of the InternationalSymposium on Computer Science vol 2 no 1 p 219 August2010

[57] C Wang X Jiang F Meng and Y Wang ldquoQuality assessmentfor MPEG-2 video streams using a neural network modelrdquoin Proceedings of the IEEE 13th International Conference onCommunication Technology (ICCT rsquo11) pp 868ndash872 IEEEJinan China September 2011

[58] P Reichl S Egger S Moller et al ldquoTowards a comprehensiveframework for QOE and user behavior modellingrdquo in Proceed-ings of the 17th InternationalWorkshop onQuality ofMultimediaExperience (QoMEX rsquo15) pp 1ndash6 IEEE Pylos-Nestoras GreeceMay 2015

[59] M Ries J Kubanek and M Rupp ldquoVideo quality estimationfor mobile streaming applications with neural networksrdquo inProceedings of the Measurement of Speech and Audio Quality inNetworks Workshop (MESAQIN rsquo06) Prague Czech RepublicJune 2006

[60] P Casas D Guerra and I Bayarres ldquoPerceived Quality of Ser-vice inVoice andVideo Services over IPrdquo School of EngineeringUniversidad de la Republica End of career project ElectricalEngineeringmdashPlan 97 Telecommunications Uruguay 2005

[61] S Mohamed and G Rubino ldquoA study of real-time packet videoquality using random neural networksrdquo IEEE Transactions onCircuits and Systems for Video Technology vol 12 no 12 pp1071ndash1083 2002

[62] VQEG ldquoVideo Quality Experts Group Final Report from theVQEG on the validation of Objective Models of Video QualityAssessment Phase IIrdquo 2003 httpwwwitsbldrdocgovvqegvqeg-homeaspx

[63] S Winkler Digital Video QualitymdashVision Models and MetricsJohn Wiley amp Sons 2005

[64] A Bath I Richardson and S Kannangara A New PerceptualQuality Metric for Compressed Video The Robert Gordon Uni-versity Aberdeen Scotland 2007


[65] M Alreshoodi JWoods and I KMusa ldquoOptimising the deliv-ery of Scalable H264 Video stream byQoSQoE correlationrdquo inProceedings of the IEEE International Conference on ConsumerElectronics (ICCE rsquo15) pp 253ndash254 IEEE Las Vegas Nev USAJanuary 2015

[66] A Kuwadekar and K Al-Begain ldquoUser centric quality of expe-rience testing for video on demand over IMSrdquo in Proceedings ofthe 1st International Conference on Computational IntelligenceCommunication Systems and Networks (CICSYN rsquo09) pp 497ndash504 Indore India July 2009

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014


Active and Passive Electronic Components

Control Scienceand Engineering

Journal of



RotatingMachinery


Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design



Shock and Vibration


Civil EngineeringAdvances in

Acoustics and VibrationAdvances in



Electrical and Computer Engineering

Journal of

Advances inOptoElectronics


Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of


Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014


Chemical EngineeringInternational Journal of Antennas and

Propagation




Navigation and Observation



DistributedSensor Networks



order to identify the optimal values for each metric guaran-teeing the experience of the viewer

According to Winkler [4] there are projects for QoEassessment VQEG (Video Quality Experts Group) QoSM(Quality of ServiceMetrics) fromATIS IPTV InteroperabilityForum and specificmetrics such as those oriented to packetsa bitstream hybrids and images metrics [5ndash8] but these aremore complex and the correlation methods have not yetbeen applied for real time services assessment Other workspropose a new relationship between KPIs (Key PerformanceIndexes)withQoE assessment onmobile environments but itcan be applied in other scenarios as fixed broadbandnetworksfocused particularly on telecommunications providers [9]

One of the main problems on the estimation of thevideo subjective quality is the lack of proper estimationor correlation models These must guarantee results withaccuracy but in many cases heavily depend on objectivemetrics [10ndash12] Also the correlation models in QoE metricsare not accurate and reliable

According to [13] three strategies were developed toperform the estimation of video quality The first one is toapply a subjective assessment with a selected group of peoplethe main drawback of the evaluations is the cost and timeThe second one is the objective quality metrics assessmentfor video in which the principal disadvantage is the lowcorrelation with regard to subjective quality metrics [14]in addition such metrics obviate the network and contentparameters The last one is to use machine learning methodsto analyse the objective and subjective assessment but themajor drawback is the difficult for configuring and testingfurthermore some methods may fail whether a suitabledesign and optimal parameter selection are not performed

According to Kuipers et al [15] the minimum thresholdof accepted quality is a MOS (Mean Opinion Score) value of35 The subjective tests (such as MOS) are widely useful inassessing the users satisfaction due to its accuracy howeverthe application of them is still very complex due to thehigh consumption of time and money therefore they areimpractical for tasks of testing in network devices and acontrolled environment is required is required (in some casesits implementation are complex)

Also if we want to develop trafficmanagement techniquesin real time it is necessary to find a relationship amongthem and the objective metrics measurable by the networkequipment

Objective methods are based on algorithms for theassessment of video quality making them less complexand furthermore can be performed on controlled simula-tion environments The objective metrics are supported inmathematical models that approximate themselves to theHuman Vision System (HVS) behavior and therefore try toestimate as accurately as possible the true QoE Howeverthe perception of each viewer is highly influenced by thequality of the data network expressed by the QoS parameters[26] A lot of proposals for assessing QoE metrics have beencreated to define the userrsquos experience ITU-T has carried outstandards for some of them [27 28]

Due to complex factors such as the HVS different kindsof solutions such as the implementation of machine learning

techniques have been proposed Some of themost usedmeth-ods are Artificial Neural Network (ANN) fuzzy logic basedon rules neural-fuzzy networks (eg ANFIS) support vectormachines (SVM) Gaussian processes and genetic algorithmamong others Nonintrusive QoE estimation methods forvideo are mainly based on application layer and networkparameters

Models as artificial neural networks have been littlestudied for estimation and prediction of video quality For theevaluation of nonintrusive methods we decided to employtwo methods of machine learning BPNN feedforward andRNN (Random Neural Network) We assess the output ofeach system estimated MOS versus expected MOS In eachcase the fit of the correlation determined by Pearson andSpearman rank order correlation coefficients RMSE (RootMean Square Error) and Outliers Ratio were calculated Theexpected MOS was calculated from the VQM (Video QualityMetric) SSIM (Structural Similarity Index Metric) PSNR(Peak Signal Noise Ratio) and QIBF (Quality Index BasedFrame) metrics

This paper presents the key objective and subjectivequality metrics and introduces a nonintrusive QoE assess-ment methodology based on machine learning techniquesshowing its main features and functionality Afterwards thedeveloped testbed is explained both results obtained as theiranalyses are presented Finally the main conclusions andfurther works will be shown

2 Related Works

21 Methods for Quality of Experience Assessment In spiteof all proposals for objective metrics they are always notclose to the human perception due to the fact that theperception is highly influenced by the performance of thenetwork defined in terms of QoS parameters According to[29 30] the objective metrics are computational models thatpredict the image quality perceived by any person and can beclassified as intrusive and nonintrusive methods as shown inFigure 1

The Subjective Pseudo Quality Assessment model orPSQA (Pseudo Subjective Quality Assessment) is an exampleof nonintrusive method (NR) This model uses an RNN(Random Neural Network) [31ndash33] to learn and recognizethe relationship between video and the characteristics of thenetwork with the quality perceived by users

Initially for the training process of the RNN a databaseis required which contains different sequences to assess thedistortions generated by several QoS and coding parametersAfterwards the training of the RNNwith any video sequenceis evaluated in order to validate the MOS measure

22 QoE Metrics

221 PSNR and MSE Metrics The PSNR (Peak Signal NoiseRatio) and MSE (Mean Square Error) metrics are the mostfrequently applied ones [11] They assess the quality of thereceived video sequence and thus can be mapped on asubjective scale PSNR aiming to compare pixel-by-pixel andframe-by-frame the quality of the received image with the


QoE assessment

Subjective methods

Objective methods

Intrusive methods

Nonintrusivemethods







sdot log10(

255


119894=0sum119873minus1

119895=0

1003817100381710038171003817119884119904 (119894 119895) minus 119884119889 (119894 119895)1003817100381710038171003817

2

)

(1)




119904and 119884

119889








SSIM =

(2119909119910 + 1198621) (2120590119909119910+ 1198622)

[(119909)2+ (119910)2

+ 1198621] (1205902119909+ 1205902119910+ 1198622)

(2)


SSIM (119909 119910) = [119897 (119909 119910)]120572

[119888 (119909 119910)]120573

[119904 (119909 119910)]120574

(3)









RR



FR






119876 (119891) = 119872(PLR (119891)) (4)


119872(119909) = 1 minus 119909 (5)


119876119891= 119902 (119891

1) 119902 (119891

119899) (6)

where 1198911119899








sum

119891=1

1 minus NFL (119891)3

minus 120572 (7)



















Network

Video trace file

Video sender

Routeredge 1

Routercore 1

Routeraccess

Evaluate trace

Video receiver 1

Receiver n


Receiver 2

Receiver 3

Sender 2

Sender 3

Sender n

Bottleneck link

Dumbbell topology

PLR = 0 1 5 10

300Mbps001ms

10Mbps011ms

10Mbps001ms

10Mbps001ms

100Mbps001ms

20Mbps001ms

10Mbps001ms

10Mbps001ms

10Mbps001ms

10Mbps5ms
















Video raw sequences

Encoder


Decoder

Display




Original data set

Datatraining

Datatest

(1) BPNNfeedforward


EstimatedMOS

Correlation











MOS Est = 119891 (1199091 1199092 119909

119899) (8)

where 1199091 1199092 119909





LW1 1

LW1 2

LW1 8

MOS Est

BI1

BI2

BI8

IW1 1

IW1 2

IW1 3

IW2 1IW2 2

IW2 3

IW7 8

IW7 7

Tangsig

Tangsig

Tangsig

Linear



Long GOP

PLR

BR

QoSclass

PSNR

SSIM

VQM

QIBF

sumsum

sum

sum

[1ndash5]

120596t+1i = 120596t

i + 120578(yi_yi)










119910 = 119898119909 + 119887 (9)



W

b

+

W

b

+

20 1

18

Input


Output


Mea

n Sq

uare

Err

or (M

SE)

100

10minus5

10minus10

7 epochs0 21 43 65 7

TrainValidation

TestBest



0 10 20 30 40 50 60 70 80 90 1001

152

253

354

455

55

Number of sequences

Estim

ated

and

desir

ed M

OS






15 2 25 3 35 4 45 51

152

253

354

455

Desired MOSEs

timat

ed M

OS









(119896)


therefore

(119896)










119901)



MOS119901=

119873

sum

119894=1

MOSobj119873

(10)










Table7Paperswith

machine

learning

metho

dsfora

ssessin

gQoE

Paper

video

sequ

ences

Resolutio

nCod

ecSSIM

VQM

PSRN

MSE

BRFR


fram

esrate)

QP

Playou

tinterrup

tions

Delay

JitterBW

MLB

SGOP

Machine

learning

Assess

MOSDMOS

Correlatio

nperfo

rmance

PCC

SROCC

RMSE

OR

Hee

tal

[17]2012

1QCI

FMPE

G4

XX

XX

X

Prob

abilistic

Neural

Network

(PNN)

Yes

09286

No

No

No

Kiplietal

[18]2012

ND

Nodata

Nodata

XX

XBP

NN

Yes

0891

No

No

No

Sing

het

al[19]

2012

4HD-720p

H264

XX

XX

RNN

Yes

No

No

037

No

Liae

tal

[20]2011

4Nodata

Nodata

BPNN

No

No

No

No

No

Khanatal

[21]2010

6QCI

FH264

XX

XANFIS

Yes

08717

No

02812

No

Duetal

[22]200

91

SDNodata

XX

XX

XX

BPNN

Yes

No

No

No

No

Piam

ratet

al[23]

2009

1Nodata

H264

XX

XPS

QAwith

RNN

Yes

No

No

No

No

Khanetal

[24]2008

3CI

FMPE

G4

XX

XX

XANFIS

Yes

R=08056

forM

OS

predicted

versus

MOS

Measure

andR=

09229

for

119876

predicted

versus119876

measure

No

RMSE

=01846

for

MOS

predicted

versus

MOS

measure

andRM

SE=006234

for119876

predicted

versus119876

measure

No

Rubino

etal[25]

2004

Only

VoIP

Nodata

Nodata

XX

RNN

No

No

No

No

No

BRbitrateFR

framerateP

LRP

acketL

ossR

ateQP


Wb

andw

idthM

LBSMeanLo

ssBu

rstS

izeGOP

Group

ofPicturesP

CCP

earson

correlationcoeffi

cientSR

OCC

Spearman

rank

orderc

orrelatio

ncoeffi

cientRM

SER

ootM

eanSquare

ErrorandOR

OutlierR

atio



NetworkBestEffort




Input parameters

RNN training script

RNN test script



MOSobj

MOSpsqaMOSp =

N

sumi=1

MOSobj

N


15 2 25 3 35 4 45 51

152

253

354

455

Desired MOS

Estim

ated

MO

S

Fit 1


MOSpsqa versus MOSp



5 Conclusion









Appendices






2(seq nt) for all


1(1) gt NFL

2(1)

NFL1(2) gt NFL

2(2) and NFL

1(3) gt NFL

2(3)


2(seq nt) for all


1(1) = NFL

2(1) =

NFL1(2) = NFL

2(2) = NFL

1(3) = NFL

2(3)





1(2) gt NFL

1(3) and

NFL2(1) gt NFL

2(2) gt NFL


asNFL1(1) gt NFL

2(1) gt NFL

1(2) gt NFL

2(2)

gt NFL1(3) gt NFL

2(3)

(A1)


2(seq nt)

QIBF1(seq nt) =

119865

sum

119891=1

1 minusNFL1(119891)

3minus 120572

QIBF2(seq nt) =

119865

sum

119891=1

1 minusNFL2(119891)

3minus 120572

(A2)

Then


1(seq nt)

= [

[

119865

sum

119891=1

1 minusNFL2(119891)

3minus 120572]

]

minus [

[

119865

sum

119891=1

1 minusNFL1(119891)

3minus 120572]

]


1(seq nt)

=

119865

sum

119891=1

1 minusNFL2(119891)

3minus

119865

sum

119891=1


3

(A3)

If NFL1(1) gt NFL

2(1) gt NFL

1(2) gt NFL

2(2) gt

NFL1(3) gt NFL



1(seq nt) larrrarr

119865

sum

119891=1


3gt

119865

sum

119891=1

1 minusNFL1(119891)

3

(A4)








2(1) = NFL

1(2) = NFL

2(2) =

NFL1(3) = NFL


1(seq nt) =














119889119894+

119873

sum

119895=1

(119901+

119894119895+ 119901plusmn

119894119895) = 1 (B1)



119894or 120582minus119894rate


119894= 0 for any neuron 119894


119873

sum

119894=1

(120582+

119894) gt 0 (B2)



119892119894= lim119905rarrinfin

119875 (119902119894(119905) gt 0) (B3)


997888997888rarr119902(119905) = 119902

1(119905) 119902





119894119895and 119875

minus

119894119895 The above


120596+

119894119895= 119877119894119875+

119894119895

120596minus

119894119895= 119877119894119875minus

119894119895

(B4)


119894119895and 120596minus(119896minus1)




(119896)


therefore (119896)




119894= 0



119894= 119889119894= 0




119878 = 1205721 1205722 120572

119878

1198781015840= 1205721015840

1 1205721015840

2 120572

1015840

119878

119875 = 1205821 1205822 120582

119899

(B5)


119878is

defined by 119881119901119904




119891(1198811119878 1198812119878 119881





Acknowledgment


References









































































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation










QoE assessment

Subjective methods

Objective methods

Intrusive methods

Nonintrusivemethods







sdot log10(

255


119894=0sum119873minus1

119895=0

1003817100381710038171003817119884119904 (119894 119895) minus 119884119889 (119894 119895)1003817100381710038171003817

2

)

(1)




119904and 119884

119889








SSIM =

(2119909119910 + 1198621) (2120590119909119910+ 1198622)

[(119909)2+ (119910)2

+ 1198621] (1205902119909+ 1205902119910+ 1198622)

(2)


SSIM (119909 119910) = [119897 (119909 119910)]120572

[119888 (119909 119910)]120573

[119904 (119909 119910)]120574

(3)









RR



FR






119876 (119891) = 119872(PLR (119891)) (4)


119872(119909) = 1 minus 119909 (5)


119876119891= 119902 (119891

1) 119902 (119891

119899) (6)

where 1198911119899








sum

119891=1

1 minus NFL (119891)3

minus 120572 (7)



















Network

Video trace file

Video sender

Routeredge 1

Routercore 1

Routeraccess

Evaluate trace

Video receiver 1

Receiver n


Receiver 2

Receiver 3

Sender 2

Sender 3

Sender n

Bottleneck link

Dumbbell topology

PLR = 0 1 5 10

300Mbps001ms

10Mbps011ms

10Mbps001ms

10Mbps001ms

100Mbps001ms

20Mbps001ms

10Mbps001ms

10Mbps001ms

10Mbps001ms

10Mbps5ms
















Video raw sequences

Encoder


Decoder

Display




Original data set

Datatraining

Datatest

(1) BPNNfeedforward


EstimatedMOS

Correlation











MOS Est = 119891 (1199091 1199092 119909

119899) (8)

where 1199091 1199092 119909





LW1 1

LW1 2

LW1 8

MOS Est

BI1

BI2

BI8

IW1 1

IW1 2

IW1 3

IW2 1IW2 2

IW2 3

IW7 8

IW7 7

Tangsig

Tangsig

Tangsig

Linear



Long GOP

PLR

BR

QoSclass

PSNR

SSIM

VQM

QIBF

sumsum

sum

sum

[1ndash5]

120596t+1i = 120596t

i + 120578(yi_yi)










119910 = 119898119909 + 119887 (9)



W

b

+

W

b

+

20 1

18

Input


Output


Mea

n Sq

uare

Err

or (M

SE)

100

10minus5

10minus10

7 epochs0 21 43 65 7

TrainValidation

TestBest



0 10 20 30 40 50 60 70 80 90 1001

152

253

354

455

55

Number of sequences

Estim

ated

and

desir

ed M

OS






15 2 25 3 35 4 45 51

152

253

354

455

Desired MOSEs

timat

ed M

OS









(119896)


therefore

(119896)










119901)



MOS119901=

119873

sum

119894=1

MOSobj119873

(10)










Table7Paperswith

machine

learning

metho

dsfora

ssessin

gQoE

Paper

video

sequ

ences

Resolutio

nCod

ecSSIM

VQM

PSRN

MSE

BRFR


fram

esrate)

QP

Playou

tinterrup

tions

Delay

JitterBW

MLB

SGOP

Machine

learning

Assess

MOSDMOS

Correlatio

nperfo

rmance

PCC

SROCC

RMSE

OR

Hee

tal

[17]2012

1QCI

FMPE

G4

XX

XX

X

Prob

abilistic

Neural

Network

(PNN)

Yes

09286

No

No

No

Kiplietal

[18]2012

ND

Nodata

Nodata

XX

XBP

NN

Yes

0891

No

No

No

Sing

het

al[19]

2012

4HD-720p

H264

XX

XX

RNN

Yes

No

No

037

No

Liae

tal

[20]2011

4Nodata

Nodata

BPNN

No

No

No

No

No

Khanatal

[21]2010

6QCI

FH264

XX

XANFIS

Yes

08717

No

02812

No

Duetal

[22]200

91

SDNodata

XX

XX

XX

BPNN

Yes

No

No

No

No

Piam

ratet

al[23]

2009

1Nodata

H264

XX

XPS

QAwith

RNN

Yes

No

No

No

No

Khanetal

[24]2008

3CI

FMPE

G4

XX

XX

XANFIS

Yes

R=08056

forM

OS

predicted

versus

MOS

Measure

andR=

09229

for

119876

predicted

versus119876

measure

No

RMSE

=01846

for

MOS

predicted

versus

MOS

measure

andRM

SE=006234

for119876

predicted

versus119876

measure

No

Rubino

etal[25]

2004

Only

VoIP

Nodata

Nodata

XX

RNN

No

No

No

No

No

BRbitrateFR

framerateP

LRP

acketL

ossR

ateQP


Wb

andw

idthM

LBSMeanLo

ssBu

rstS

izeGOP

Group

ofPicturesP

CCP

earson

correlationcoeffi

cientSR

OCC

Spearman

rank

orderc

orrelatio

ncoeffi

cientRM

SER

ootM

eanSquare

ErrorandOR

OutlierR

atio



NetworkBestEffort




Input parameters

RNN training script

RNN test script



MOSobj

MOSpsqaMOSp =

N

sumi=1

MOSobj

N


15 2 25 3 35 4 45 51

152

253

354

455

Desired MOS

Estim

ated

MO

S

Fit 1


MOSpsqa versus MOSp



5 Conclusion









Appendices






2(seq nt) for all


1(1) gt NFL

2(1)

NFL1(2) gt NFL

2(2) and NFL

1(3) gt NFL

2(3)


2(seq nt) for all


1(1) = NFL

2(1) =

NFL1(2) = NFL

2(2) = NFL

1(3) = NFL

2(3)





1(2) gt NFL

1(3) and

NFL2(1) gt NFL

2(2) gt NFL


asNFL1(1) gt NFL

2(1) gt NFL

1(2) gt NFL

2(2)

gt NFL1(3) gt NFL

2(3)

(A1)


2(seq nt)

QIBF1(seq nt) =

119865

sum

119891=1

1 minusNFL1(119891)

3minus 120572

QIBF2(seq nt) =

119865

sum

119891=1

1 minusNFL2(119891)

3minus 120572

(A2)

Then


1(seq nt)

= [

[

119865

sum

119891=1

1 minusNFL2(119891)

3minus 120572]

]

minus [

[

119865

sum

119891=1

1 minusNFL1(119891)

3minus 120572]

]


1(seq nt)

=

119865

sum

119891=1

1 minusNFL2(119891)

3minus

119865

sum

119891=1


3

(A3)

If NFL1(1) gt NFL

2(1) gt NFL

1(2) gt NFL

2(2) gt

NFL1(3) gt NFL



1(seq nt) larrrarr

119865

sum

119891=1


3gt

119865

sum

119891=1

1 minusNFL1(119891)

3

(A4)








2(1) = NFL

1(2) = NFL

2(2) =

NFL1(3) = NFL


1(seq nt) =














119889119894+

119873

sum

119895=1

(119901+

119894119895+ 119901plusmn

119894119895) = 1 (B1)



119894or 120582minus119894rate


119894= 0 for any neuron 119894


119873

sum

119894=1

(120582+

119894) gt 0 (B2)



119892119894= lim119905rarrinfin

119875 (119902119894(119905) gt 0) (B3)


997888997888rarr119902(119905) = 119902

1(119905) 119902





119894119895and 119875

minus

119894119895 The above


120596+

119894119895= 119877119894119875+

119894119895

120596minus

119894119895= 119877119894119875minus

119894119895

(B4)


119894119895and 120596minus(119896minus1)




(119896)


therefore (119896)




119894= 0



119894= 119889119894= 0




119878 = 1205721 1205722 120572

119878

1198781015840= 1205721015840

1 1205721015840

2 120572

1015840

119878

119875 = 1205821 1205822 120582

119899

(B5)


119878is

defined by 119881119901119904




119891(1198811119878 1198812119878 119881





Acknowledgment


References









































































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation















RR



FR






119876 (119891) = 119872(PLR (119891)) (4)


119872(119909) = 1 minus 119909 (5)


119876119891= 119902 (119891

1) 119902 (119891

119899) (6)

where 1198911119899








sum

119891=1

1 minus NFL (119891)3

minus 120572 (7)



















Network

Video trace file

Video sender

Routeredge 1

Routercore 1

Routeraccess

Evaluate trace

Video receiver 1

Receiver n


Receiver 2

Receiver 3

Sender 2

Sender 3

Sender n

Bottleneck link

Dumbbell topology

PLR = 0 1 5 10

300Mbps001ms

10Mbps011ms

10Mbps001ms

10Mbps001ms

100Mbps001ms

20Mbps001ms

10Mbps001ms

10Mbps001ms

10Mbps001ms

10Mbps5ms
















Video raw sequences

Encoder


Decoder

Display




Original data set

Datatraining

Datatest

(1) BPNNfeedforward


EstimatedMOS

Correlation











MOS Est = 119891 (1199091 1199092 119909

119899) (8)

where 1199091 1199092 119909





LW1 1

LW1 2

LW1 8

MOS Est

BI1

BI2

BI8

IW1 1

IW1 2

IW1 3

IW2 1IW2 2

IW2 3

IW7 8

IW7 7

Tangsig

Tangsig

Tangsig

Linear



Long GOP

PLR

BR

QoSclass

PSNR

SSIM

VQM

QIBF

sumsum

sum

sum

[1ndash5]

120596t+1i = 120596t

i + 120578(yi_yi)










119910 = 119898119909 + 119887 (9)



W

b

+

W

b

+

20 1

18

Input


Output


Mea

n Sq

uare

Err

or (M

SE)

100

10minus5

10minus10

7 epochs0 21 43 65 7

TrainValidation

TestBest



0 10 20 30 40 50 60 70 80 90 1001

152

253

354

455

55

Number of sequences

Estim

ated

and

desir

ed M

OS






15 2 25 3 35 4 45 51

152

253

354

455

Desired MOSEs

timat

ed M

OS









(119896)


therefore

(119896)










119901)



MOS119901=

119873

sum

119894=1

MOSobj119873

(10)










Table7Paperswith

machine

learning

metho

dsfora

ssessin

gQoE

Paper

video

sequ

ences

Resolutio

nCod

ecSSIM

VQM

PSRN

MSE

BRFR


fram

esrate)

QP

Playou

tinterrup

tions

Delay

JitterBW

MLB

SGOP

Machine

learning

Assess

MOSDMOS

Correlatio

nperfo

rmance

PCC

SROCC

RMSE

OR

Hee

tal

[17]2012

1QCI

FMPE

G4

XX

XX

X

Prob

abilistic

Neural

Network

(PNN)

Yes

09286

No

No

No

Kiplietal

[18]2012

ND

Nodata

Nodata

XX

XBP

NN

Yes

0891

No

No

No

Sing

het

al[19]

2012

4HD-720p

H264

XX

XX

RNN

Yes

No

No

037

No

Liae

tal

[20]2011

4Nodata

Nodata

BPNN

No

No

No

No

No

Khanatal

[21]2010

6QCI

FH264

XX

XANFIS

Yes

08717

No

02812

No

Duetal

[22]200

91

SDNodata

XX

XX

XX

BPNN

Yes

No

No

No

No

Piam

ratet

al[23]

2009

1Nodata

H264

XX

XPS

QAwith

RNN

Yes

No

No

No

No

Khanetal

[24]2008

3CI

FMPE

G4

XX

XX

XANFIS

Yes

R=08056

forM

OS

predicted

versus

MOS

Measure

andR=

09229

for

119876

predicted

versus119876

measure

No

RMSE

=01846

for

MOS

predicted

versus

MOS

measure

andRM

SE=006234

for119876

predicted

versus119876

measure

No

Rubino

etal[25]

2004

Only

VoIP

Nodata

Nodata

XX

RNN

No

No

No

No

No

BRbitrateFR

framerateP

LRP

acketL

ossR

ateQP


Wb

andw

idthM

LBSMeanLo

ssBu

rstS

izeGOP

Group

ofPicturesP

CCP

earson

correlationcoeffi

cientSR

OCC

Spearman

rank

orderc

orrelatio

ncoeffi

cientRM

SER

ootM

eanSquare

ErrorandOR

OutlierR

atio



NetworkBestEffort




Input parameters

RNN training script

RNN test script



MOSobj

MOSpsqaMOSp =

N

sumi=1

MOSobj

N


15 2 25 3 35 4 45 51

152

253

354

455

Desired MOS

Estim

ated

MO

S

Fit 1


MOSpsqa versus MOSp



5 Conclusion









Appendices






2(seq nt) for all


1(1) gt NFL

2(1)

NFL1(2) gt NFL

2(2) and NFL

1(3) gt NFL

2(3)


2(seq nt) for all


1(1) = NFL

2(1) =

NFL1(2) = NFL

2(2) = NFL

1(3) = NFL

2(3)





1(2) gt NFL

1(3) and

NFL2(1) gt NFL

2(2) gt NFL


asNFL1(1) gt NFL

2(1) gt NFL

1(2) gt NFL

2(2)

gt NFL1(3) gt NFL

2(3)

(A1)


2(seq nt)

QIBF1(seq nt) =

119865

sum

119891=1

1 minusNFL1(119891)

3minus 120572

QIBF2(seq nt) =

119865

sum

119891=1

1 minusNFL2(119891)

3minus 120572

(A2)

Then


1(seq nt)

= [

[

119865

sum

119891=1

1 minusNFL2(119891)

3minus 120572]

]

minus [

[

119865

sum

119891=1

1 minusNFL1(119891)

3minus 120572]

]


1(seq nt)

=

119865

sum

119891=1

1 minusNFL2(119891)

3minus

119865

sum

119891=1


3

(A3)

If NFL1(1) gt NFL

2(1) gt NFL

1(2) gt NFL

2(2) gt

NFL1(3) gt NFL



1(seq nt) larrrarr

119865

sum

119891=1


3gt

119865

sum

119891=1

1 minusNFL1(119891)

3

(A4)








2(1) = NFL

1(2) = NFL

2(2) =

NFL1(3) = NFL


1(seq nt) =














119889119894+

119873

sum

119895=1

(119901+

119894119895+ 119901plusmn

119894119895) = 1 (B1)



119894or 120582minus119894rate


119894= 0 for any neuron 119894


119873

sum

119894=1

(120582+

119894) gt 0 (B2)



119892119894= lim119905rarrinfin

119875 (119902119894(119905) gt 0) (B3)


997888997888rarr119902(119905) = 119902

1(119905) 119902





119894119895and 119875

minus

119894119895 The above


120596+

119894119895= 119877119894119875+

119894119895

120596minus

119894119895= 119877119894119875minus

119894119895

(B4)


119894119895and 120596minus(119896minus1)




(119896)


therefore (119896)




119894= 0



119894= 119889119894= 0




119878 = 1205721 1205722 120572

119878

1198781015840= 1205721015840

1 1205721015840

2 120572

1015840

119878

119875 = 1205821 1205822 120582

119899

(B5)


119878is

defined by 119881119901119904




119891(1198811119878 1198812119878 119881





Acknowledgment


References









































































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation























Network

Video trace file

Video sender

Routeredge 1

Routercore 1

Routeraccess

Evaluate trace

Video receiver 1

Receiver n


Receiver 2

Receiver 3

Sender 2

Sender 3

Sender n

Bottleneck link

Dumbbell topology

PLR = 0 1 5 10

300Mbps001ms

10Mbps011ms

10Mbps001ms

10Mbps001ms

100Mbps001ms

20Mbps001ms

10Mbps001ms

10Mbps001ms

10Mbps001ms

10Mbps5ms
















Video raw sequences

Encoder


Decoder

Display




Original data set

Datatraining

Datatest

(1) BPNNfeedforward


EstimatedMOS

Correlation











MOS Est = 119891 (1199091 1199092 119909

119899) (8)

where 1199091 1199092 119909





LW1 1

LW1 2

LW1 8

MOS Est

BI1

BI2

BI8

IW1 1

IW1 2

IW1 3

IW2 1IW2 2

IW2 3

IW7 8

IW7 7

Tangsig

Tangsig

Tangsig

Linear



Long GOP

PLR

BR

QoSclass

PSNR

SSIM

VQM

QIBF

sumsum

sum

sum

[1ndash5]

120596t+1i = 120596t

i + 120578(yi_yi)










119910 = 119898119909 + 119887 (9)



W

b

+

W

b

+

20 1

18

Input


Output


Mea

n Sq

uare

Err

or (M

SE)

100

10minus5

10minus10

7 epochs0 21 43 65 7

TrainValidation

TestBest



0 10 20 30 40 50 60 70 80 90 1001

152

253

354

455

55

Number of sequences

Estim

ated

and

desir

ed M

OS






15 2 25 3 35 4 45 51

152

253

354

455

Desired MOSEs

timat

ed M

OS









(119896)


therefore

(119896)










119901)



MOS119901=

119873

sum

119894=1

MOSobj119873

(10)










Table7Paperswith

machine

learning

metho

dsfora

ssessin

gQoE

Paper

video

sequ

ences

Resolutio

nCod

ecSSIM

VQM

PSRN

MSE

BRFR


fram

esrate)

QP

Playou

tinterrup

tions

Delay

JitterBW

MLB

SGOP

Machine

learning

Assess

MOSDMOS

Correlatio

nperfo

rmance

PCC

SROCC

RMSE

OR

Hee

tal

[17]2012

1QCI

FMPE

G4

XX

XX

X

Prob

abilistic

Neural

Network

(PNN)

Yes

09286

No

No

No

Kiplietal

[18]2012

ND

Nodata

Nodata

XX

XBP

NN

Yes

0891

No

No

No

Sing

het

al[19]

2012

4HD-720p

H264

XX

XX

RNN

Yes

No

No

037

No

Liae

tal

[20]2011

4Nodata

Nodata

BPNN

No

No

No

No

No

Khanatal

[21]2010

6QCI

FH264

XX

XANFIS

Yes

08717

No

02812

No

Duetal

[22]200

91

SDNodata

XX

XX

XX

BPNN

Yes

No

No

No

No

Piam

ratet

al[23]

2009

1Nodata

H264

XX

XPS

QAwith

RNN

Yes

No

No

No

No

Khanetal

[24]2008

3CI

FMPE

G4

XX

XX

XANFIS

Yes

R=08056

forM

OS

predicted

versus

MOS

Measure

andR=

09229

for

119876

predicted

versus119876

measure

No

RMSE

=01846

for

MOS

predicted

versus

MOS

measure

andRM

SE=006234

for119876

predicted

versus119876

measure

No

Rubino

etal[25]

2004

Only

VoIP

Nodata

Nodata

XX

RNN

No

No

No

No

No

BRbitrateFR

framerateP

LRP

acketL

ossR

ateQP


Wb

andw

idthM

LBSMeanLo

ssBu

rstS

izeGOP

Group

ofPicturesP

CCP

earson

correlationcoeffi

cientSR

OCC

Spearman

rank

orderc

orrelatio

ncoeffi

cientRM

SER

ootM

eanSquare

ErrorandOR

OutlierR

atio



NetworkBestEffort




Input parameters

RNN training script

RNN test script



MOSobj

MOSpsqaMOSp =

N

sumi=1

MOSobj

N


15 2 25 3 35 4 45 51

152

253

354

455

Desired MOS

Estim

ated

MO

S

Fit 1


MOSpsqa versus MOSp



5 Conclusion









Appendices






2(seq nt) for all


1(1) gt NFL

2(1)

NFL1(2) gt NFL

2(2) and NFL

1(3) gt NFL

2(3)


2(seq nt) for all


1(1) = NFL

2(1) =

NFL1(2) = NFL

2(2) = NFL

1(3) = NFL

2(3)





1(2) gt NFL

1(3) and

NFL2(1) gt NFL

2(2) gt NFL


asNFL1(1) gt NFL

2(1) gt NFL

1(2) gt NFL

2(2)

gt NFL1(3) gt NFL

2(3)

(A1)


2(seq nt)

QIBF1(seq nt) =

119865

sum

119891=1

1 minusNFL1(119891)

3minus 120572

QIBF2(seq nt) =

119865

sum

119891=1

1 minusNFL2(119891)

3minus 120572

(A2)

Then


1(seq nt)

= [

[

119865

sum

119891=1

1 minusNFL2(119891)

3minus 120572]

]

minus [

[

119865

sum

119891=1

1 minusNFL1(119891)

3minus 120572]

]


1(seq nt)

=

119865

sum

119891=1

1 minusNFL2(119891)

3minus

119865

sum

119891=1


3

(A3)

If NFL1(1) gt NFL

2(1) gt NFL

1(2) gt NFL

2(2) gt

NFL1(3) gt NFL



1(seq nt) larrrarr

119865

sum

119891=1


3gt

119865

sum

119891=1

1 minusNFL1(119891)

3

(A4)








2(1) = NFL

1(2) = NFL

2(2) =

NFL1(3) = NFL


1(seq nt) =














119889119894+

119873

sum

119895=1

(119901+

119894119895+ 119901plusmn

119894119895) = 1 (B1)



119894or 120582minus119894rate


119894= 0 for any neuron 119894


119873

sum

119894=1

(120582+

119894) gt 0 (B2)



119892119894= lim119905rarrinfin

119875 (119902119894(119905) gt 0) (B3)


997888997888rarr119902(119905) = 119902

1(119905) 119902





119894119895and 119875

minus

119894119895 The above


120596+

119894119895= 119877119894119875+

119894119895

120596minus

119894119895= 119877119894119875minus

119894119895

(B4)


119894119895and 120596minus(119896minus1)




(119896)


therefore (119896)




119894= 0



119894= 119889119894= 0




119878 = 1205721 1205722 120572

119878

1198781015840= 1205721015840

1 1205721015840

2 120572

1015840

119878

119875 = 1205821 1205822 120582

119899

(B5)


119878is

defined by 119881119901119904




119891(1198811119878 1198812119878 119881





Acknowledgment


References









































































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation










Video raw sequences

Encoder


Decoder

Display




Original data set

Datatraining

Datatest

(1) BPNNfeedforward


EstimatedMOS

Correlation











MOS Est = 119891 (1199091 1199092 119909

119899) (8)

where 1199091 1199092 119909





LW1 1

LW1 2

LW1 8

MOS Est

BI1

BI2

BI8

IW1 1

IW1 2

IW1 3

IW2 1IW2 2

IW2 3

IW7 8

IW7 7

Tangsig

Tangsig

Tangsig

Linear



Long GOP

PLR

BR

QoSclass

PSNR

SSIM

VQM

QIBF

sumsum

sum

sum

[1ndash5]

120596t+1i = 120596t

i + 120578(yi_yi)










119910 = 119898119909 + 119887 (9)



W

b

+

W

b

+

20 1

18

Input


Output


Mea

n Sq

uare

Err

or (M

SE)

100

10minus5

10minus10

7 epochs0 21 43 65 7

TrainValidation

TestBest



0 10 20 30 40 50 60 70 80 90 1001

152

253

354

455

55

Number of sequences

Estim

ated

and

desir

ed M

OS






15 2 25 3 35 4 45 51

152

253

354

455

Desired MOSEs

timat

ed M

OS









(119896)


therefore

(119896)










119901)



MOS119901=

119873

sum

119894=1

MOSobj119873

(10)










Table7Paperswith

machine

learning

metho

dsfora

ssessin

gQoE

Paper

video

sequ

ences

Resolutio

nCod

ecSSIM

VQM

PSRN

MSE

BRFR


fram

esrate)

QP

Playou

tinterrup

tions

Delay

JitterBW

MLB

SGOP

Machine

learning

Assess

MOSDMOS

Correlatio

nperfo

rmance

PCC

SROCC

RMSE

OR

Hee

tal

[17]2012

1QCI

FMPE

G4

XX

XX

X

Prob

abilistic

Neural

Network

(PNN)

Yes

09286

No

No

No

Kiplietal

[18]2012

ND

Nodata

Nodata

XX

XBP

NN

Yes

0891

No

No

No

Sing

het

al[19]

2012

4HD-720p

H264

XX

XX

RNN

Yes

No

No

037

No

Liae

tal

[20]2011

4Nodata

Nodata

BPNN

No

No

No

No

No

Khanatal

[21]2010

6QCI

FH264

XX

XANFIS

Yes

08717

No

02812

No

Duetal

[22]200

91

SDNodata

XX

XX

XX

BPNN

Yes

No

No

No

No

Piam

ratet

al[23]

2009

1Nodata

H264

XX

XPS

QAwith

RNN

Yes

No

No

No

No

Khanetal

[24]2008

3CI

FMPE

G4

XX

XX

XANFIS

Yes

R=08056

forM

OS

predicted

versus

MOS

Measure

andR=

09229

for

119876

predicted

versus119876

measure

No

RMSE

=01846

for

MOS

predicted

versus

MOS

measure

andRM

SE=006234

for119876

predicted

versus119876

measure

No

Rubino

etal[25]

2004

Only

VoIP

Nodata

Nodata

XX

RNN

No

No

No

No

No

BRbitrateFR

framerateP

LRP

acketL

ossR

ateQP


Wb

andw

idthM

LBSMeanLo

ssBu

rstS

izeGOP

Group

ofPicturesP

CCP

earson

correlationcoeffi

cientSR

OCC

Spearman

rank

orderc

orrelatio

ncoeffi

cientRM

SER

ootM

eanSquare

ErrorandOR

OutlierR

atio



NetworkBestEffort




Input parameters

RNN training script

RNN test script



MOSobj

MOSpsqaMOSp =

N

sumi=1

MOSobj

N


15 2 25 3 35 4 45 51

152

253

354

455

Desired MOS

Estim

ated

MO

S

Fit 1


MOSpsqa versus MOSp



5 Conclusion









Appendices






2(seq nt) for all


1(1) gt NFL

2(1)

NFL1(2) gt NFL

2(2) and NFL

1(3) gt NFL

2(3)


2(seq nt) for all


1(1) = NFL

2(1) =

NFL1(2) = NFL

2(2) = NFL

1(3) = NFL

2(3)





1(2) gt NFL

1(3) and

NFL2(1) gt NFL

2(2) gt NFL


asNFL1(1) gt NFL

2(1) gt NFL

1(2) gt NFL

2(2)

gt NFL1(3) gt NFL

2(3)

(A1)


2(seq nt)

QIBF1(seq nt) =

119865

sum

119891=1

1 minusNFL1(119891)

3minus 120572

QIBF2(seq nt) =

119865

sum

119891=1

1 minusNFL2(119891)

3minus 120572

(A2)

Then


1(seq nt)

= [

[

119865

sum

119891=1

1 minusNFL2(119891)

3minus 120572]

]

minus [

[

119865

sum

119891=1

1 minusNFL1(119891)

3minus 120572]

]


1(seq nt)

=

119865

sum

119891=1

1 minusNFL2(119891)

3minus

119865

sum

119891=1


3

(A3)

If NFL1(1) gt NFL

2(1) gt NFL

1(2) gt NFL

2(2) gt

NFL1(3) gt NFL



1(seq nt) larrrarr

119865

sum

119891=1


3gt

119865

sum

119891=1

1 minusNFL1(119891)

3

(A4)








2(1) = NFL

1(2) = NFL

2(2) =

NFL1(3) = NFL


1(seq nt) =














119889119894+

119873

sum

119895=1

(119901+

119894119895+ 119901plusmn

119894119895) = 1 (B1)



119894or 120582minus119894rate


119894= 0 for any neuron 119894


119873

sum

119894=1

(120582+

119894) gt 0 (B2)



119892119894= lim119905rarrinfin

119875 (119902119894(119905) gt 0) (B3)


997888997888rarr119902(119905) = 119902

1(119905) 119902





119894119895and 119875

minus

119894119895 The above


120596+

119894119895= 119877119894119875+

119894119895

120596minus

119894119895= 119877119894119875minus

119894119895

(B4)


119894119895and 120596minus(119896minus1)




(119896)


therefore (119896)




119894= 0



119894= 119889119894= 0




119878 = 1205721 1205722 120572

119878

1198781015840= 1205721015840

1 1205721015840

2 120572

1015840

119878

119875 = 1205821 1205822 120582

119899

(B5)


119878is

defined by 119881119901119904




119891(1198811119878 1198812119878 119881





Acknowledgment


References









































































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation










LW1 1

LW1 2

LW1 8

MOS Est

BI1

BI2

BI8

IW1 1

IW1 2

IW1 3

IW2 1IW2 2

IW2 3

IW7 8

IW7 7

Tangsig

Tangsig

Tangsig

Linear



Long GOP

PLR

BR

QoSclass

PSNR

SSIM

VQM

QIBF

sumsum

sum

sum

[1ndash5]

120596t+1i = 120596t

i + 120578(yi_yi)










119910 = 119898119909 + 119887 (9)



W

b

+

W

b

+

20 1

18

Input


Output


Mea

n Sq

uare

Err

or (M

SE)

100

10minus5

10minus10

7 epochs0 21 43 65 7

TrainValidation

TestBest



0 10 20 30 40 50 60 70 80 90 1001

152

253

354

455

55

Number of sequences

Estim

ated

and

desir

ed M

OS






15 2 25 3 35 4 45 51

152

253

354

455

Desired MOSEs

timat

ed M

OS









(119896)


therefore

(119896)










119901)



MOS119901=

119873

sum

119894=1

MOSobj119873

(10)










Table7Paperswith

machine

learning

metho

dsfora

ssessin

gQoE

Paper

video

sequ

ences

Resolutio

nCod

ecSSIM

VQM

PSRN

MSE

BRFR


fram

esrate)

QP

Playou

tinterrup

tions

Delay

JitterBW

MLB

SGOP

Machine

learning

Assess

MOSDMOS

Correlatio

nperfo

rmance

PCC

SROCC

RMSE

OR

Hee

tal

[17]2012

1QCI

FMPE

G4

XX

XX

X

Prob

abilistic

Neural

Network

(PNN)

Yes

09286

No

No

No

Kiplietal

[18]2012

ND

Nodata

Nodata

XX

XBP

NN

Yes

0891

No

No

No

Sing

het

al[19]

2012

4HD-720p

H264

XX

XX

RNN

Yes

No

No

037

No

Liae

tal

[20]2011

4Nodata

Nodata

BPNN

No

No

No

No

No

Khanatal

[21]2010

6QCI

FH264

XX

XANFIS

Yes

08717

No

02812

No

Duetal

[22]200

91

SDNodata

XX

XX

XX

BPNN

Yes

No

No

No

No

Piam

ratet

al[23]

2009

1Nodata

H264

XX

XPS

QAwith

RNN

Yes

No

No

No

No

Khanetal

[24]2008

3CI

FMPE

G4

XX

XX

XANFIS

Yes

R=08056

forM

OS

predicted

versus

MOS

Measure

andR=

09229

for

119876

predicted

versus119876

measure

No

RMSE

=01846

for

MOS

predicted

versus

MOS

measure

andRM

SE=006234

for119876

predicted

versus119876

measure

No

Rubino

etal[25]

2004

Only

VoIP

Nodata

Nodata

XX

RNN

No

No

No

No

No

BRbitrateFR

framerateP

LRP

acketL

ossR

ateQP


Wb

andw

idthM

LBSMeanLo

ssBu

rstS

izeGOP

Group

ofPicturesP

CCP

earson

correlationcoeffi

cientSR

OCC

Spearman

rank

orderc

orrelatio

ncoeffi

cientRM

SER

ootM

eanSquare

ErrorandOR

OutlierR

atio



NetworkBestEffort




Input parameters

RNN training script

RNN test script



MOSobj

MOSpsqaMOSp =

N

sumi=1

MOSobj

N


15 2 25 3 35 4 45 51

152

253

354

455

Desired MOS

Estim

ated

MO

S

Fit 1


MOSpsqa versus MOSp



5 Conclusion









Appendices






2(seq nt) for all


1(1) gt NFL

2(1)

NFL1(2) gt NFL

2(2) and NFL

1(3) gt NFL

2(3)


2(seq nt) for all


1(1) = NFL

2(1) =

NFL1(2) = NFL

2(2) = NFL

1(3) = NFL

2(3)





1(2) gt NFL

1(3) and

NFL2(1) gt NFL

2(2) gt NFL


asNFL1(1) gt NFL

2(1) gt NFL

1(2) gt NFL

2(2)

gt NFL1(3) gt NFL

2(3)

(A1)


2(seq nt)

QIBF1(seq nt) =

119865

sum

119891=1

1 minusNFL1(119891)

3minus 120572

QIBF2(seq nt) =

119865

sum

119891=1

1 minusNFL2(119891)

3minus 120572

(A2)

Then


1(seq nt)

= [

[

119865

sum

119891=1

1 minusNFL2(119891)

3minus 120572]

]

minus [

[

119865

sum

119891=1

1 minusNFL1(119891)

3minus 120572]

]


1(seq nt)

=

119865

sum

119891=1

1 minusNFL2(119891)

3minus

119865

sum

119891=1


3

(A3)

If NFL1(1) gt NFL

2(1) gt NFL

1(2) gt NFL

2(2) gt

NFL1(3) gt NFL



1(seq nt) larrrarr

119865

sum

119891=1


3gt

119865

sum

119891=1

1 minusNFL1(119891)

3

(A4)








2(1) = NFL

1(2) = NFL

2(2) =

NFL1(3) = NFL


1(seq nt) =














119889119894+

119873

sum

119895=1

(119901+

119894119895+ 119901plusmn

119894119895) = 1 (B1)



119894or 120582minus119894rate


119894= 0 for any neuron 119894


119873

sum

119894=1

(120582+

119894) gt 0 (B2)



119892119894= lim119905rarrinfin

119875 (119902119894(119905) gt 0) (B3)


997888997888rarr119902(119905) = 119902

1(119905) 119902





119894119895and 119875

minus

119894119895 The above


120596+

119894119895= 119877119894119875+

119894119895

120596minus

119894119895= 119877119894119875minus

119894119895

(B4)


119894119895and 120596minus(119896minus1)




(119896)


therefore (119896)




119894= 0



119894= 119889119894= 0




119878 = 1205721 1205722 120572

119878

1198781015840= 1205721015840

1 1205721015840

2 120572

1015840

119878

119875 = 1205821 1205822 120582

119899

(B5)


119878is

defined by 119881119901119904




119891(1198811119878 1198812119878 119881





Acknowledgment


References









































































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation










W

b

+

W

b

+

20 1

18

Input


Output


Mea

n Sq

uare

Err

or (M

SE)

100

10minus5

10minus10

7 epochs0 21 43 65 7

TrainValidation

TestBest



0 10 20 30 40 50 60 70 80 90 1001

152

253

354

455

55

Number of sequences

Estim

ated

and

desir

ed M

OS






15 2 25 3 35 4 45 51

152

253

354

455

Desired MOSEs

timat

ed M

OS









(119896)


therefore

(119896)










119901)



MOS119901=

119873

sum

119894=1

MOSobj119873

(10)










Table7Paperswith

machine

learning

metho

dsfora

ssessin

gQoE

Paper

video

sequ

ences

Resolutio

nCod

ecSSIM

VQM

PSRN

MSE

BRFR


fram

esrate)

QP

Playou

tinterrup

tions

Delay

JitterBW

MLB

SGOP

Machine

learning

Assess

MOSDMOS

Correlatio

nperfo

rmance

PCC

SROCC

RMSE

OR

Hee

tal

[17]2012

1QCI

FMPE

G4

XX

XX

X

Prob

abilistic

Neural

Network

(PNN)

Yes

09286

No

No

No

Kiplietal

[18]2012

ND

Nodata

Nodata

XX

XBP

NN

Yes

0891

No

No

No

Sing

het

al[19]

2012

4HD-720p

H264

XX

XX

RNN

Yes

No

No

037

No

Liae

tal

[20]2011

4Nodata

Nodata

BPNN

No

No

No

No

No

Khanatal

[21]2010

6QCI

FH264

XX

XANFIS

Yes

08717

No

02812

No

Duetal

[22]200

91

SDNodata

XX

XX

XX

BPNN

Yes

No

No

No

No

Piam

ratet

al[23]

2009

1Nodata

H264

XX

XPS

QAwith

RNN

Yes

No

No

No

No

Khanetal

[24]2008

3CI

FMPE

G4

XX

XX

XANFIS

Yes

R=08056

forM

OS

predicted

versus

MOS

Measure

andR=

09229

for

119876

predicted

versus119876

measure

No

RMSE

=01846

for

MOS

predicted

versus

MOS

measure

andRM

SE=006234

for119876

predicted

versus119876

measure

No

Rubino

etal[25]

2004

Only

VoIP

Nodata

Nodata

XX

RNN

No

No

No

No

No

BRbitrateFR

framerateP

LRP

acketL

ossR

ateQP


Wb

andw

idthM

LBSMeanLo

ssBu

rstS

izeGOP

Group

ofPicturesP

CCP

earson

correlationcoeffi

cientSR

OCC

Spearman

rank

orderc

orrelatio

ncoeffi

cientRM

SER

ootM

eanSquare

ErrorandOR

OutlierR

atio



NetworkBestEffort




Input parameters

RNN training script

RNN test script



MOSobj

MOSpsqaMOSp =

N

sumi=1

MOSobj

N


15 2 25 3 35 4 45 51

152

253

354

455

Desired MOS

Estim

ated

MO

S

Fit 1


MOSpsqa versus MOSp



5 Conclusion









Appendices






2(seq nt) for all


1(1) gt NFL

2(1)

NFL1(2) gt NFL

2(2) and NFL

1(3) gt NFL

2(3)


2(seq nt) for all


1(1) = NFL

2(1) =

NFL1(2) = NFL

2(2) = NFL

1(3) = NFL

2(3)





1(2) gt NFL

1(3) and

NFL2(1) gt NFL

2(2) gt NFL


asNFL1(1) gt NFL

2(1) gt NFL

1(2) gt NFL

2(2)

gt NFL1(3) gt NFL

2(3)

(A1)


2(seq nt)

QIBF1(seq nt) =

119865

sum

119891=1

1 minusNFL1(119891)

3minus 120572

QIBF2(seq nt) =

119865

sum

119891=1

1 minusNFL2(119891)

3minus 120572

(A2)

Then


1(seq nt)

= [

[

119865

sum

119891=1

1 minusNFL2(119891)

3minus 120572]

]

minus [

[

119865

sum

119891=1

1 minusNFL1(119891)

3minus 120572]

]


1(seq nt)

=

119865

sum

119891=1

1 minusNFL2(119891)

3minus

119865

sum

119891=1


3

(A3)

If NFL1(1) gt NFL

2(1) gt NFL

1(2) gt NFL

2(2) gt

NFL1(3) gt NFL



1(seq nt) larrrarr

119865

sum

119891=1


3gt

119865

sum

119891=1

1 minusNFL1(119891)

3

(A4)








2(1) = NFL

1(2) = NFL

2(2) =

NFL1(3) = NFL


1(seq nt) =














119889119894+

119873

sum

119895=1

(119901+

119894119895+ 119901plusmn

119894119895) = 1 (B1)



119894or 120582minus119894rate


119894= 0 for any neuron 119894


119873

sum

119894=1

(120582+

119894) gt 0 (B2)



119892119894= lim119905rarrinfin

119875 (119902119894(119905) gt 0) (B3)


997888997888rarr119902(119905) = 119902

1(119905) 119902





119894119895and 119875

minus

119894119895 The above


120596+

119894119895= 119877119894119875+

119894119895

120596minus

119894119895= 119877119894119875minus

119894119895

(B4)


119894119895and 120596minus(119896minus1)




(119896)


therefore (119896)




119894= 0



119894= 119889119894= 0




119878 = 1205721 1205722 120572

119878

1198781015840= 1205721015840

1 1205721015840

2 120572

1015840

119878

119875 = 1205821 1205822 120582

119899

(B5)


119878is

defined by 119881119901119904




119891(1198811119878 1198812119878 119881





Acknowledgment


References









































































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation
















119901)



MOS119901=

119873

sum

119894=1

MOSobj119873

(10)










Table7Paperswith

machine

learning

metho

dsfora

ssessin

gQoE

Paper

video

sequ

ences

Resolutio

nCod

ecSSIM

VQM

PSRN

MSE

BRFR


fram

esrate)

QP

Playou

tinterrup

tions

Delay

JitterBW

MLB

SGOP

Machine

learning

Assess

MOSDMOS

Correlatio

nperfo

rmance

PCC

SROCC

RMSE

OR

Hee

tal

[17]2012

1QCI

FMPE

G4

XX

XX

X

Prob

abilistic

Neural

Network

(PNN)

Yes

09286

No

No

No

Kiplietal

[18]2012

ND

Nodata

Nodata

XX

XBP

NN

Yes

0891

No

No

No

Sing

het

al[19]

2012

4HD-720p

H264

XX

XX

RNN

Yes

No

No

037

No

Liae

tal

[20]2011

4Nodata

Nodata

BPNN

No

No

No

No

No

Khanatal

[21]2010

6QCI

FH264

XX

XANFIS

Yes

08717

No

02812

No

Duetal

[22]200

91

SDNodata

XX

XX

XX

BPNN

Yes

No

No

No

No

Piam

ratet

al[23]

2009

1Nodata

H264

XX

XPS

QAwith

RNN

Yes

No

No

No

No

Khanetal

[24]2008

3CI

FMPE

G4

XX

XX

XANFIS

Yes

R=08056

forM

OS

predicted

versus

MOS

Measure

andR=

09229

for

119876

predicted

versus119876

measure

No

RMSE

=01846

for

MOS

predicted

versus

MOS

measure

andRM

SE=006234

for119876

predicted

versus119876

measure

No

Rubino

etal[25]

2004

Only

VoIP

Nodata

Nodata

XX

RNN

No

No

No

No

No

BRbitrateFR

framerateP

LRP

acketL

ossR

ateQP


Wb

andw

idthM

LBSMeanLo

ssBu

rstS

izeGOP

Group

ofPicturesP

CCP

earson

correlationcoeffi

cientSR

OCC

Spearman

rank

orderc

orrelatio

ncoeffi

cientRM

SER

ootM

eanSquare

ErrorandOR

OutlierR

atio



NetworkBestEffort




Input parameters

RNN training script

RNN test script



MOSobj

MOSpsqaMOSp =

N

sumi=1

MOSobj

N


15 2 25 3 35 4 45 51

152

253

354

455

Desired MOS

Estim

ated

MO

S

Fit 1


MOSpsqa versus MOSp



5 Conclusion









Appendices






2(seq nt) for all


1(1) gt NFL

2(1)

NFL1(2) gt NFL

2(2) and NFL

1(3) gt NFL

2(3)


2(seq nt) for all


1(1) = NFL

2(1) =

NFL1(2) = NFL

2(2) = NFL

1(3) = NFL

2(3)





1(2) gt NFL

1(3) and

NFL2(1) gt NFL

2(2) gt NFL


asNFL1(1) gt NFL

2(1) gt NFL

1(2) gt NFL

2(2)

gt NFL1(3) gt NFL

2(3)

(A1)


2(seq nt)

QIBF1(seq nt) =

119865

sum

119891=1

1 minusNFL1(119891)

3minus 120572

QIBF2(seq nt) =

119865

sum

119891=1

1 minusNFL2(119891)

3minus 120572

(A2)

Then


1(seq nt)

= [

[

119865

sum

119891=1

1 minusNFL2(119891)

3minus 120572]

]

minus [

[

119865

sum

119891=1

1 minusNFL1(119891)

3minus 120572]

]


1(seq nt)

=

119865

sum

119891=1

1 minusNFL2(119891)

3minus

119865

sum

119891=1


3

(A3)

If NFL1(1) gt NFL

2(1) gt NFL

1(2) gt NFL

2(2) gt

NFL1(3) gt NFL



1(seq nt) larrrarr

119865

sum

119891=1


3gt

119865

sum

119891=1

1 minusNFL1(119891)

3

(A4)








2(1) = NFL

1(2) = NFL

2(2) =

NFL1(3) = NFL


1(seq nt) =














119889119894+

119873

sum

119895=1

(119901+

119894119895+ 119901plusmn

119894119895) = 1 (B1)



119894or 120582minus119894rate


119894= 0 for any neuron 119894


119873

sum

119894=1

(120582+

119894) gt 0 (B2)



119892119894= lim119905rarrinfin

119875 (119902119894(119905) gt 0) (B3)


997888997888rarr119902(119905) = 119902

1(119905) 119902





119894119895and 119875

minus

119894119895 The above


120596+

119894119895= 119877119894119875+

119894119895

120596minus

119894119895= 119877119894119875minus

119894119895

(B4)


119894119895and 120596minus(119896minus1)




(119896)


therefore (119896)




119894= 0



119894= 119889119894= 0




119878 = 1205721 1205722 120572

119878

1198781015840= 1205721015840

1 1205721015840

2 120572

1015840

119878

119875 = 1205821 1205822 120582

119899

(B5)


119878is

defined by 119881119901119904




119891(1198811119878 1198812119878 119881





Acknowledgment


References









































































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation










Table7Paperswith

machine

learning

metho

dsfora

ssessin

gQoE

Paper

video

sequ

ences

Resolutio

nCod

ecSSIM

VQM

PSRN

MSE

BRFR


fram

esrate)

QP

Playou

tinterrup

tions

Delay

JitterBW

MLB

SGOP

Machine

learning

Assess

MOSDMOS

Correlatio

nperfo

rmance

PCC

SROCC

RMSE

OR

Hee

tal

[17]2012

1QCI

FMPE

G4

XX

XX

X

Prob

abilistic

Neural

Network

(PNN)

Yes

09286

No

No

No

Kiplietal

[18]2012

ND

Nodata

Nodata

XX

XBP

NN

Yes

0891

No

No

No

Sing

het

al[19]

2012

4HD-720p

H264

XX

XX

RNN

Yes

No

No

037

No

Liae

tal

[20]2011

4Nodata

Nodata

BPNN

No

No

No

No

No

Khanatal

[21]2010

6QCI

FH264

XX

XANFIS

Yes

08717

No

02812

No

Duetal

[22]200

91

SDNodata

XX

XX

XX

BPNN

Yes

No

No

No

No

Piam

ratet

al[23]

2009

1Nodata

H264

XX

XPS

QAwith

RNN

Yes

No

No

No

No

Khanetal

[24]2008

3CI

FMPE

G4

XX

XX

XANFIS

Yes

R=08056

forM

OS

predicted

versus

MOS

Measure

andR=

09229

for

119876

predicted

versus119876

measure

No

RMSE

=01846

for

MOS

predicted

versus

MOS

measure

andRM

SE=006234

for119876

predicted

versus119876

measure

No

Rubino

etal[25]

2004

Only

VoIP

Nodata

Nodata

XX

RNN

No

No

No

No

No

BRbitrateFR

framerateP

LRP

acketL

ossR

ateQP


Wb

andw

idthM

LBSMeanLo

ssBu

rstS

izeGOP

Group

ofPicturesP

CCP

earson

correlationcoeffi

cientSR

OCC

Spearman

rank

orderc

orrelatio

ncoeffi

cientRM

SER

ootM

eanSquare

ErrorandOR

OutlierR

atio



NetworkBestEffort




Input parameters

RNN training script

RNN test script



MOSobj

MOSpsqaMOSp =

N

sumi=1

MOSobj

N


15 2 25 3 35 4 45 51

152

253

354

455

Desired MOS

Estim

ated

MO

S

Fit 1


MOSpsqa versus MOSp



5 Conclusion









Appendices






2(seq nt) for all


1(1) gt NFL

2(1)

NFL1(2) gt NFL

2(2) and NFL

1(3) gt NFL

2(3)


2(seq nt) for all


1(1) = NFL

2(1) =

NFL1(2) = NFL

2(2) = NFL

1(3) = NFL

2(3)





1(2) gt NFL

1(3) and

NFL2(1) gt NFL

2(2) gt NFL


asNFL1(1) gt NFL

2(1) gt NFL

1(2) gt NFL

2(2)

gt NFL1(3) gt NFL

2(3)

(A1)


2(seq nt)

QIBF1(seq nt) =

119865

sum

119891=1

1 minusNFL1(119891)

3minus 120572

QIBF2(seq nt) =

119865

sum

119891=1

1 minusNFL2(119891)

3minus 120572

(A2)

Then


1(seq nt)

= [

[

119865

sum

119891=1

1 minusNFL2(119891)

3minus 120572]

]

minus [

[

119865

sum

119891=1

1 minusNFL1(119891)

3minus 120572]

]


1(seq nt)

=

119865

sum

119891=1

1 minusNFL2(119891)

3minus

119865

sum

119891=1


3

(A3)

If NFL1(1) gt NFL

2(1) gt NFL

1(2) gt NFL

2(2) gt

NFL1(3) gt NFL



1(seq nt) larrrarr

119865

sum

119891=1


3gt

119865

sum

119891=1

1 minusNFL1(119891)

3

(A4)








2(1) = NFL

1(2) = NFL

2(2) =

NFL1(3) = NFL


1(seq nt) =














119889119894+

119873

sum

119895=1

(119901+

119894119895+ 119901plusmn

119894119895) = 1 (B1)



119894or 120582minus119894rate


119894= 0 for any neuron 119894


119873

sum

119894=1

(120582+

119894) gt 0 (B2)



119892119894= lim119905rarrinfin

119875 (119902119894(119905) gt 0) (B3)


997888997888rarr119902(119905) = 119902

1(119905) 119902





119894119895and 119875

minus

119894119895 The above


120596+

119894119895= 119877119894119875+

119894119895

120596minus

119894119895= 119877119894119875minus

119894119895

(B4)


119894119895and 120596minus(119896minus1)




(119896)


therefore (119896)




119894= 0



119894= 119889119894= 0




119878 = 1205721 1205722 120572

119878

1198781015840= 1205721015840

1 1205721015840

2 120572

1015840

119878

119875 = 1205821 1205822 120582

119899

(B5)


119878is

defined by 119881119901119904




119891(1198811119878 1198812119878 119881





Acknowledgment


References









































































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation











NetworkBestEffort




Input parameters

RNN training script

RNN test script



MOSobj

MOSpsqaMOSp =

N

sumi=1

MOSobj

N


15 2 25 3 35 4 45 51

152

253

354

455

Desired MOS

Estim

ated

MO

S

Fit 1


MOSpsqa versus MOSp



5 Conclusion









Appendices






2(seq nt) for all


1(1) gt NFL

2(1)

NFL1(2) gt NFL

2(2) and NFL

1(3) gt NFL

2(3)


2(seq nt) for all


1(1) = NFL

2(1) =

NFL1(2) = NFL

2(2) = NFL

1(3) = NFL

2(3)





1(2) gt NFL

1(3) and

NFL2(1) gt NFL

2(2) gt NFL


asNFL1(1) gt NFL

2(1) gt NFL

1(2) gt NFL

2(2)

gt NFL1(3) gt NFL

2(3)

(A1)


2(seq nt)

QIBF1(seq nt) =

119865

sum

119891=1

1 minusNFL1(119891)

3minus 120572

QIBF2(seq nt) =

119865

sum

119891=1

1 minusNFL2(119891)

3minus 120572

(A2)

Then


1(seq nt)

= [

[

119865

sum

119891=1

1 minusNFL2(119891)

3minus 120572]

]

minus [

[

119865

sum

119891=1

1 minusNFL1(119891)

3minus 120572]

]


1(seq nt)

=

119865

sum

119891=1

1 minusNFL2(119891)

3minus

119865

sum

119891=1


3

(A3)

If NFL1(1) gt NFL

2(1) gt NFL

1(2) gt NFL

2(2) gt

NFL1(3) gt NFL



1(seq nt) larrrarr

119865

sum

119891=1


3gt

119865

sum

119891=1

1 minusNFL1(119891)

3

(A4)








2(1) = NFL

1(2) = NFL

2(2) =

NFL1(3) = NFL


1(seq nt) =














119889119894+

119873

sum

119895=1

(119901+

119894119895+ 119901plusmn

119894119895) = 1 (B1)



119894or 120582minus119894rate


119894= 0 for any neuron 119894


119873

sum

119894=1

(120582+

119894) gt 0 (B2)



119892119894= lim119905rarrinfin

119875 (119902119894(119905) gt 0) (B3)


997888997888rarr119902(119905) = 119902

1(119905) 119902





119894119895and 119875

minus

119894119895 The above


120596+

119894119895= 119877119894119875+

119894119895

120596minus

119894119895= 119877119894119875minus

119894119895

(B4)


119894119895and 120596minus(119896minus1)




(119896)


therefore (119896)




119894= 0



119894= 119889119894= 0




119878 = 1205721 1205722 120572

119878

1198781015840= 1205721015840

1 1205721015840

2 120572

1015840

119878

119875 = 1205821 1205822 120582

119899

(B5)


119878is

defined by 119881119901119904




119891(1198811119878 1198812119878 119881





Acknowledgment


References









































































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation














Appendices






2(seq nt) for all


1(1) gt NFL

2(1)

NFL1(2) gt NFL

2(2) and NFL

1(3) gt NFL

2(3)


2(seq nt) for all


1(1) = NFL

2(1) =

NFL1(2) = NFL

2(2) = NFL

1(3) = NFL

2(3)





1(2) gt NFL

1(3) and

NFL2(1) gt NFL

2(2) gt NFL


asNFL1(1) gt NFL

2(1) gt NFL

1(2) gt NFL

2(2)

gt NFL1(3) gt NFL

2(3)

(A1)


2(seq nt)

QIBF1(seq nt) =

119865

sum

119891=1

1 minusNFL1(119891)

3minus 120572

QIBF2(seq nt) =

119865

sum

119891=1

1 minusNFL2(119891)

3minus 120572

(A2)

Then


1(seq nt)

= [

[

119865

sum

119891=1

1 minusNFL2(119891)

3minus 120572]

]

minus [

[

119865

sum

119891=1

1 minusNFL1(119891)

3minus 120572]

]


1(seq nt)

=

119865

sum

119891=1

1 minusNFL2(119891)

3minus

119865

sum

119891=1


3

(A3)

If NFL1(1) gt NFL

2(1) gt NFL

1(2) gt NFL

2(2) gt

NFL1(3) gt NFL



1(seq nt) larrrarr

119865

sum

119891=1


3gt

119865

sum

119891=1

1 minusNFL1(119891)

3

(A4)








2(1) = NFL

1(2) = NFL

2(2) =

NFL1(3) = NFL


1(seq nt) =














119889119894+

119873

sum

119895=1

(119901+

119894119895+ 119901plusmn

119894119895) = 1 (B1)



119894or 120582minus119894rate


119894= 0 for any neuron 119894


119873

sum

119894=1

(120582+

119894) gt 0 (B2)



119892119894= lim119905rarrinfin

119875 (119902119894(119905) gt 0) (B3)


997888997888rarr119902(119905) = 119902

1(119905) 119902





119894119895and 119875

minus

119894119895 The above


120596+

119894119895= 119877119894119875+

119894119895

120596minus

119894119895= 119877119894119875minus

119894119895

(B4)


119894119895and 120596minus(119896minus1)




(119896)


therefore (119896)




119894= 0



119894= 119889119894= 0




119878 = 1205721 1205722 120572

119878

1198781015840= 1205721015840

1 1205721015840

2 120572

1015840

119878

119875 = 1205821 1205822 120582

119899

(B5)


119878is

defined by 119881119901119904




119891(1198811119878 1198812119878 119881





Acknowledgment


References









































































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation













119889119894+

119873

sum

119895=1

(119901+

119894119895+ 119901plusmn

119894119895) = 1 (B1)



119894or 120582minus119894rate


119894= 0 for any neuron 119894


119873

sum

119894=1

(120582+

119894) gt 0 (B2)



119892119894= lim119905rarrinfin

119875 (119902119894(119905) gt 0) (B3)


997888997888rarr119902(119905) = 119902

1(119905) 119902





119894119895and 119875

minus

119894119895 The above


120596+

119894119895= 119877119894119875+

119894119895

120596minus

119894119895= 119877119894119875minus

119894119895

(B4)


119894119895and 120596minus(119896minus1)




(119896)


therefore (119896)




119894= 0



119894= 119889119894= 0




119878 = 1205721 1205722 120572

119878

1198781015840= 1205721015840

1 1205721015840

2 120572

1015840

119878

119875 = 1205821 1205822 120582

119899

(B5)


119878is

defined by 119881119901119904




119891(1198811119878 1198812119878 119881





Acknowledgment


References









































































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation










































































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation









Research Article Nonintrusive Method Based on Neural...

Documents

Transcript of Research Article Nonintrusive Method Based on Neural...