Received: 20 June 2018 Accepted: 11 July 2019

S P E C I A L I S S U E PA P ER

DOI: 10.1002/nem.2083

Quality of experience for unified communications: A survey

Jasmina Baraković Husić1,2 | Sabina Baraković3,4 | Enida Cero2,4 | Nina Slamnik1 |

Merima Oćuz5 | Azer Dedović6 | Osman Zupčić7

1Faculty of Electrical Engineering,University of Sarajevo, Sarajevo, Bosniaand Herzegovina2BH Telecom, Joint Stock Company,Sarajevo, Bosnia and Herzegovina3Faculty of Transport andCommunications, University of Sarajevo,Sarajevo, Bosnia and Herzegovina4American University in Bosnia andHerzegovina, Sarajevo, Bosnia andHerzegovina5Atlantbh d.o.o Sarajevo, Sarajevo, Bosniaand Herzegovina6Symphony Sarajevo, Sarajevo, Bosniaand Herzegovina7ZIRA Ltd., Sarajevo, Bosnia andHerzegovina

CorrespondenceJasmina Baraković Husić, Faculty ofElectrical Engineering, University ofSarajevo, Zmaja od Bosne bb, 71000Sarajevo, Bosnia and Herzegovina.Email: jbarakovic@etf.unsa.ba

Int J Network Mgmt. 2019;e2083.

https://doi.org/10.1002/nem.2083

Summary

This paper presents a quality of experience (QoE) conceptual model to the con-

text of unified communications (UC) through summary, classification, and dis-

cussion of multiple influence factors (IFs) and dimensions affecting it. A deep

and comprehensive understanding of the IFs and their impact on QoE for a

given service is an essential precondition for successful QoE management with

the overall goal of prominently optimizing end‐user QoE, while making effi-

cient use of network resources and maintaining a satisfied user base. The pro-

posed conceptual model was used to conduct a qualitative meta‐analytical

review of selected papers. The results of the qualitative review include various

IFs, QoE dimensions, and key findings in the form of research recommenda-

tions for QoE in the context of UC.

1 | INTRODUCTION

The trends and convergence of telecommunications and information technology have resulted in the appearance of uni-fied communications (UC), representing managed communication which integrates the traditional and novel commu-nication media (eg, audio, video, text) and devices (eg, smartphones, tablets, computers).1 Technological andorganizational aspects are driving the UC development. Advances in internet protocol (IP)‐based infrastructure andsoftware‐based communication media determined a technological aspect of UC development. However, specific prob-lems related to distributed communication in modern business environment (eg, increased number of possible commu-nication options or various range of available media and devices) have caused an organizational aspect of UCdevelopment. This generally led to a high communication complexity both for the receiver, managing a multitude ofcommunication devices and media, and for the sender, trying out various communication options in order to finallyreach the receiver, which resulted in a poor communication availability. In order to solve this issue and improve user'scommunication management,2 the UC service was proposed. This service provides benefit both for the individual andbusiness users, such as streamlined mechanisms for managing communications, community building, notifications,increased reach, and tying multiple devices together.3

The objective of UC is to make it easier for users to communicate anytime, anywhere, by the most appropriate com-munication channel on any device, and aided by contextual access to information. Since connectivity is increasing in

© 2019 John Wiley & Sons, Ltd.wileyonlinelibrary.com/journal/nem 1 of 25

2 of 25 BARAKOVIĆ HUSIĆ ET AL.

criticality to support explosive device growth, it is expected around 80 billion connected devices worldwide and averageof five connected devices for every user by 2020.4 The move to UC may be considered as evolutionary transition for mostbusinesses, and it is not a question of if—it is a question of how.5 The time that business users spend collaborating hasincreased by at least 50% over the past two decades, and up to 80% of their time is spent on communication activitiessuch as meetings, calls, and responding to emails.6 Therefore, it is expected that what Apple FaceTime, Google Hang-outs, and Viber have done for user's social communication, products like Skype for Business, Zoom, and Spark will dofor business communications in the future. For example, these business applications will allow features such as instantmessages anytime or meeting with up to 250 people recording. In this regard, 56% of enterprises and 66% of small andmedium‐sized enterprise (SME) organizations plan to implement or upgrade to UC solutions within the near future.7

By enabling users to communicate anytime, anywhere, and anyhow, UC will dramatically improve the lifestyle.Nowadays, and especially tomorrow, humans will be quality meters, and their expectations, perceptions, and needs withrespect to a particular product, service, or application will carry a great value.8 This shall apply to the UC as well, giventhat the increase in its usage is evident (as aforementioned), and that is why addressing the quality of experience (QoE)in that context is of great importance. Although there have been numerous attempts to define QoE, recent definitiongiven by Brunnström et al9 describes QoE as “the degree of delight or annoyance of the user of an application or service.It results from the fulfilment of his or her expectations with respect to the utility and/or enjoyment of the application or ser-vice in the light of the user's personality and current state. In the context of communication services, QoE is influenced byservice, content, device, application, and context of use.” Therefore, it is necessary to identify and understand multipleinfluence factors (IFs) and perceptual dimensions (hereinafter referred to as perceptual dimensions (PDs)) from thepoint of view of various actors in the service delivery chain and determine how they affect QoE. This is an essential pre-requisite for QoE management, which determines the parameters to be monitored and measured, and finally used todevelop, test, and implement the QoE control and optimization strategies.

Many research studies addressing different IFs that affect and describe the user QoE focus on a limited set of factors,and hence offer an incomplete view of QoE. Generally, it may be noted that a multidimensional approach to QoE man-agement is missing,10 which requires a deeper understanding of multiple IFs affecting QoE along with their mutualinterplay. Since it is clear that not all factors can be addressed in a single study, a key set of IFs should be identifiedand investigated in terms of their impact on the user rating of overall perceived QoE for a given service. Therefore, thispaper focuses on the challenge of recognizing the most relevant factors that influence QoE dimensions in the context ofUC. The popularization of UC and collaboration tools additionally increases the challenge since QoE metrics do notsimply migrate from one service to another without losing validity. Hence, the objective of this paper is to propose aQoE conceptual model in the context of UC through identification, classification, and discussion of key set of IFsaffecting various QoE PDs. This paper presents a systematic literature review, which covers references published inhigh‐quality journals and conferences during more than last 10 years. As this paper provides the comprehensive,state‐of‐the‐art survey of multiple IFs and QoE PDs for UC, it is intended to the research community and other stake-holders interested in this contemporary field.

The rest of the paper is organized as follows. Section 2 describes the research approach and design. Section 3 proposesthe QoE conceptual model for UC. A range of complex and interrelated IFs falling into three categories, ie, human, con-textual, and system IFs affecting quality of UC, are analyzed in Section 4, which is followed by the discussion on cor-responding QoE PDs. Section 6 presents the key findings of analysis and recommendations for future work. Section 7concludes the paper by pointing out the challenges and open research issues.

2 | RESEARCH APPROACH AND DESIGN

The main objectives of this paper are (1) to propose the concept for QoE modeling of UC, (2) to synthesize and organizethe literature to understand the key set of IFs affecting different PDs of QoE for UC, and (3) to recognize the researchgaps and recommend new research directions in the field of QoE modeling of UC.

The comprehensive investigation of different factors influencing QoE for UC underlines to what extent certain fac-tors and QoE dimensions have been studied in the previous works and emphasizes the possible research areas to bestudied in the future. A deep understanding of relevant IFs affecting QoE and its PDs for UC is an important prerequi-site for QoE modeling which tends to describe and quantify those relationships for a given service. Resulting QoEmodels may be further used for QoE evaluation and determination of the parameters to be monitored and measuredin order to fulfill the objective of QoE control and optimization.

BARAKOVIĆ HUSIĆ ET AL. 3 of 25

The research approach used in this paper is illustrated in Figure 1 and includes the following phases:Phase 1 included searching and extracting the papers addressing the impact of various IFs on QoE of UC. There are

many papers dealing with QoE for UC service components individually (eg, audio or video), but they were not takeninto consideration in this review. We were dealing with those components only in terms of UC. A total of 150 referenceswere selected according to their relevance to a given topic. Figure 2 presents the total number of references per publi-cation type.

Phase 2 included the categorization of papers based on the IFs affecting different PDs of QoE and QoE for UC. Paperswere firstly grouped into three categories, ie, studies considering human, context, and system IFs. Then, papers fromeach category were analyzed accordingly. Finally, papers were categorized based on corresponding QoE PDs. Thishas resulted in the proposition of the QoE conceptual model for UC.

Phase 3 involved the meta‐analytical review of 61 selected papers according to the proposed concept. The criterion forpaper selection was that it had to include the experimental study on the impact of different IFs on QoE for UC. Papersthat quantify the impact of IFs on QoE for UC were considered according to comparison criteria11 in respective tables.Other papers that qualitatively described the relations between various IFs and QoE were discussed in correspondingsubsections. Based on summarization, integration, and interpretation of selected papers, we identified the prominent

IGURE 1 Research design and approach

IGURE 2 References distribution by type of publication

F

F

4 of 25 BARAKOVIĆ HUSIĆ ET AL.

IFs and QoE PDs and recommended the ones that should be additionally considered in the QoE modeling process of UCservices.

3 | QOE CONCEPTUAL MODEL FOR UC

QoE modeling is the first step of QoE management process, which additionally involves the following steps, ie, QoEmonitoring and measurement and QoE control and optimization.12 QoE modeling provides a basis for deep and com-prehensive understanding of the IFs and multiple PDs of quality perception and how they affect QoE for future net-works and emerging services. In general, QoE modeling strives to quantify the relationship between differentmeasurable QoE IFs, quantifiable QoE PDs, and QoE for a given service.

A QoE IF has been defined as “any characteristic of a user, system, service, application, or context whose actual state orsetting may have influence on the Quality of Experience for the user.”9 Further, a feature can be seen as a dimension of amultidimensional perceptual situation‐ and context‐dependent event, in a multidimensional perceptual space. A qualityfeature has been defined as “a perceivable, recognized and nameable characteristic of the individual's experience of servicewhich contributes to its quality.”13 According to the quality formation process, feature has to be relevant for a quality tobecome the quality feature. As stated in the literature,10,14,15 QoE PDs depend on each other and may be represented ina multidimensional perceptual space.

Having in mind the definitions of QoE IFs and PDs, one may notice that the modeling of QoE as multidimensionalconcept is challenging.16 QoE modeling becomes even more challenging in future networks since key IFs and PDs ableto capture the variation of QoE differ among various services and applications such as UC, online gaming, instant mes-saging, telepresence, and many more to come within the concept of Internet of Everything (IoE). According to theauthors' best knowledge, there is no standardized QoE solution for combined services, such as UC. The complex natureof these services requires analysis of multiple media types involved in them, individually and jointly, which makes theinvestigation of their interactions and development of QoE models quite challenging.17,18

Most research studies addressing different IFs and PDs that describe QoE focus on a limited set of IFs and PDs, andhence offer an incomplete view of QoE.16 There is a lack of a multidimensional approach to QoE modeling, ie, the quan-tification and deeper understanding of multiple IFs affecting QoE and PDs describing it, together with their mutualinterplay.11 Service innovation increases the challenge since metrics do not simply migrate from one service to anotherwithout losing validity. In addition, mobility concept introduces a wide range of context IFs which are difficult to quan-tify and that dominantly modifies user's perception of product and service quality. So, it is clear that there is not a singlestudy addressing all IFs together as well as their perceptions, especially for UC as a combined service.

Therefore, on the basis of existing theoretical approach to QoE,9,11,16 we propose the multidimensional QoE conceptualmodel for UC. The concept is depicted in Figure 3 and contains three elements. First, the outer circle shows the threegroups of IFs (ie, human, contextual, system) described earlier as affecting QoE for UC. Second, the middle circle showsthe quality multiple UC service components (eg, audio, video, text, etc). Third, the inner circle presents the multipleQoE PDs for UC (eg, enjoyment, satisfaction, involvement, availability, etc). The characteristic of the middle circle is thatit is moving so one can consider and investigate different combinations of impacts of IFs on the quality of multiple UC ser-vice components and then their individual or combined influence on PDs which ultimately affect QoE for UC.

In other words, QoE which is in focus of this multidimensional model for UC service is being influenced by the per-ceptual dimensions while using UC service. Those PDs are affected by the quality of the relevant mix of different UCservice components. The quality of those components is lastly impacted by the cocktail of different IFs from aforemen-tioned three categories.

We believe that the proposed concept is multidimensional and comprehensive in a way that it encompasses all IFsand enables direct way to identify and address various QoE PDs relevant for different UC through quality of UC servicecomponents. Thus, the proposed conceptual model depicted in Figure 3, with its central focus on QoE, offers specificguidance on the implementation of any UC project along with potential outcomes. Three categories of IFs were usedfor the presentation of qualitative review in the previous studies that relates to the QoE assessment of UC. The benefitof using these IF categories for the literature review is found in both the structure they provide for the discussion tofollow, as well as to help highlight any areas that are lacking investigation. The following section will provide a quali-tative review of research studies considering QoE for UC using the proposed conceptual model to demonstrate its valid-ity. Using the outer and inner circles of the conceptual model in Figure 3, we looked at the key set of IFs and theirperceptions (QoE PDs) in collected studies analyzing QoE for UC shown in middle circle.

FIGURE 3 A proposed QoE modeling concept for UC

BARAKOVIĆ HUSIĆ ET AL. 5 of 25

4 | QOE INFLUENCE FACTORS AND QUALITY OF UC COMPONENTS

The following sets of analysis pertain to various IFs studied among the selected papers focusing on QoE of UC. We focuson identifying the factors that have been addressed in previous research and analyze their representation, while theidentification of significance and importance of different IFs in context of QoE for UC is out of the scope of this paper.In doing so, the independent variables studied are described under each of the model categories of Figure 3. The reviewresults are contained in three respective tables, which have analyzed each group of IFs (ie, human, contextual, system)as independent variables and associated QoE PDs as dependent variables. Additional table summarizes the analysisresults of mutual interplay between these three IF categories. These four review tables (Table A, Table B, Table C,and Table D) consider different UC service components in terms of addressed QoE IFs, corresponding QoE PDs, typeof task and study, used scale, conducted analysis, and key findings. Interested reader could find these four tables atthe following link: https://drive.google.com/open?id=1uWyZytYI8w3530WRUUtkEYK1pUY2q9H8.

4.1 | Human influence factors

Human IFs (HIFs) represent “any variant or invariant property or characteristic of a human user. The characteristic candescribe the demographic and socio‐economic background, the physical and mental constitution, or the user's emotionalstate.”9 These factors are divided into two subcategories16,19: (1) low‐level processing influence factors related to thephysical, emotional, and mental constitution (eg, gender, age, lower order emotions, user's mood, personality traits,motivation, attention level, etc) and (2) higher level cognitive processing influence factors (eg, socioeconomic situation,educational background, attitudes and values, expectations, needs, knowledge, previous experiences, etc).

Additionally, HIFs may be defined as “the overall assessment of human needs, feelings, performance, and intentions.”20

In this regard, they may be classified as subjective and objective factors on the basis of psychological and physiologicalfactors. The subjective factors refer to both quantitative and qualitative aspects of human needs and requirements(eg, ease of use, joy of use, usefulness, etc). By their nature, they are psychological factors that take into account humanperceptions, intentions, and needs.20,21 On the other side, the objective factors are associated on physiological (eg, brainwaves, heart rate, blood pressure, etc) and cognitive aspects (eg, memory, attention, human activity, human task perfor-mance, language, etc).

6 of 25 BARAKOVIĆ HUSIĆ ET AL.

On the basis of the foregoing, it is clear that HIFs are highly complex because of their subjectivity and relation tointernal states and processes. In addition, they are strongly interrelated and may also deeply interplay with other groupsof IFs. Therefore, the more recent research activities on HIFs have gain the popularity resulting in the increase of liter-ature and number of studies dealing with the impact of HIFs on QoE. Nevertheless, the understanding of more complexHIFs and their impact on QoE is still relatively limited due to the fact that the wide range of these factors was marginedby the research community for a long period of time.22

The importance of HIFs' impact on user satisfaction, experience, and its quality has been emphasized by severalresearch studies.20,23-40 Based on the state‐of‐the‐art literature review, the following HIFs are identified as commonlyanalyzed in research studies: gender,20,26-28,31,33-35,38 age,29-31,33 culture,23-25,34 personality,23,29 attitude,29 mood,29 emo-tions,32 and interest.34,36,37 However, these factors have been individually considered to the limited extent, and stillremain not well comprehended.

4.1.1 | Age and gender

A comparative research studies on various real‐time communication services show that there exist both age and genderdifferences in forming QoE. For example, as far as age is concerned, some studies suggest that age affects the perceivedQoE.29 However, the opposite results illustrate that user experience and usability are not affected by age.31,33 On theother hand, the research activities considering gender show that both male and female differ in their localization per-formance capabilities20,26 and emoticon use.28,38 Enjoyment, involvement, and user experience are not affected by gen-der.31,34 Nevertheless, it is shown that age and gender differences are insignificant in terms of QoE.29

4.1.2 | Culture

There is growing evidence to demonstrate that QoE is affected by the cultural background. It is observed that there is adifference in the level of attention and the kind of perception between people in Western and East Asian cultures.24 Inthis regard, Westerners have developed context‐independent and analytical visual perception with ability to observedetails, whereas Asians have a context‐dependent and holistic visual perception, sensitive to the content.25 Moreover,people in Western cultures are primarily focused on verbal communication, whereas people in Asian cultures pay moreattention to nonverbal communication. Although there are no effects of cultural background on the communicationefficiency, the differences are found in the quality of interaction between Westerns and Asians.24 Moreover, a significantdifference between these two cultural groups is found on their QoE ratings in terms of satisfaction, enjoyment, andendurability.23,34 Since the culture is a collective concept, several individual‐level traits (ie, individualism, uncertaintyavoidance, masculinity, pragmatism, and indulgence) are considered in terms of perceptual multimedia quality.23 Inthis regard, the traits of masculinity, individualism, and pragmatism are important factors affecting perceived quality,whereas uncertainty avoidance and indulgence do not impact perceived quality.

4.1.3 | Personality and attitude

Personality is a set of differences in the pattern of behavior, perceptions, and emotions of the individual in relation toother people. Personality affects the interaction performance,29 perception of quality, and overall enjoyment.23 Prefer-ences and attitudes can also be regarded as stable factors that can affect the QoE at a higher level. Attitudes have cog-nitive, affective, and motivational components and affect perceived quality and interaction performance in terms of tasksuccess and task duration.29

4.1.4 | Mood and emotions

Emotions play a major role in communication as well as in entertainment. This implies that any consumption of visualinformation besides sensations and perceptions involves emotions.40 Depending on the type of service, one or more ofthese quality dimensions may have more weight. At the level of human affective state, the influence of mood and emo-tion on QoE has a growing interest in research, and has been analyzed in the previous studies.29,32 In addition, both are

BARAKOVIĆ HUSIĆ ET AL. 7 of 25

characterized by very short duration, ie, mood (range from 1 h to 1 day) usually takes longer than emotions (range fromseconds to minutes). Emotions represent an immediate response, and they are linked to a particular object or event.16

4.1.5 | Interest

Some research studies indicate that personal interest in video content significantly affects user's QoE judgment.36 More-over, it is shown that users tend to evaluate a video with the same bitrate as higher in QoE when they are more inter-ested in the content of the video.37

4.2 | Contextual influence factors

Contextual IFs (CIFs) “embrace any situational property to describe the user's environment in terms of physical, temporal,social, economic, task, and technical characteristics.”9 As such, they represent the general framework for detailed discus-sion on various IFs that are considered contextual. It means that there is no strict rule of what can be considered CIF, sothe task of creating simple and reasonable overview of topic papers is extremely difficult and challenging task.

The importance of CIFs' impact on user satisfaction, experience, and its quality has been emphasized by severalresearch studies.20,26,30,31,33,34,41-61 Based on the state‐of‐the‐art literature review, the following CIFs are identified ascommonly analyzed in research studies: physical context,20,26,34,42-44,46-55,60 temporal context,31,42,56,58,59,62-72 social con-text,34,44,68,70,73-75 task context,41,42,45,76 technical and information context,30,33,42,60,77 and economic context.78-80 How-ever, these factors have been individually analyzed only in a few studies and are not fully understood.

4.2.1 | Physical context

Physical context describes characteristics of three environmental contexts. The first context includes indoor (eg, home,office, school, café) or outdoor (eg, bus, train, station, etc) locations. The second context refers to environment attributes(eg, surrounding noise, light, temperature, etc). The third context includes movements (eg, sitting, standing, etc) andmobility parameters (eg, walking, running, etc).

Location context analysis has shown that subjects at public crowd (outdoor) are less tolerant to the quality degrada-tions than the subjects in the laboratory (indoor).44 QoE of video streaming is better in indoor than outdoor locations,even though two‐dimensional (2D) video shows better performance than three‐dimensional (3D) video regardless ofphysical location (ie, laboratory, simulated home, local bus, and café). These results are justified by finding that locationcontext affects the attention of the user.60 Location changes lead to a change of network conditions, which can cause asignificant variation of user experience. For example, shifting from high‐speed wireless fidelity (WiFi) network at hometo low‐speed third generation (3G) network outside the home has been proved to be the reason of QoE variation becauseof the longer video loading time.46 Apart from video streaming, impact of location context on QoE has been analyzed interms of internet protocol television (IPTV),47 surveillance videos,48 3D television content,42 and music.49-51 Althoughlocation context does not have a main effect on the user acceptance or satisfaction evaluation, it influences both enter-tainment and information recognition. The best results are generated in cafe, while the worst ones in the bus, wherenoise, social, audio, visual, and other distractions, are at the highest level.42

Environment attributes encompass light, sound, haptic and multimodal environment, temperature, weather condi-tions, and humidity, but there are many more attributes one can come across while investigating physical context.52

Light and noise affect the user's watching and listening experience when using mobile video, voice over internet proto-col (VoIP), or multiplayer games.46 Further on, seating position (eg, viewing conditions and viewing height), lightningconditions, and environmental disturbances (eg, incoming phone calls or short message service (SMS) alerts) may affectthe user experience.34 A shorter viewing distance increases the field of view and makes viewer more involved with thecontent, but may make artifacts better visible as well.34 Noisy surrounding requires loud and error‐free audio playbackin less extreme environments (eg, bus station over bus drive). Environmental attributes are less obstructive to users, butaudio content is the most affected content type. Even cafés and similar calm surroundings are described inappropriatefor listening and viewing.42 Analysis of size of virtual acoustic room has shown that medium size room resulted in bestperceived quality.26 This could be related to performance of gender in spatial and localization abilities. In instant mes-saging (IM), text size should be larger in real‐life conditions compared with laboratory experiments.42

8 of 25 BARAKOVIĆ HUSIĆ ET AL.

Movements and mobility parameters53 have been varied and analyzed as a part of physical context. Viewer's sittingposition and mobility alter the QoE evaluation.34 In case of global mobility, higher number of adjustments occurs inreal‐life environment comparing with laboratory when viewing videos with different content.42 According to Ickinet al,54 dominant mobility levels when using UC are “sitting,” “standing,” and “other,” specified by the user preciselyas “bed.” Low QoE ratings occur mostly when user is at home or school while being alone and sitting. Audio contentquality is mostly analyzed through music recommendation systems, where authors recommend music genres associat-ing them with different movements, such as walking, running, sleeping, relaxing, and shopping.55

4.2.2 | Temporal context

Temporal context consists of many temporal parameters, such are time of day, time of the week, year, etc., as well asduration of the content, frequency of use, before/during/after experience, actions related to time, synchronism, andmany others. This context has been analyzed in combination with other contextual factors in Jumisko‐Pyykkö andUtriainen.42 Even some papers highlight the time of experiment (during macrobreaks to fill extra time in previous stud-ies42,62), duration of experiment (from a couple of minutes up to 40 min, typically 10‐15 minutes56,63,64), or time of a day(the prime time is in the early morning, during lunch, and early in the evening, before dinner time65,66), the clear influ-ence of temporal context on perceived quality or other PDs has not been sufficiently discussed so far. It is shown thatSkype usage frequency does not impact the perceived quality.31 Video is most frequently used in the morning, in theevening before going to sleep,70 or during lunch time in the early evening.42 Music recommendations differ in morning,afternoon, and evening.67 Further on, movie recommendations depend on a time when movie will be seen (eg, week-ends or weekdays, in the morning/afternoon/evening, during the opening night, etc).68

Since QoE of both long‐ and short‐duration audio‐video contents is equally evaluated, authors prefer to use short‐duration contents.42-56,62-71,81-86 The chosen duration of the video can depend on user's available time, because they con-sider locations such as bus appropriate for watching videos if longer traveling occurs instead of a short local journey ontrembling roads.42 This implies that the impact of duration on perceived quality of UC should be considered. Further on,importance of temporal context was highlighted in Vandenbroucke et al,72 where authors shown that momentary QoEwas mostly dependent on spatial and social context, as well as technical distortions determined by device‐related andnetwork‐related context. Even though some of the temporal contexts are logical and seem to be important, they arenot enough experimentally investigated due to their complexity or because they have not been recognized as importantcontext for UC.

4.2.3 | Social context

Social context represents interpersonal relations existing during the experience.16 Hence, the most important is to con-sider if user consumes a content alone or in a group and how other people affect the experience with their actions. It isshown that coviewing videos with friends increases the user's level of enjoyment and enhances the endurability of theexperience, but it has no significant effect on involvement.34 Some content, such as mobile video, is more appropriatefor one person viewing, and is used to minimize solitude, avoid social engagement, and create private space.74 Thus,mobile video is watched alone in 80% of cases.70 Different experiment setups (eg, laboratory vs crowded place) can beconsidered as social contexts since users tend to be more sensitive to artifacts in laboratory conditions than in crowdedplaces.44 Other interesting aspects of group and stand‐alone using of different applications and services can be found inthe previous studies.68,73,75 Moreover, there are many other social contexts that can be analyzed, such as entertainment,type of event (a meeting, leisure time, traveling, etc), or cultural/educational/professional level. Users highly rank appli-cations that support their lifestyle choices (eg, sports, fashion, nutrition or leisure).54

4.2.4 | Task context

Task context is defined by the nature of experience, which can be perceived in single tasking or multitasking situations.Variation in task complexity along with the environment nature has no significant on acceptance or satisfaction withthe perceived quality.42 Multitasking may lower the user execrations and perceivable distortions with interfere of othertasks.76 Therefore, greater focus on audio or video component during videoconference call affects the quality perceived

BARAKOVIĆ HUSIĆ ET AL. 9 of 25

by user.45 Furthermore, interactivity can be considered as an additive IF able to increase the overall quality of the videoexperiences.57 However, it is shown that interactivity context does not affect either video or audio quality.61 In addition,quality variation shrank in passing from a passive to an interactive context.41

4.2.5 | Technical and information context

Technical and information context describes the relationship between different systems and services including devices,applications, networks, or additional informational artifacts.77 Presentation mode affects satisfaction and accep-tance,42,60 and in addition to modality30,33 can be viewed as an example of technical context. It has been shown thatsatisfaction in 2D is greater than in 3D presentation mode regardless of bitrate. The availability of these technicaland information IFs will help to improve the user's QoE.

4.2.6 | Economic context

Economic context includes costs, subscription type, or brand of the system/service. It is shown that the brand of mobiledevices has a significant impact on the QoE of mobile video streaming.78 The influences of quality, payment, and con-tent choices on QoE were investigated in Sackl et al.79 The authors found a positive feedback of QoE when the userswere free to choose video contents based on their own preferences. The importance of service cost on the quality per-ception is also reported in Shaikh et al.80 Cost of applications and data usage prohibit user from experiencing differentmobile applications. Thus, importance of aforementioned economic CIFs should be examined in terms of UC service.

4.3 | System influence factors

System IFs (SIFs) include “properties and characteristics that determine the technically produced quality of anapplication or service. They are related to media capture, coding, transmission, storage, rendering, andreproduction/display, as well as to the communication of information itself from content production to user.”9 SIFscan divided into four subgroups.9 Their importance has been emphasized by several research studies: content‐relatedSIFs,16,20,21,34,37,42,58,59,71,81,84,87-100 media‐related SIFs,20,23,31,34-37,41,42,44,45,56,60,69,81-84,86,96-128 network‐relatedSIFs,11,20,31,36,41,42,44,45,54,56,57,77,80,85,86,93,96,98,100,107,109,111-114,118,126,129-144 and device‐related SIFs.16,46,75,77,84,97,111,128,145

4.3.1 | Content‐related system influence factors

Content‐related SIFs refer to the content type (ie, color depth, texture, temporal or spatial requirements, etc) and con-tent reliability.9

AudioIn case of audio, there are voice/spoken content (relevant for UC) and music content (irrelevant for UC). Content‐related SIFs relevant for voice/spoken content are audio bandwidth and dynamic range.16 These factors should becorrelated with four perceptual dimensions relevant for speech quality,87 ie, directness/frequency content, continuity,noisiness, and loudness.

VideoIn case of video, the amount of detail and amount of motion are important for quality perception,16,88 whereas amountof depth is important for the viewing comfort.89 Since video is a set of images, perceptual image features, such as colorsaturation, brightness, amount of details,90,91 and sharpness,92 were found to contribute to the final quality assess-ment.21,91,93 According to Atzori et al,21 distortions visibility depends on spatial details and amount of motion in thecontent.42 Another important IF is amount of depth analyzed in the previous studies.42,71 There exits correlationbetween perceived quality and depth and between depth and visual comfort.94 Video modification in time or spacedomain (ie, change of frame per second rate and frame resolution) strongly affects the QoE.81 Further on, integralinfluence of visual spatial details, visual temporal details, and amount of depth on audio‐visual quality were analyzed

10 of 25 BARAKOVIĆ HUSIĆ ET AL.

in different physical locations.42 It was found out that in case of strong movement, it is hard to detect the details of theimage, but if audio is functioning properly, then video can be followed at an acceptable level.

TextImportant IFs that affect IM are grammar, spelling, way of expression (concisely, precisely, extensively, etc), line char-acter sticker use, abbreviations, and humor. It was found that in case of IM with a longstanding communication part-ner, one is more likely to use abbreviations. Moreover, IM was perceived to be more effective for conveyancecommunication (both production and social).95

4.3.2 | Media‐related system influence factors

Media‐related SIFs refer to media configuration factors.9 These factors include frame rate (FR), resolution, bitrate, sam-pling rate, type of codec, group of picture (GoP) size, and media synchronization.

Frame rateFR refers to the number of frames displayed by a viewing device in 1 second.101 Increase in FR results in better per-ceived quality102-105 and enjoyment.69 A constant FR video has a better quality than a video with FR variation.104 Fur-thermore, the increase in FR results in higher perceived quality in videos with low to moderate object motions101

compared with videos with full of extremely high motions. This can be explained by the fact that human visual systemis limited and cannot perceive more information from the scene101 by increasing FR of full motion videos.96 In addition,reduction of FR and resolution leads to lower perceived video quality.81 In that case, video quality rating depends onvideo content type. Also, authors in Ljubojević et al102 claim that QoE can be improved by the proper selection of tech-nical video characteristics values. In case of low bitrate video, the perceived video quality can be improved byincrementing FR.97 It has been stated that the MOS reduces as the FR decreases and quantization step (QS) increases.Moreover, increasing QS is preferred than decreasing FR to achieve the least quality degradation.104 It has been shownthat gaze pattern is significantly affected by the FR dispersion.106 In general, conservative critical value for the videoconferencing applications is 10 fps, and quantization step size is 30.107 Dependence of perceived quality on the FRcan be explained by exponential falling function.108

ResolutionThe resolution represents the number of particular pixels in each dimension that can be displayed and perceived by theend user. Increase of resolution leads to increased perceived quality.81,102 Authors in Zinner et al98 used structural sim-ilarity (SSIM) and video quality metric (VQM) to quantify the behavior of different content on the QoE. They found outthat the quality degradation in terms of VQM is smaller for lower resolutions than for lower frame rates, irrespective ofthe interpolation mechanism. In a combination with loss rate and error concealment, resolution has lower impact onperceived quality.44

BitrateBitrate is defined as number of transmitted or processed bits per unit of time. In addition to bitrate, it is important toinvestigate the impact of magnitude and frequency of rate change on QoE.109 Perceived video quality is higher forhigher bitrates,34,97,110 but the strength of the bitrate's impact depends not only on network conditions but also on audioand video content.110 In order to achieve the same quality, higher bitrates are needed for larger displays.97 In particular,if the bitrate is low, its influence on QoE is highly determined by media content.110 In audio‐visual system, the audiobitrate is significantly more influential compared with video bitrate.99 The dependence of user's perceived quality onbitrate is strongly logarithmic in the case of Skype call.109

Sampling rateThe importance of sampling rate simply lies in analog to digital conversion, since it determines the number of samplesper second needed for digital system to record the signal. It has been shown that perceived quality depends of samplingdensity of the signal.111 Since this is codec parameters, researchers prefer to test different codec than different codecparameters.

BARAKOVIĆ HUSIĆ ET AL. 11 of 25

Codec typeThe codec can be represented as an integrated circuit or a chip that performs data conversion (encoding and decoding).Better QoE is usually achieved if wideband codecs are supported over the complete transmission chain in case of VoIPcalls.109 In this regard, SILK represents an ultrawideband modern voice codec, which represents its superiority over anyother codec.109,112 In VoIP calls, user satisfaction index depends on call duration and user interactivities,113 since stron-ger degradation occurs in case of longer speech transmission.112 Adaptive multirate (AMR) codec is the best to utilizethe resources guaranteeing a good subjective audiovisual quality.99 Thai users found no significant difference betweenthree of these tested codecs,114 but they give a slight preference for G.722 over G.711A‐law and G.729 codecs. In case ofvideo codec, important parameter is GoP size, which consists of group of successive I, P, and B frames.96,115 Differentvideo codecs have different GoP size,70,110 and GoP structure affects video bitrate.81,116 Larger GoP size and smallerchunk size are better for perceived quality.117 Latency, deadlines, and bitrate affect chunk loss ratio and reflect high cor-relation with perceived quality.117 Additional research needs to be done based on the comparison of GoP structure ofvideo codecs.118 Since video codec affects the perceived quality, a variety of video coding standards have been analyzedin this context, such as H.264/advanced video coding (AVC) or H.265/high efficiency video coding (HEVC).119-121

Media synchronizationMultimedia synchronization consists of definition and establishment of temporal relationships among audio, video, andother data.122 Intramedia synchronization issue is solved, and no further research has been carried out for the lastdecade.123 However, inter‐media synchronization solutions are challenging to be compared qualitatively, since theyare application specific and have been evaluated subjectively.123 When it comes to mulsemedia (that takes into accountnot only the traditional media components such as audio, video, text, and their combination but also media objectswhich target other human senses), the results provided by Yuan et al124 suggest that users still reflect the positive atti-tudes even if the skew is present. It does not decrease their enjoyment and only affects the sense of reality of a certainservice. The area of noticeable synchronization errors spans a skew between 0 and 5 seconds when haptic and airflowmedia are ahead of the video, respectively. Some authors claim that synchronization errors do not have a significantimpact on the general perceived quality.125 Users' opinions reflected positive attitudes about the enjoyment of the mul-timedia experience, as well as the sense of the olfactory media relevance and reality both in the absence and presence ofintermedia skew.125

4.3.3 | Network‐related system influence factors

Network‐related SIFs refer to data transmission over a network.9 These IFs are usually related to basic set of quality ofservice (QoS) parameters, such as delay, jitter, packet loss, bandwidth, throughput, and error probability.109,145 Theycannot be straightforwardly translated into QoE.145

DelayThe QoS for VoIP is predominantly measured by the delay, jitter, and packet loss rate.31 Packet delay dramatically affectthe quality of VoIP call.86 It is an important factor that influences the QoE in terms of MOS.114 In case of video services,users rated videos with delays above 8 milliseconds as “bad.”93 Examples of acceptable and nonacceptable delay valuesfor audio and video services are given in Schmitt et al.129 In conversation analysis, turn‐taking and role theory detailedanalysis of delay influence on QoE was done, concluding that the actual interactivity pattern of each participant in theconversation results on different noticeability thresholds of delays.129 End‐to‐end delays are important to facilitate net-work designs optimized for QoS. Measuring and analyzing actual user experience are critical to close the loop for QoEcentric network designs.109

JitterHigh jitter may cause significant degradation of perceived quality of audio signal.130 In case of audio‐video communi-cation, predicted quality decreases with the increase of the jitter.118,126

Packet lossSmall increment in the packet loss rate causes lower MOS ratings of audio.130 Analysis of different applications(ie, Skype, MSN Messenger, AIM) showed that the best Skype's audio quality can be achieved under different loss rates,

12 of 25 BARAKOVIĆ HUSIĆ ET AL.

whereas AIM performs the best in the nonloss and bandwidth limited scenarios.131 In audio‐video communication, pre-dicted quality decreases with the increase of the packet loss levels.118,126 In addition, lower audio quality results in lesssimilarity between the perceived video and audio quality ratings.132 Perceived difference in video quality follows the per-ception of audio quality, although the video quality was never altered.132

ThroughputIncrease in throughput results in increase of MOS values and confidence interval as shown in Wang et al.54 This impliesthat there are many different throughput ranges resulting in the same MOS level. Network throughput based videoadaptation without considering a user's QoE could result in poor video QoE or wastage of bandwidth.111 If the offeredend‐to‐end throughput is not sufficient for playing back the video file in maximum quality, it is possible to reduce thedelivered frame rate, image quality or resolution.98 In light of the above discussion, Microsoft UC's recommendation isto right‐provision all links to the throughput of 45 kbps per stream for audio (and 300 kbps per stream for video ifenabled) for busy hour traffic.133

BandwidthAll video delivery methods are bandwidth or storage constrained in some way.36 Thus, some research studies are tryingto optimize QoE in limited bandwidth networks.107 In Sackl et al,134 authors present the results of an empirical web‐QoE user study which provides insights into the impact of a single outage (ie, zero network bandwidth) events on sub-jective quality perception. In Chen et al,131 authors change the network bandwidth and test IM applications includingSkype, MSN Messenger, and AIM and conclude that AIM has the best performance in bandwidth‐limited scenarios.

Other network‐related SIF that affects the perceived quality of video is frame freezing/dropping. User's visual discom-fort when frame freezing/dropping occurs also depends on content type.100 Correlation between different quality ratingsand network QoS classes does not always bear a close coherent relationship.135 Many QoE models are proposed basedon QoS measured from network impairments,136 such as adaptation schemes for video.96 Existing QoS metrics, such aspacket loss rate, packet delay rate, packet jitter rate, and throughput, are typically used to indicate the impact on thevideo quality level from the network's point of view, but do not reflect the user's perception.137 Consequently, theseQoS parameters fail in capturing subjective aspects associated with human experience. The impact of network‐relatedSIFs can be easily modeled because of high availability and measurability of network parameters.77 Having in mind thatQoS relies on analytic approaches and empirical or simulative measurements, it has been extensively studied andmodeled.138-140 Different QoE estimation methods for voice and data services using various parameters for characteriz-ing network quality (ie, bit rate, loss rate, and bandwidth) are presented in Reichl et al.141 Mathematical low‐complexity, no‐reference method that performs real‐time estimation of QoE for Opus‐based voice services was describedin Orosz et al.130 Further on, QoE‐aware real‐time multimedia management (QoE2M), which provides end‐to‐end qual-ity control of real‐time multimedia applications over heterogeneous networks, is presented in Mu et al.142

Network‐related SIFs have been extensively studied. Thus, their impact on QoS is well known, but it cannot be directlyrelated to QoE. Obviously, generic QoS problems (eg, loss, delay, jitter, reordering, and throughput limitations) implygeneric QoE problems (eg, glitches, artifacts, excessive waiting times).85 Moving towards the fifth generation (5G) net-works will enable high QoS for UC and remove these generic QoE problems. High QoS will be accomplished with mini-mum peak data of 10 Gbps, “zero latency,” reliability of 99.999%, and the perception of 100% coverage.143 Since today'sresearch in this field is focused on mapping between QoS and QoE,11,80,130 given the fact that QoE for UC is not yet wellstudied, mappings are even harder to accomplish. Therefore, future research should cover the exact mappings betweenQoS and different QoE dimensions, whereas QoE design should be based on context aware communication.144

4.3.4 | Device‐related system influence factors

Device‐related SIFs refer to the end systems or devices involved along the end‐to‐end communication path includingincludes equipment specifications, device capabilities, system specifications, as well as provider specification and capa-bilities.9 Examples of device‐related SIFs are16 display, resolution, colors, brightness, audio channel count, etc.

Equipment specificationsEquipment specifications refer to type/complexity/usability, ergonomic aspects, and mobility. User's perception is influ-enced by the device.10,75,145 It has been shown that users give better MOS rates to their favorite brand devices,127 even

BARAKOVIĆ HUSIĆ ET AL. 13 of 25

though they have worse specifications and performance. Interaction between media content and the device on which itwas played was shown to have a statistically significant impact on user subjective satisfaction.84 Mobile devices will bethe key equipment used for UC, so the aim is to achieve the acceptable quality for any UC service component over thesedevices.

Device capabilitiesDevice capabilities are category that consider many aspects of the devices: mainly their hardware and computationalparameters, such as display size, screen resolution, color depth, user interface capabilities, loudspeakers, headphones,luminance, audio loudness, computational power, memory, and battery lifetime. It seems that for the most of contenttypes, display size does not affect QoE, since users can adjust their viewing distance.111 Larger displays require higherbitrates in order to produce the same video quality.97 High‐quality video sequences displayed on small devices are sub-jectively rated just as highly as on large high‐definition televisions.128

System specificationsSystem specifications refer to interoperability, personalization, security, and privacy. Through user experience (UX)framework analysis of video content, it is determined that usability and interactivity are directly connected to customer'suse of service.46

Provider specification and capabilitiesProvider specification and capabilities refer to server performance and availability. Quality of multimedia service deliv-ery can be much higher if content server is aware which type of device requests the audio‐visual content.128 Althoughthe content provider's specifications and capabilities are also device‐related, their impacts cannot be directly evaluatedthrough content‐, media‐ or network‐related SIFs. Therefore, the studies of device‐related SIFs are mainly conducted atmobile terminals.77

5 | QOE PERCEPTUAL DIMENSIONS

Having reviewed QoE IFs, one may conclude that developing a model for multidimensional concept such as QoE rep-resents a challenge. QoE for UC is a fast emerging multidisciplinary field that relies on social psychology, cognitive sci-ence, and engineering science focused on understanding overall human quality requirements.4 According tononexhaustive literature review and ITU‐T Recommendation G.1091, there is a broad range of IFs and PDs affectingQoE for UC, indicating that QoE should be considered as multidimensional concept if aiming to gain understandingthat is more comprehensive. However, previous research studies have neglected to consider the simultaneous impactsof various IFs and PDs on the QoE for UC, which can be explained by the complexity that such studies incur. Moreover,a number of studies20,23,26,30,31,33-37,41,42,44,45,56-60,111-113 have simultaneously considered multiple IFs and QoE PDsaffecting the UC, whereas there is a smaller number of studies20,37,41,45,56,57,112,113 that have resulted in quantifyingand modeling the relations between them.

The perceived quality is commonly analyzed in terms of QoE PD that is affected by many relevant IFs.26,36,37,44,45,57,58

Several research studies have considered the perceived quality along with other PDs that contribute to overall QoE forUC: enjoyment,23,34 satisfaction, involvement, and endurability,34 as well as availability.20 Key results have shown thatonly small number of SIFs have statistically significant impact on enjoyment,23 which is along with endurability affectedby gender, culture, and social context.34 Satisfaction is positively correlated with user interest and culture, while involve-ment differs according to gender and video genre.34 Finally, availability is affected by change of gender and virtual roomsize.20

In the context of video service component, authors in Jumisko‐Pyykkö and Utriainen42 used a multidimensionalapproach to QoE by focusing on studying several PDs: acceptance, satisfaction, entertainment, and information assim-ilation. Summarizing key findings one may conclude that context of use does not affect the acceptance or satisfaction,but influence both the entertainment and information assimilation. The most entertaining quality is provided in thecafé and station contexts compared with the bus context, whereas information assimilation in the café context is higherthan in any other contexts. Further, acceptance and satisfaction are slightly higher in the controlled laboratory than inthe home‐like context. On the contrary, in the context of audio service component, authors in Naumann et al30 consid-ered multidimensional approach to QoE by investigating the following PDs: effectiveness, efficiency, satisfaction, andmental effort. Key results have shown that multimodality is superior to the best of the single modalities speech and

14 of 25 BARAKOVIĆ HUSIĆ ET AL.

motion control, and slightly better than touch modality in terms of effectiveness, efficiency, and satisfaction. Addition-ally, modality affects mental effort.

However, although that is the case with IFs which are discussed above, in order to have multidimensional approachto QoE in terms of UC and to understand and manage it more effectively, the research community also needs to singleout the dimensions of QoE that contribute to it and describe it in the best way. In accordance with the proposed con-ceptual model, there is a need to find perceptual dimensions that explain QoE which are the result of certain combina-tion of UC components whose quality is affected by a cocktail of IFs. This requires additional research studies thatwould deal with investigating perceptual dimensions affecting QoE and extracting the most prominent ones, given thatthis study only recognizes the most represented ones in the literature.

6 | META ‐ANALYTICAL REVIEW OF QOE STUDIES

The literature review of research on QoE for UC service components performed so far is contained in respectivetables (Table A, Table B, Table C, and Table D) available at the following https://drive.google.com/open?id=1uWyZytYI8w3530WRUUtkEYK1pUY2q9H8. Overall studies considering QoE of UC have been focused on investigat-ing HIFs (26.229%), followed by CIFs (32.787%) and SIFs (83.607%). (Note: distribution exceeds 100% as multiple areasmay have been studied in a single study.) Table 1 shows a percentage of representation of each IFs and QoE PDs used inthe reviewed studies.

HIF analysis is performed based on 6 papers, which treat their individual impact on QoE for UC (Table A, availableat: https://drive.google.com/open?id=1HMNqy2GWHAFtxmL6Bp73oDw97OtllOxP), and 10 papers that consider theirinterplay with other IFs (Table D, available at: https://drive.google.com/open?id=1UQZ3GA2skiRqc‐F‐p3PBQWud_90IMVym). The most prominent HIF studied in terms of QoE for UC was gender (14.754%). Age (6.557%), interest(4.918%), and culture (4.918%) were also analyzed in selected papers. The effect of personality was explored in 3.279%of papers studying the impact of HIFs on QoE for UC. Attitude, mood, and emotions were investigated in just 1.639%of reviewed papers. With only 26.229% of papers being the case, there is limited research considering the interest asone of multiple HIFs. None of the reviewed papers studied the role of physical disabilities, prior experience, user rou-tine, and lifestyle on QoE. It becomes apparent from these statistics that existing research activities have been limited inthe range and frequency of HIFs studied. Thus, there still exists the need and corresponding opportunities for studyingthe impact of HIFs on multiple PDs of QoE for UC.

CIF analysis is conducted based on 2 papers that analyze their individual impact on QoE for UC (Table B, availableat: https://drive.google.com/open?id=1h2H23Eq1jwLPdMpdlNlMkCGtOy1NLpHo), and 18 papers that consider inter-play between CIFs and other IFs (Table D, available at: https://drive.google.com/open?id=1UQZ3GA2skiRqc‐F‐p3PBQWud_90IMVym). Most of the analyzed papers consider the influence of physical context (11.475%), which mainlyinvolves location type, viewing conditions, and room size. Temporal context (9.836%) was studied in terms of usage fre-quency, duration, and startup time. Task context (6.557%) was analyzed in terms of interactivity and type of task,whereas technical and information context (6.557%) in terms of modality preferences and presentation mode. Less stud-ied IFs refer to social context (1.639%). However, there is a need for a deeper analysis of the following CIFs: moving andmobility, spatial characteristics (eg, brightness, surrounding noise, light, temperature), frequency of use, price, actionsrelated to time and synchronism, as well as different technical and information contexts.

SIF analysis is based on 31 papers that consider their impact on QoE for UC (Table C, available at: https://drive.goo-gle.com/open?id=14Co7hIVJE5NJ9hUZWak23oHrVH3CNl40), and 19 papers that analyze their impact in combinationwith HIFs and/or CIFs (Table D, available at: https://drive.google.com/open?id=1UQZ3GA2skiRqc‐F‐p3PBQWud_90IMVym). The most analyzed SIF was bitrate (32.787%), followed by frame rate (22.951%), content type (22.951%),and packet loss (19.672%). This leads us to the conclusion that the most analyzed of all SIFs are media‐ and content‐related SIFs. Less analyzed were resolution and codec (11.475%). Jitter, quantization level, and device type/model wereconsidered in 6.557% of selected papers, and followed by delay, packet error rate, and spatial degradations, which areanalyzed in 4.918% of reviewed papers. The percentage of other SIFs mentioned in this review is given in Table 1.

Based on the aforementioned discussion, one may notice the lack of analysis of the following network‐related SIFs.These are related to the physical phenomena of wireless/wired channel, such as variability (affected by noise, fading,and interference), reliability (interception, security issues), capacity (speed, coverage, limited bandwidth, sharedresources), channel sharing among users, signal strength (affected by the temperature, humidity, distance from theantenna, base station antenna gain), signaling traffic overload, and handover. Furthermore, there is a need for deeper

TABLE 1 Percentage of QoE IFs and QoE PDs in the reviewed studies

Category QoE IF/PD Reference Percentage, %

Human IF Gender 20,26-28,31,33-35,38 14.754Age 29-31,33 6.557Interest 34,36,37 4.918Culture 23,25,34 4.918Personality 23,29 3.279Attitude 29 1.639Mood 29 1.639Emotions 32 1.639

Contextual IF Physical context 20,26,45-47,106,146 11.475Temporal context 31,67,107,108,145,147 9.836Technical and informationcontext

30,33,45,146 6.557

Task context 44,48,144,148 6.557Social context 34 1.639

System IF Bitrate 20,23,31,34,36,37,44,45,48,51,67,97,98,105,108,112,124,129,142,146 32.787Frame rate 23,45,49,52,92-95,97-101,143 22.951Content type 20,34,37,45,52,67,90,97,98,102,105,129,145,147 22.951Packet loss 20,44,47,48,51,107,109,113,123-125,130 19.672Resolution 37,47,49,93,102,106,124 11.475Codec 48,72,105,107,109,113,124 11.475Jitter 108,113,123,124 6.557Quantization level 92,95,145,147 6.557Device model/type 52,67,142,143 6.557Delay 20,109,130 4.918Packet error rate 44,45,97 4.918Spatial degradation 49,50,92 4.918Inter‐media skew variation 119,120 3.279Bandwidth 125,130 3.279Frame size 23,92 3.279Display size 98,106 3.279Compression levels 35,100 3.279Temporal information 49 1.639Timbral degradation 50 1.639Scaling 102 1.639Chunk size 112 1.639Encoding (GoP) 112 1.639Queuing disciplines 124 1.639Audio‐video codec combination 124 1.639Rate magnitude 103 1.639Rate frequency 103 1.639Bulk loss 107 1.639Quality degradation 126 1.639Video presentation 51 1.639Video freezing/dropping 129 1.639Brightness 106 1.639Packet reorder 20 1.639Listening mode 31 1.639Error concealment 47 1.639Throughput 67 1.639Buffering ratio 67 1.639Starvation errors 144 1.639

QoE PD Perceived quality 20,23,26,29,31,32,34,36,37,44,46-52,72,90,92-95,97,98,100-

102,105,106,112,113,119,123,124,126,129,130,142-145,14870.492

Satisfaction 30,34,45,46,67,108,125,146 13.115

(Continues)

BARAKOVIĆ HUSIĆ ET AL. 15 of 25

TABLE 1 (Continued)

Category QoE IF/PD Reference Percentage, %

Emotional properties 28,38 3.279Enjoyment 23,34 3.279Visual comfort/attention 90,99 3.279User experience 31,103 3.279Estimated quality 107,109 3.279Efficiency 25,30 3.279Performance 25,29 3.279Quality of interaction 25 1.639Persuasion 25 1.639Sensorial abilities 27 1.639Modality choice 29 1.639Spatial quality 46 1.639Temporal quality 46 1.639Focus level 46 1.639Depth perception 90 1.639Involvement 34 1.639Endurability 34 1.639Availability 20 1.639Quality of acceptance 45 1.639Entertainment 45 1.639Information assimilation 45 1.639Usability 33 1.639Effectiveness 30 1.639Mental effort 30 1.639Quality expectations 147 1.639Perception of thesynchronization effect

120 1.639

Perception of time‐compressedspeech

35 1.639

Abbreviations: ICT, information communication technology; IF, influence factor; PD, perceptual dimension; QoE, quality of experience.

16 of 25 BARAKOVIĆ HUSIĆ ET AL.

understanding of the following media‐related SIFs. These are related to adaptation capabilities (eg, capability to adaptvarious application parameters to fit the device, network, and usage context constraints, etc), synchronization and par-allelism capabilities (eg, capability to synchronize various parallel UC service components), adjustment to the devicepower consumption, data access, offline capabilities, transparent synchronization with backend systems, security issues,and lack of add‐ons. In addition, there is a need for further consideration of content‐related SIFs. Therefore, additionalresearch activities should include the analysis of IM content or its interaction with other UC service components, suchas audio or video. Furthermore, device‐related SIFs should be additionally analyzed in terms of screen responsiveness,keyboard size and responsiveness, microphone quality, speakers/headset quality, battery lifetime‐energy consumption,computational power/resources, storage capacity, signaling traffic overload, operating system, and usability.

QoE PDs measured in all reviewed studies are summarized at the bottom of Table 1. Upon review of their relativeappearance in the reviewed literature, two core QoE PDs appear to be the following: perceived quality (70.492%) andsatisfaction (13.115%). The remaining QoE PDs are analyzed in 3.279% of selected papers and include emotional prop-erties, enjoyment, visual comfort/attention, user experience, estimated quality, efficiency, and performance. The afore-mentioned findings are arguably neither surprising nor favorable for the field, as these QoE PDs have been set as thestandard for more than a decade, regardless of significant technology advances and use scenarios. However, since UCis allowing the connecting living that blurs the line between work and personal life, the IFs and PDs studied for QoEfor UC may be revisited before too long. For example, less investigated QoE PDs (1.639%) include endurability, involve-ment, acceptance, entertainment, information assimilation, effectiveness, efficiency, mental effort, usability, etc. Someof these peripheral QoE PDs need to be further analyzed in order to achieve better working experience (ie, involvement,acceptance, information assimilation, effectiveness, efficiency, mental effort, usability). Some of them should be furtherconsidered for connecting private and business usage leading to the connected life (ie, endurability, entertainment).

TABLE 2 Aspects to be considered and stressed when modeling QoE in UC context

Category Human Contextual System

Aspects impacting QoEto be considered andstressed in the UCcontext

AgeCultureGenderEmotional statePersonalityAttitudeMoodInterestPhysical disabilitiesPrior experienceProductivityUser routineLifestyle

LocationMovementsMobilityTime of day/week/yearDuration of contentFrequency of useEase of useFlexibilitySimplicityPriceType of taskBefore/during/after experienceTime‐related actionsSynchronismSocial contextSpatial characteristics (brightness,surrounding noise, light, temperature)

Technical and information context(presence of other media/devices foraccessing similar type of content,interoperability between and acrossdevices, informational artefacts andaccess relates to other artefacts thatcontain relevant information)

Device/terminal context (device identity,location, status (turned on or off,volume), device capabilities includingthe screen capacity, device proximitywith respect to the user, availablenetwork connectivity at the device,active session at the device)

ContentQuality of UC components' contents (eg,information obtained in read text orseen video)

Visual and audio representation of UCcontent

MediaAdaptation capabilities (eg, capability toadapt various application parameters tofit the device, network, usage contextconstraints, etc)

Synchronization and parallelismcapabilities (eg, capability to syncvarious parallel UC components)

Adjustment to the device powerconsumption

Quality of UC components (eg, bitrate,frame rate)

Data accessOffline capabilitiesTransparent synchronization withbackend systems

Security issuesLack of add‐onsNetworkPhysical phenomena of wireless/wiredchannel—variability (affected by noise,fading, and interference)

Wireless/wired channel reliability(interception, security issues)

Wireless/wired capacity (speed, coverage,limited bandwidth, shared resources)

Wireless/wired channel variability (delay,jitter, loss)

Channel sharing among usersSignal strength (affected by thetemperature, humidity, distance fromthe antenna, base station antenna gain)

Signaling traffic overloadHandoverDeviceScreen size and responsivenessKeyboard size and responsivenessMicrophone qualitySpeakers/headset qualityBattery lifetime ‐ energy consumptionComputational power/resourcesStorage capacitySignaling traffic overloadOperating systemUsability

Abbreviation: UC, unified communications.

BARAKOVIĆ HUSIĆ ET AL. 17 of 25

18 of 25 BARAKOVIĆ HUSIĆ ET AL.

Hence, in order to gain better understanding of QoE modeling for UC, additional aspects that need to be considered andstressed in addition to reviewed ones have been listed in Table 2.

7 | RECOMMENDATIONS AND CONCLUSIONS

In accordance with the objectives of the paper, we have proposed the concept for QoE modeling for UC service andbased on that approach we have provided the analysis of IFs and PDs investigated in the studies considering QoE forUC published update. Previously published results improved our understanding of research considerations related toQoE for UC and serve as a basis for future research activities in this field. The domain would benefit by placing greatemphasis on the complexity of QoE for UC and answering research questions related to the following categories:

• Human—what human characteristics should be considered when studying QoE for UC (eg, previous experience,emotional state, physical disabilities, etc)?

• Contextual—how do different contexts affect the QoE for UC (eg, task, mobility, price, etc)?• System—how do various system components (eg, media, content, network, device) affect the QoE for UC?• QoE PDs—what characteristics of human's experience of the service contributes to its quality?

Therefore, based on the model and the synthesis and organization of the literature, we were able to pose theabovementioned research questions and broadcast them to the research community to be tackled in the future work.

Further on, the results of the meta‐analytical review of research studies on QoE for UC identified 8 HIFs, 5 compositeCIFs, 37 SIFs, and 29 QoE PDs. The most represented IFs studied in terms of QoE of UC service components are genderin terms of HIFs, physical context (ie, location) in terms of CIFs, and bitrate in terms of SIFs. The most researched QoEPDs studied in selected papers are perceived quality and satisfaction. Therefore, the following considerations areoutlined for future work:

• Which peripheral QoE PDs listed in Table 1 may improve the QoE of UC products and services, especially for oftenoverlooked audiences?

• Which IFs affect such peripheral PDs for UC product or service, and how do they impact QoE?• What are the relations between various QoE PDs for UC products and services?

These findings in turn call for a critical review of the current standardization of QoE as according to authors' knowl-edge, there exist no international standard capturing various IFs and QoE PDs of UC.

This paper offers several contributions and implications for both academic and industry communities. First, on aca-demic level, this meta‐analytical review is the first attempt, to our best knowledge, to offer a comprehensive view of var-ious IFs and QoE PDs found in studies considering QoE of UC. Second, the identification of a common IFs and QoEPDs with the proposed QoE modeling concept would support a future quantitative analysis of QoE of UC and contributeto its management. As a result, this could offer a unified view of studies considering QoE for UC. We hope that the pro-posed approach and the findings of this study will be used as the basis for continuing research with the aim to improveour understanding of QoE considerations for UC.

As already explained, in this paper, we focus on identifying the factors and PDs that have been addressed in previousQoE for UC research and analyze their representation, ie, frequency, while the identification of significance and impor-tance of different IFs in context of QoE for UC is out of the scope of this paper. The future studies should utilize thefindings from this one and examine the significance of investigated IFs and PDs in context of QoE for UC. This separateQoE study is needed given that the frequency of appearance of IFs and PDs in relevant publications cannot show theirimportance in context of QoE for UC. The future studies should include real‐life experiments using standardized ques-tionnaires with high number of participants in order to gain meaningful and accurate results, such as identification ofthe most important IFs and PDs for QoE for UC and the quantification of their relations that would contribute to allinterested stakeholders in service supply chain.

This paper also provides a couple of important implications for industry community. First, the results of this papermay encourage different stakeholders in the industry to pay more attention to the key IFs and QoE PDs when theydevelop their UC services and/or products. Second, since the actual QoE evaluation process is conducted without struc-tured framework and there is a need for such approach to evaluate the QoE, the QoE conceptual model proposed in thispaper can be used during a QoE evaluation of UC services and/or products.

BARAKOVIĆ HUSIĆ ET AL. 19 of 25

However, this study comes with some limitations. Although the authors searched intensively for all availableresearch papers considering QoE for UC, the case may be that relevant papers were omitted in this process. Furtheron, although the meta‐analytical review conducted in this paper followed the procedures suggested by the previousstudies,149,150 some subjective decisions were made when some QoE IFs and PDs were collapsed into a singlemeasure.

Beyond the benefit of a standard view of QoE, an important opportunity for future research arises from the data inTable 2. For example, privacy, trust, and security appear to be the most challenging research areas. Again, this observa-tion is not surprising given the growing popularity of this type of research, increasing levels of legislative support, andcommunity interest. Further exploration of this construct, including its relationship with remaining QoE dimensions, iswarranted.

Finally, we hope that the above findings and the suggested research activities will stimulate further research in thisfield resulting in benefits for different stakeholders inside academic and industry communities, as well as everyday UCusers. The following are key points raised in this paper:

• Study the human characteristics and identify cognitive factors and physical disabilities that UC services could bedesigned to accommodate.

• Design UC and its service components that fit particular contextual settings, while being flexible to accommodateothers.

• Focus on different system components when developing UC services.• Consider the wide range of QoE dimensions identified in this study when evaluating the QoE of UC and its service

components.

ORCID

Jasmina Baraković Husić https://orcid.org/0000-0001-6119-6447

REFERENCES

1. Reimer K, Taing S. Unified communications. Bus Inf Syst Eng. 2009;1(4):326‐330.

2. Christensen LT, Fuat Firat A, Cornelissen J. New tensions and challenges in integrated communications. Corp Commun Int J.2009;14(2):207‐219.

3. Evans D. An introduction to unified communications: challenges and opportunities. Aslib Proc New Inf Perspect. 2004;56(5):308‐314.

4. Frost and Sullivan. Key Trends in Unified Communications and Collaboration: Catching the Next Adoption Wave 2016. https://www.frost.com. Accessed May 10, 2018.

5. Cisco Systems. The Unified Communications Journey 2012. https://www.cisco.com/web/offer/grs/88009/32/The_UC_Journey_10_18.pdf. Accessed May 10, 2018.

6. Frost and Sullivan. Demystifying Unified communications: Best Practices for Maximum ROI. https://www.frost.com. Accessed May 10,2018.

7. IDG Enterprise. Unified Communications & Collaboration Study 2015. https://www.idg.com/tools‐for‐marketers/2015‐unified‐commu-nications‐collaboration‐study/. Accessed May 10, 2018.

8. Laghari KUR, Crespi N, Connelly K. Toward total quality of experience: a QoE modelling in a communication ecosystem. IEEECommun Mag. 2012;50(4):58‐65.

9. Brunnström K, Beker SA, de Moor K, Dooms A, Egger S, Garcia M‐N, Hossfeld T, Jumisko‐Pyykkö S, Keimel C, Larabi M‐C. QualinetWhite Paper on Definitions of Quality of Experience 2013. https://hal.archives‐ouvertes.fr/hal‐00977812/document. Accessed May 10,2018.

10. Baraković S, Baraković J, Bajrić H. QoE Dimensions and QoE Measurement of NGN Services. In: Proc. of the 18th Telecommunicationsforum TELFOR. Serbia, Belgrade: Telecommunications Society; 2010.

11. Baraković S, Skorin‐Kapov L. Survey of research on quality of experience modelling for web Browsing. Qual User Exp. 2017;2(6):1-31.

12. Baraković S, Skorin‐Kapov L. Survey and challenges of QoE management issues in wireless networks. Int J Comput Netw CommunSecur. 2013;2013:1‐28.

13. Möller S, Wältermann M, Garcia MN. Features of quality of experience. In: Möller S, Raake A, eds. Quality of Experience. T‐Labs Series inTelecommunication Services. Cham: Springer; 2014.

14. Wältermann M, Raake A, Möller S. Direct quantification of latent speech quality dimensions. J Audio Eng Soc. 2012;60(4):246‐254.

20 of 25 BARAKOVIĆ HUSIĆ ET AL.

15. Baraković S, Skorin‐Kapov L. Multidimensional modelling of quality of experience for mobile web browsing. Comput Hum Behav.2015;50:314‐332.

16. Reiter U, Brunnström K, de Moor K, et al. Factors influencing quality of experience. In: Möller S, Raake A, eds. Quality of Experience. T‐Labs Series in Telecommunication Services. Cham: Springer; 2014.

17. Siris VA, Balampekos K, Marina MK. Mobile Quality of Experience: Recent Advances and Challenges. In: Proc. of the IEEE Interna-tional Conference on Pervasive Computing and Communications Workshops (PERCOM 2014); Budapest, Hungary; 2014.

18. Laghari KUR, Crespi N, Molina B, Palau CE. QoE Aware Service Delivery in Distributed Environment. In: Proc. of the IEEE Workshopsof International Conference on Advanced Information Networking and Applications (WAINA); Singapore, Singapore; 2011.

19. Baraković Husić J, Baraković S, Muminović S. Is there any impact of human influence factors on quality of experience? In: Proc of the40th International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO 2017);Opatija, Croatia; 2017.

20. Laghari KUR. On quality of experience (QoE) for multimedia services in communication ecosystem. PhD Thesis. Institut National desTelecommunictions, 2012.

21. Atzori L, Chen C, Dagiuklas T, Wu H. QoE management in emerging multimedia services. IEEE Commun Mag. 2012;50(4):18‐19.

22. Varela M, Skorin‐Kapov L, de Moor K, Reichl M. In: Chen CW, Chatzimisios P, Dagiuklas T, Atzori L, eds. QoE – Defining a User‐CentricConcept for Service Quality. In: Multimedia Quality of Experience (QoE): Currnet Status and Future Requirements. United Kingdom: JohnWiley & Sons, Ltd; 2016.

23. Scott MJ, Guntuku SC, Huan Y, Lin W, Ghinea G. Modelling Human Factors in Perceptual Multimedia Quality: On The Role of Person-ality and Culture. In: Proc. of the 23rd ACM International Conference on Multimedia; Brisbane, Australia; 2015.

24. Nisbett RI, Miyamoto Y. The influence of culture: holistic versus analytic perception. Trends Cogn Sci. 2005;9(10):467‐473.

25. Setlock LD, Quinones PA, Fussell SR. Does Culture Interact with Media Richness? The Effects of Audio vs. Video Conferencing on Chi-nese and American Dyads. In: Proc. of the 40th Annual Hawaii International Conference on System Sciences (HICSS'07); Waikoloa, HI,USA; 2007.

26. Hyder M, Laghari KUR, Crespi N, Hoene C. Are QoE requirements for multimedia services different for men and women? Analysis ofgender differences in forming QoE in virtual acoustic environments. In: Chowdhry BS, Shaikh FK, Hussain DMA, Uqaili MA, eds.Emerging Trends and Applications in Information Communication Technologies IMTIC 2012. Communications in Computer and Informa-tion Science. Berlin, Heidelberg: Springer; 2012.

27. Zündorf IC, Karnath HO, Lewald J. Male advantage in sound localization at cocktail parties. Cortex. 2011;47(6):741‐749.

28. Lee C. How Does Instant Messaging Affect Interaction Between the Genders? 2003. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.557.218&rep=rep1&type=pdf. Accessed May 10, 2018.

29. Wechsung I, Schulz S, Klaus‐Peter E, Niemann J, Möller S. All Users Are (Not) Equal—The Influence of User Characteristics on Per-ceived Quality, Modality Choice and Performance. In: Delgado RC, Kobayashi T, eds. Proceedings of the Paralinguistic Informationand its Integration in Spoken Dialogue Systems Workshop. New York: Springer; n.d.

30. Naumann AB, Wechsung I, Hurtienne J. Multimodal interaction: a suitable strategy for including older users. Interact Comput.2010;22(6):465‐474.

31. Yen YC, Chu CY, Yeh, S‐L, Chu, HH, Huang P. Lab experiment vs. crowdsourcing: a comparative user study on Skype call quality. In:Proc. of the Asian Internet Engineering Conference (AINTEC); Chiang Mai, Thailand; 2013.

32. Gupta R, Laghari K, Banville H, Falk TH. Using affective brain‐computer interfaces to characterize human influential factors for speechquality‐of‐experience perception modelling. HCIS. 2016;6(5):1-19.

33. Weiss B, Möller S, Schulz M. Modality Preferences of Different User Groups. In: Proc. of the 5th International Conference on Advancesin Computer‐Human Interactions (ACHI 2012); Valencia, Spain; 2012.

34. Zhu Y, Heynderickx I, Redi JA. Understanding the role of social context and user factors in video quality of experience. Comput HumBehav. 2016;49:412‐426.

35. Prabhu P, Sujan MJ, Rakshith S. Effect of compression ratio on perception of time compressed phonemically balanced words in Kannadaand monosyllables. Audiol Res. 2015;5(1):34‐37.

36. Kortum P, Sullivan M. The effect of content desirability on subjective video quality ratings. Hum Factors. 2010;52(1):105‐118.

37. Palhais JSC. Quality of Experience Assessment in Internet TV. In: Proc. of the International Conference on Mobile Networks and Man-agement (MONAMI); Aveiro, Portugal; 2011.

38. Wolf A. Emotional expression online: gender differences in emoticon use. Cyber Psihol Behavior. 2000;3(5):827‐833.

39. Goldstein EB. Encyclopedia of Perception. First ed. Newbury Park, CA: Sage Publications; 2007.

40. Kelley N, Schmeichel BJ. The effects of negative emotions on sensory perception: fear but not anger decreases tactile sensitivity. FrontPsychol. 2014;5:942.

41. Belmudez B, Moller S. Audiovisual quality integration for interactive communications. J Audio Speech Music Process. 2013;2013(24):1‐23.

42. Jumisko‐Pyykkö S, Utriainen T. A hybrid method for quality evaluation in the context of use for mobile (3D) television. Multimed ToolsAppl. 2011;55(2):185‐225.

BARAKOVIĆ HUSIĆ ET AL. 21 of 25

43. Ketykó I, De Moor K, Joseph W, Martens L, De Marez L. Performing QoE‐measurements in an actual 3G network. In: Proc. of the Inter-national Symposium on Broadband Multimedia Systems and Broadcasting (BMSB); Shanghai, China; 2010.

44. Maki T, Varela M, Laulajainen JP. Estimating the effect of context on QoE of audiovisual services: laboratory vs. public places. In: Proc.of the International Conference on Smart Communications in Network Technologies (SaCoNeT); Paris, France; 2013.

45. Belmudez B, Moeller S, Lewcio B, Raake A, Mehmood A. Audio and video channel impact on perceived audio‐visual quality in differentinteractive contexts. In: Proc. of the IEEE International Workshop on Multimedia Signal Processing (MMSP'09); Rio De Janeiro, Brazil;2009.

46. Song W, Tjondronegoro DW, Michae D. Understanding user experience of mobile video: framework, measurement, and optimization.In: Tjondronegoro DW, Din W, eds. Mobile Multimedia – User and Technology Perspectives. INTECH Open Access; 2012.

47. Dabrowski M, Gromada J, Moustafa H, Forestier J. A context‐aware architecture for IPTV services personalization. J Internet Serv Inf Sec(JISIS). 2013;3(1/2):49‐70.

48. Wang Y, Krum DM, Coelho EM, Bowman DA. Contextualized videos: combining videos with environment models to support situationalunderstanding. IEEE Trans Vis Comput Graph. 2007;13(6):1568‐1575.

49. Raynolds G, Barry D, Burke T, Coyle E. Interacting with large music collections:towards the use of environmental metadata. In: Proc. ofthe IEEE International Conference on Multimedia and Expo; Germany, Hannover; 2008.

50. Dias R, Cunha R, Fonseca MJ. A User‐centered Music Recommendation Approach for Daily Activities. In: Proc. of the Workshop onNew Trends in Content‐based Recommender Systems (CBRecSys2014); Sillicon Valley, US; 2014.

51. Schedl M, Schnitzer D. Location‐Aware Music Artist Recommendation. In: Proc. of the 20th Anniversary International Conference onMultiMedia Modeling (MMM 2014); Dublin, Ireland; 2014.

52. Jumisko‐Pyykkö S, Vainio T. Framing the context of use for mobile HCI. Int J Mobile Human Comput Interact. 2010;2(4):1‐28.

53. Wac K, Ickin S, Hong JH, Janowski L, Fiedler M, Dey AK. Studying the Experience of Mobile Applications Used in DifferentContexts of Daily Life. In: Proc. of the 1st ACM SIGCOMM workshop on Measurements up the stack; Toronto, Ontario, Canada;2011.

54. Ickin S, Wac K, Fiedler M, Janowski L, Hong JH, Dey AK. Factors influencing quality of experience of commonly used mobile applica-tions. IEEE Commun Mag. 2012;50(4).

55. Wang X, Rosenblum D, Wang Y. Context‐aware mobile music recommendation for daily activities. In: Proc. of the 20th ACM Interna-tional Conference on Multimedia; Nara, Japan; 2012.

56. Diallo MT. Quality of experience and video services adaptation. PhD Thesis. Institut National des Telecommunications; 2015.

57. Floris A, Atzori L. Managing the quality of experience in the multimedia internet of things: a layered‐based approach. Sensors.2016;16(12):2057.

58. Fröhlich P, Egger S, Schatz R, Mühlegger M, Masuch K, Gardlo B. QoE in 10 seconds: Are short video clip lengths sufficient for Qualityof Experience assessment?. In: Proc. of the 4th International Workshop on Quality of Multimedia Experience (QoMEX'12); Melbourne,Australia; 2012.

59. Borowiak A, Reiter U. Long duration audiovisual content: impact of content type and impairment appearance on user quality expecta-tions over time. In: Proc. of the 5th International Workshop on Quality of Multimedia Experience (QoMEX'13); Klagenfurt amWörthersee, Austria; 2013.

60. Jumisko‐Pyykkö S, Utriainen T. User‐centered quality of experience: is mobile 3D video good enough in the actual context of use. In:Proc. of the International Workshop on Video Processing and Quality Metrics for Consumer Electronics (VPQM 2010); Scottsdale, Ari-zona, USA; 2010.

61. Belmudez B. Audiovisual Quality Assessment and Prediction for Videotelephony. T‐Labs Series in Telecommunication Services. Berlin:Springer International Publishing; 2015.

62. Cui Y, Chipchase J, Jung Y. Personal TV: A Qualitative Study of Mobile TV Users. In: Proc. of the European Conference on interactiveTV 2007 (EuroITV 2007); Amsterdm, Netherland; 2007.

63. Södergård C. Mobile television—technology and user experiences 2003. http://nordic‐ictfore.vtt.fi/materiaali/mobile_television.pdf.Accessed May 10, 2018.

64. Buchinger S, Kriglstein S, Hlavacs H. A comprehensive view on user studies: survey and open issues for mobile TV. In: Proc. of the 7th

European Conference on Interactive TV and Video (EuroITV2009); Leuven, Belgium; 2009.

65. Oksman V, Ollikainen V, Noppari E, Herrero C, Tammela A. ‘Podracing’: experimenting with mobile TV content consumption anddelivery methods. Multimed Syst. 2008;14(2):105‐114.

66. Madhi K. Impact of QoS and contextual parametars on QoE in a video conferencing application. MSc Thesis. KTH Royal Institute ofTechnology. 2016.

67. Su JH, Yeh HH, Yu PS, Tseng VS. Music recommendation using content and context information mining. IEEE Intell Syst.2010;25(1):1541‐1672.

68. Adomavicius G, Tuzhilin A. Toward the next generation of recommender systems: a survey of the state‐of‐the‐art and possible exten-sions. IEEE Trans Knowl Data Eng. 2005;17(6):1041‐4347.

22 of 25 BARAKOVIĆ HUSIĆ ET AL.

69. Guntuku S, Lin W, Scott MJ, Ghinea G. Modelling the Influence of Personality and Culture on Affect and Enjoyment in Multimedia. In:Proc. of the International Conference on Affective Computing and Intelligent Interaction (ACII), Xi'an, China; 2015.

70. Ickin S. Identification of influential factors on android smartphone‐based video quality of experience. PhD Thesis. School of Computing,Blekinge Institute of Technology, Sweden. 2013.

71. Häkkinen J, Kawai T, Takatalo J, Leisti T, Radun J, Hirsaho A, Nyman G. Measuring stereoscopic image quality experience with inter-pretation based quality methodology. In: Proc. of the SPIE – The International Society for Optical Engineering 6808; San Jose, California,USA; 2008.

72. Vandenbroucke K, Núñez Castellar EP, De Marez L. Subjective insights from time and place shifters in assessing temporal qualityof experience. In: Proc. of the 7th International Workshop on Quality of Multimedia Experience (QoMEX'15); Pylos‐Nestoras, Greece;2015.

73. Bradley NA, Dunlop MD. Towards a multidisciplinary model of context to support context‐aware computing. Hum Comput Interact.2005;20(4):403‐446.

74. O'Hara K, Mitchell AS, Vorbau A. Consuming video on mobile devices. In: Proc. of the SIGCHI Conference on Human Factors in Com-puting Systems (CHI'07), San Jose, California, USA, 2007.

75. Skorin‐Kapov L, Varela M. A multi‐dimensional view of QoE: the ARCU model. In: Proc. of the 35th International Convention on Infor-mation and Communication Technology, Electronics and Microelectronics (MIPRO 2012); Opatija, Croatia; 2012.

76. Xue J, Chen CW. A study on perception of mobile video with surrounding contextual influences. In: Proc. of the 4th International Work-shop on Quality of Multimedia Experience 2012 (QoMEX'12); Melbourne, Australia; 2012.

77. Zhao T, Liu Q, Chen CW. QoE in video transmission: a user experience‐driven strategy. IEEE Commun Surveys Tutor.2017;19(1):285‐302.

78. Kara PA, Bokor L, Sackl A, Mourão M. What your phone makes you see: investigation of the effect of end‐user devices on the assessmentof perceived multimedia quality. In: Proc. of the 7th International Workshop on Quality of Multimedia Experience (QoMEX'15); Pylos‐Nestoras, Greece; 2015.

79. Sackl A, Zwickl P, Reichl P. The trouble with choice: an empirical study to investigate the influence of charging strategies and contentselection on QoE. In: Proc. of the 9th International Conference on Network and Service Management (CNSM'13); Zürich, Switzerland;2013.

80. Shaikh J, Fiedler M, Collange D. Quality of Experience from user and network perspectives. Ann Telecommun. 2010;65(1‐2):47‐57.

81. Janowski L, Romaniak P, Papir Z. Content driven QoE assessment for video frame rate and frame resolution reduction. Multimed ToolsAppl. 2012;61(3):769‐786.

82. Choi I, Shinn‐Cunningham B, Chon SB, Sung KM. Objective measurement of perceived auditory quality in multi‐channel audio com-pression coding systems. J Audio Eng Soc. 2008;56(1/2):3‐17.

83. Brunnström K, Stålenbring D, Pettersson M, Gustafsson J. The impact of transmission errors on progressive 720 lines HDTV coded withH.264. In: Proc. of the Human Vision and Electronic Imaging XV – pat of the IS&T‐SPIE Electronic Imaging Symposium; San Jose, CA,USA, 2010.

84. Ghinea G, Patterson KJ. Perceived Multimedia Quality: The Impact of Device Characteristics. In: Proc. of the International Conferenceon Human‐Computer Interaction; Orlando, FL, USA; 2011.

85. Fiedler M. A generic quantitative relationship between quality of experience and quality of service. IEEE Netw. 2015;24(2):36‐41.

86. Daengsi T, Khitmoh N, Wuttidittachotti P. VoIP quality measurement: subjective VoIP quality estimation model for G.711 and G.729based on native Thai users. Multimed Syst. 2015;22(5):575‐586.

87. Côté N, Koehl V, Möller S, Raake A, Wältermann M, Gautier‐Turbin, V. Diagnostic instrumental speech quality assessment in asuper‐wideband context. In: Proc. of the 3rd International Workshop on Perceptual Quality of Systems (PQS 2010); Bautzen, Germany;2010.

88. Silva VD, Kim C, Haddad N, Politis I, Kordelas A, Dagiuklas T, Weitnauer M, Hartmann C. An End‐to‐End QoE Measurement Frame-work For Immersive 3D Media Delivery Systems. In: Proc. of the 1st ROMEO Workshop; Athens, Greece; 2012.

89. Chen W, Fournier J, Barkowsky M, Le Callet P. New stereoscopic video shooting rule based on stereoscopic distortion parameters andcomfortable viewing zone. In: Proc. of the SPIE – The International Society for OpticalEngineering; San Franciso, California, USA; 2011.

90. Bae SH, Juang BH. Subjective evaluation of spatial resolution and quantization noise tradeoffs. IEEE Trans Image Process.2009;18(3):495‐508.

91. AL‐Nuaim H, Abukhodair N. A user perceived quality assessment of lossy compressed images. Int J Comput Graph. 2011;2(2):23‐36.

92. Nyman G, Radun J, Leisti T, Vuori T. From image fidelity to subjective quality: a hybrid qualitative/quantitative methodology for mea-suring subjective image quality for different image contents. In: Proc. of the 12th International Display Workshops in Conjunction withAsia Display (IDW/AD'05); Tokyo, Japan; 2005.

93. Deng C, Ma L, Lin W, Ngan KN. Visual Signal Quality Assessment. Springer International Publishing; 2015.

94. Lebreton P, Raake A, Barkowsky M, Le Callet P. Evaluating depth perception of 3D stereoscopic videos. IEEE J Sel Top Sign Proces.2012;6(6):710‐720.

BARAKOVIĆ HUSIĆ ET AL. 23 of 25

95. Caisy Hung YT, Kong WC, Chua AL, Hull CE. Reexamining Media Capacity Theories using Workplace Instant Messaging. In: Proc. ofthe 39th Annual Hawaii International Conference on System Sciences; Kauia, HI, USA; 2006.

96. Khan A, Sun L, Jammeh E, Ifeachor E. Quality of experience‐driven adaptation scheme for video applications over wireless networks.IET Commun. 2010;4(11):1337‐1347.

97. Joskowicz J, López Ardao JC. Combining the effects of frame rate, bit rate, display size and video content in a parametric video qualitymodel. In: Proc. of the 6th Latin America Networking Conference (LANC'11); Quito, Ecuador; 2011.

98. Zinner T, Hohlfeld O, Abboud O, Hossfeld T. Impact of Frame Rate and Resolution on Objective QoE Metrics. In: Proc. of the 2nd Inter-national Workshop on Quality of Multimedia Experience (QoMEX'10); Trondheim, Norway; 2010.

99. Puglia R. Influence of audio and video quality on subjective audiovisual quality: H.263 and Adaptive Multi Rate (AMR) coding. MScThesis. Vienna University of Technology. 2011.

100. Tavakoli S, Gutiérrez J, García N. Quality assessment of adaptive 3D video streaming. In: Proc. of the SPIE 8650, Three‐DimensionalImage Processing (3DIP) and Applications. Burlingame, California, United States: IS&T/SPIE Electronic Imaging; 2013.

101. Nasiri RM, Wang J, Rehman A, Wang S, Wang Z. Perceptual quality assessment of high frame rate video. In: Proc. of the 17th Interna-tional Workshop on Multimedia Signal Processing (MMSP); Xiamen, China; 2015.

102. Ljubojević M, Vasković V, Babić Z, Starčević D. Influence of resolution and frame rate on the linear in‐stream video ad QoE. J InfTechnol Appl (JITA). 2014;4(1):15‐22.

103. Banitalebi‐Dehkordi A, Pourazad MT, Nasiopoulos P. Effect of High Frame Rates on 3D Video Quality of Experience. In: Proc. of theIEEE International Conference on Consumer Electronics (ICCE); Las Vegas, NV, USA; 2014.

104. Ou YF, Lin W, Zeng H, Wang Y. Perceptual Quality of Video with Periodic Frame Rate and Quantization Variation–Subjective Studiesand Analytical Modeling 2014. https://arxiv.org/abs/1406.2018. Accessed May 10, 2018.

105. Karlsson LS. Coding for Improved Perceived Quality of 2D and 3D Video over Heterogeneous Networks. PhD Thesis. Department ofInformtion Technology and Media. Mid Sweden University. 2010.

106. Cheon M, Leeab JS. Temporal resolution vs. visual saliency in videos: analysis of gaze patterns and evaluation of saliency models. SignalProcess Image Commun. 2015;39:405‐417.

107. Vakili A, Grégoire J‐C. QoE Management in a Video Conferencing Application. Future Information Technology, Application, and Service.Dordrecht: Springer; 2012.

108. Ou JF, Liu T, Zhao Z, Ma Z, Wang Y. Modeling the impact of frame rate in perceptual quality of video. In: Proc. of the 15th InternationalConference on Image Processing (ICIP 2008); San Diego, CA, USA; 2008.

109. Chen S, Chu CY, Yeh SL, Chu HH, Huang P. Modeling the QoE of Rate Changes in Skype/SILK VoIP Calls. IEEE/ACM Trans Network-ing. 2014;6:1781‐1793.

110. Joskowicz J, López Ardao JC. A General Parametric Model for Perceptual Video Quality Estimation. In: Proc. of the International Work-shop Technical Committee on Communications Quality and Reliability (CQR); Vancouver, BC, Canada; 2010.

111. Rehman A, Zeng K, Wang Z. Display Device‐Adapted Video Quality‐of‐Experience Assessment. In: Proc. of the SPIE/IS&T HumanVision and Electronic Imaging 9394; 2015.

112. Schlosser D, Jarschel M, Burger V, Pries R. Monitoring the user perceived quality of SILK‐based voice calls. In: Proc. of the AustralalsianTelecommunication Networks and Applications Conference (ATNAC); Auckland, New Zealand; 2010.

113. Chen KT, Huang CY, Huang P, Leil CL. Quantifying Skype user satisfaction. In: Proc. of the Conference on Applications, technologies,architectures, and protocols for computer communications (SIGCOMM'06); Pisa, Italy; 2006.

114. Daengsi T. VoIP Quality Measurement: Recommendation of MOS and Enhanced Objective Measurement Method for Standard ThaiSpoken Language. PhD Thesis. King Mongkut's University of Technology, North Bangkok; 2012.

115. Pimentel‐Niño MA, Saxena P, Vazquez‐Castro MA. Reliable adaptive video streaming driven by perceptual semantics for situationalawareness. Scientific World Journal. 2015;2015:1‐16.

116. Raake A, Gustafsson J, Argyropoulos S, et al. IP based mobile and fixed network audiovisual media services. IEEE Signal Process Mag.2011;28(6):68‐79.

117. Pham‐Thi H, Hoang‐Van H, Miyoshi T. Correlating objective factors with video quality experienced by end users on P2PTV. Int JComput Netw Commun (IJCNC). 2015;7(3):59‐73.

118. Addu RK, Kumar V. Effect of Codec Performance on Video QoE for videos encoded with Xvid, H.264 and WebM/VP8. MSc Thesis.School of Computing, Blekinge Institute of Technology, 2014.

119. Ohm J, Sullivan GJ, Schwarz H, Tan TK, Weigand T. Comparison of the coding efficiency of video coding standards—including highefficiency video coding (HEVC). IEEE Trans Circuits Syst Video Technol. 2012;22(12):1669‐1684.

120. Řeřábek M, Hanhart P, Korshunov P, Ebrahimi T. Quality evaluation of HEVC and VP9 video compression in real‐time applications. In:Proc. of the 7th International Workshop on Quality of Multimedia Experience (QoMEX'15); Pylos‐Nestoras, Greece; 2015.

121. Grois D, Marpe D, Mulayoff A, Itzhaky B, Hadar O. Performance comparison of h. 265/mpeg‐hevc, vp9, and h. 264/mpeg‐avc encoders.In: Proc. of the Picture Coding Symposium (PCS); San Jose, CA, USA; 2013.

122. Steinmetz R. Human perception of jitter and media synchronization. IEEE J Sel Areas Commun. 1996;14(1):61‐72.

24 of 25 BARAKOVIĆ HUSIĆ ET AL.

123. Ud Din S, Bulterman D. Synchronization Techniques in Distributed Multimedia Presentation. In: Proc. of the International Conferenceson Advances in Multimedia (MMEDIA 2013); Chamonix/Mont Blanc, France; 2012.

124. Yuan Z, Bi T, Muntean GM, Ghinea G. Perceived Synchronization of Mulsemedia Services. IEEE Trans Multimed. 2015;17(7):957‐966.

125. Ghinea G, Ademoye OA. Perceived Synchronization of Olfactory Multimedia. IEEE Trans Syst Man Cybern Part A Syst Hum.2010;40(4):657‐663.

126. Calyam P, Chandrasekaran P, Trueb G, et al. Multi‐resolution multimedia QoE models for IPTV applications. Int J Digit MultimedBroadcast. 2012;2012:1‐13.

127. Elmnsi A, Osman N, Mkwawa IH. The Impact of Mobile Device Preference on the Quality of Experience. Int J Comput Inf Sci.2016;12(1):89‐94.

128. Catellier A, Pinson M, Ingram W, Webster A. Impact of mobile devices and usage location on perceived multimedia quality. In: Proc. ofthe 4th International Workshop on Quality of Multimedia Experience (QoMEX'12); Melbourne, Australia; 2012.

129. Schmitt M, Gunkel S, Cesar P, Bulterman D. The Influence of Interactivity Patterns on the Quality of Experience in Multi‐party Video‐mediated Conversations under Symmetric Delay Conditions. In: Proc. of the 3rd International Workshop on Socially‐Aware Multimedia(SAM 2014); Orlando, Florida, USA; 2014.

130. Orosz P, Skopko T, Nagy Z, Lukovics T. A No‐reference Voice Quality Estimation Method for Opus‐based VoIP Services. Int J AdvTelecommun. 2014;7(1&2):12‐21.

131. Chen KT, Tu CC, Xiao WC. OneClick: A Framework for Measuring Network Quality of Experience. In: Proc. of the 28th Conference onComputer Communications (IEEE INFOCOM 2009); Rio de Janeiro, Brazil; 2009.

132. Langvik S. Influence of audio on perception and comprehension of video sequences. MSc Thesis. KTH Royal Institute of Technology.2015.

133. Juniper Networks. Planning Guide for Deploying Microsoft Office Communication Server 2007 R2 in a Juniper Distributed Enterprise2010. https://www.juniper.net/us/en/local/pdf/implementation‐guides/8010074‐en.pdf Accessed June 10, 2018.

134. Sackl A, Casas P, Schatz R, Janowski L, Irmer R. Quantifying the impact of network bandwidth fluctuations and outages onWeb QoE. In: Proc. of the 7th International Workshop on Quality of Multimedia Experience (QoMEX'15); Pylos‐Nestoras, Greece;2015.

135. Bhattacharya A, Wu W, Yang Z. Quality of experience evaluation of voice communication: an affect based approach. HCIS. 2012;2(7):7.

136. Calyam P, Ekici E, Lee C‐G, Haffner M, Howes N. A gap‐model based framework for online VoIP QoE measurement. J Commun Net-works. 2007;9(4):446‐456.

137. Serral‐Gracià ER, Cerqueira E, Curado M, Yannuzzi M, Monteiro E, Masip‐Bruin X. An Overview of Quality of Experience Measure-ment Challenges for Video Applications in IP Networks. In: Proc. of the International Conference on Wired/Wireless InternetCommunications (WWIC), Luleå, Sweden; 2010.

138. Laghari KUR, Pham TT, Nguyen H, Crespi N. QoM: a new quality of experience framework for multimedia services. In: Proc. of the 17th

IEEE Symposium on Computers and Communications (ISCC'12); Cappadocia, Turkey; 2012.

139. Venkataraman M, Chatterjee M, Chattopadhyay S. Evaluating Quality of Experience for Streaming Video in Real Time. In: Proc. of theIEEE Global Telecommunications Conference (IEEE GLOBECOM 2009); Honolulu, HI, USA; 2009.

140. Kawano T, Yamagishi K, Watanabe K, Okamoto J. No reference video‐quality‐assessment model for video streaming services. In: Proc. ofthe 18th International Packet Video Workshop (PV 2010); Hong Kong, China; 2010.

141. Reichl P, Egger S, Schatz R, D'Alconzo A. The Logarithmic Nature of QoE and the Role of the Weber‐Fechner Law in QoEAssessment Sign In or Purchase. In: Proc. of the IEEE International Conference on Communications (ICC 2010); Cape Town, SouthAfrica; 2010.

142. Mu M, Cerqueira E, Boavida F, Mauthe A. Quality of Experience management framework for real‐time multimedia applications. Int JInternet Protoc Technol. 2009;4(1):54‐64.

143. Cero E, Baraković Husić J, Baraković S. IoT's Tiny Steps towards 5G: Telco's Perspective. Symmetry. 2017;9(10):1-38.

144. Masood H, Rizvi S, Iqbal BM. An Overview of QoE for 5G Networks. Int J Inf Technol Electr Eng. 2016;5(6):1‐5.

145. Stankiewicz R, Jajszczyk A. A survey of QoE assurance in converged networks. Comput Netw. 2011;55(7):1459‐1473.

146. Rosenthal R. Meta‐analytic procedures for social research. Newbury Park, CA: Sage Publications; 1991.

147. Glass G, McGaw B, Smith M. Meta‐analysis in social research. Newbury Park, CA: Sage Publications; 1981.

148. Arndt S, Ammar D, Voigt‐Antons JN, Heegaard PE. Exploring diverse measures for evaluating QoE in the context of WebRTC. In: Proc.of the 10th International Conference on Quality Multimedia Experience (QoMEX'17); Erfurt, Germany; 2017.

149. Korhonen H, Arrasvuori J, Väänänen‐Vainio‐Mattila K. Analysing User Experience of Personal Mobile Products throughContextual Factors. In: Proc. of the 9th International Conference on Mobile and Ubiquitous Multimedia (MUM'10); Limassol, Cyprus;2010.

150. Westerink JH, Roufs JA. Subjective image quality as a function of viewing distance, resolution, and picture size. SMPTE J.1989;98(2):113‐119.

BARAKOVIĆ HUSIĆ ET AL. 25 of 25

AUTHOR BIOGRAPHIESJasmina Baraković Husić received her PhD degree in 2009 at the Faculty of Electrical Engineering and Comput-ing, University of Zagreb. She is an associate professor at the Faculty of Electrical Engineering, University of Sara-jevo. She is employed by BH Telecom, Joint Stock Company, Sarajevo, as a specialist in the Department for Planningand Development of Technologies and Services. Her research interests include a variety of topics in signalling andQoS/QoE management, IoT, and 5G communications.

Sabina Baraković received her PhD degree in 2014 at the Faculty of Electrical Engineering and Computing, Uni-versity of Zagreb. She is an expert adviser at the Ministry of Security of Bosnia and Herzegovina and an assistantprofessor at the University of Sarajevo and American University in Bosnia and Herzegovina. Her research interestsinclude multidimensional QoE modelling of multimedia, mobile web‐based applications, and unified communica-tions, and management of QoS/QoE in mobile and NGN environment.

Enida Cero received her Master's degree in Engineering in 2013 at the Faculty of Electrical Engineering, Universityof Sarajevo. She is professional associate at the BH Telecom, Joint Stock Company, Sarajevo, and assistant at Amer-ican University in Bosnia and Herzegovina. Her research interests include IoT (application, protocols, and architec-tures), intelligent process optimization, network technologies, and QoS/QoE techniques.

Nina Slamnik obtained her Master of Electrical Engineering degree in 2016 at the Faculty of Electrical Engineering,University of Sarajevo. She is a teaching assistant at Faculty of Electrical Engineering, University of Sarajevo, who isengaged in different teaching modules in a variety of electrical engineering fields. The range of her research interestsinclude heterogeneous communication networks, low‐power communication networks, software‐defined wirelessnetworks, IoT, and QoS/QoE management.

Merima Oćuz received Master's degree in Telecommunication Engineering in 2016 at the Faculty of ElectricalEngineering, University of Sarajevo. Her research areas of interest during studies were computer networks, traffictheory, QoS/QoE, and telecommunication networks. Currently, she is employed by Atlantbh d.o.o. Sarajevo as Soft-ware Engineer. Her work and research interests include quality assurance and test automation for large enterprisesoftware solutions.

Azer Dedović received his Master of Electrical Engineering degree in 2016 at the Faculty of Electrical Engineering,University of Sarajevo. He is Software Engineer at Symphony Sarajevo, working with cutting‐edge technologies.

Osman Zupčić received his Master's degree in 2016 at the Faculty of Electrical Engineering, University of Sarajevo.He is employed by ZIRA Ltd. Sarajevo, Bosnia and Herzegovina as a software engineer. His research interestsinclude a variety of topics in trading, routing and rating management, big data management, QoS/QoE manage-ment, and capacity management.

SUPPORTING INFORMATION

Additional supporting information may be found online in the Supporting Information section at the end of the article.

How to cite this article: Baraković Husić J, Baraković S, Cero E, et al. Quality of experience for unifiedcommunications: A survey. Int J Network Mgmt. 2019;e2083. https://doi.org/10.1002/nem.2083