Verification and correlation of attributes used for ... · 2Institute of Sound Recording,...

Verification and correlation of attributes used for

describing the spatial quality of reproduced sound

Jan Berg1 and Francis Rumsey2

1School of Music, Luleå University of Technology, Sweden2Institute of Sound Recording, University of Surrey, Guildford, UK

When the spatial quality of reproduced sound is to be assessed, knowledge of the dimensionsforming the quality is essential since the quality is known to be multi-dimensional. Thedimensions could be indicated by attributes describing them. Attributes encountered during aprevious experiment are considered by a group of subjects and their responses are analysed forfinding the attributes’ applicability and dimensionality over an extended number of soundstimuli.

INTRODUCTIONOne main characteristic of the more recent audiorecording and reproducing systems is an increasedability to reproduce the spatial features of sound.Spatial features can be exemplified by location ofsounds, sense of the acoustical environment in whichthe sound source is located, or as expressed in [1],“the three-dimensional nature of the sound sourcesand their environment”. The performance of a soundsystem in that respect could be denoted as “spatialquality”.

A knowledge and an understanding of the factorsor dimensions forming and thereby affecting the spa-tial quality in a sound reproduction system isessential in the different fields of audio work:• recording – for knowing the chosen recording tech-

nique’s influence on the spatial quality of therecording;

• post-production – for knowing how to enhancespatial quality;

• reproduction – for knowing possibilities and limi-tations of different modes of reproduction, e g two-channel mixdowns of multichannel recordings;

• coding – for assessing audio coding algorithms’impact on spatial quality;

• verification of ‘objective’ measurements – forverifying the correlation between measurements ofphysical parameters and effects perceived bylisteners;

• audio quality assessments – for evaluating thespatial quality part of total audio quality.

One key issue for those involved in audio work –from recording to reproduction – is the perceivedresult at the listener’s end. Since this raises the ques-tion of which components in a reproduced sound that

are perceivable or not, and how these are interrelated,methods used in the behavioural sciences have to beconsidered for getting a better understanding of thephenomena. The central problem seems to be how to‘measure’ a person’s (a subject) conception of anauditory event. This information has to be elicitedand communicated from the subject in some way.Ways of collecting and analysing data are reviewedby Rumsey in [2]. The methods available for thismainly rely on verbal or graphical techniques.Examples of verbal techniques are the Repertory GridTechnique (used in Personal Construct Psychology)[3, 4, 5] and Quantitative Descriptive Analysis (usedin food research) [6]. Graphical elicitation has beendiscussed and used by Mason et al [7] and Wenzel[8]. Both verbal and graphical methods have theiradvantages and disadvantages. The authors’ approachis to use verbal communication with the subjects.

In an attempt to find the dimensions of spatialquality, an experiment was conducted in 1998. Theexperiment is described in [1], and its approach wasto try to elicit information from the participatingsubjects by playing a number of reproduced soundsto them, where after they were asked for verbal de-scriptions of similarities and differences between thesounds. The subjects then graded the different soundson scales constructed from their own words. This wasan example of a technique where the subjects cameup with descriptions using their own vocabulary withknown meaning to them, instead of being providedwith the experimenter’s descriptors for the scales.The data was subsequently analysed by methods usedin the Repertory Grid Technique, aimed to find apattern or a structure not necessarily known to thesubjects (or the experimenters) themselves. The ex-

BERG and RUMSEY VERIFICATION AND CORRELATION OF ATTRIBUTES

AES 19TH INTERNATIONAL CONFERENCE 2

perimental idea was to investigate if a pattern withdistinguishable groups of descriptors showed, and ifso, it would be regarded as an indicator of the pres-ence of the dimensions searched for. The results fromthis experiment have been reported in [1,9,10,11],and indicated the existence of a number of dimen-sions described by attributes generally used by thesubjects for describing perceived dimensions ofspatial audio. In [11] the correlation between some ofthe attributes was reported.

The problem with verbal descriptions of a per-ceived event is addressed by the authors in [1]. Manyfactors affect a person’s conception and his/herdescription of the event, e g memory, other senses,emotion/sentiment, training, terminology, etc.Another important issue is the interpretation of verbaldata and the bias involved with this [10]. With this inmind, in addition to the experiences and the resultsfrom the 1998 experiment, a number of questionsarise concerning the feasibility of attributes in theform of verbal descriptors as means of assessing spa-tial quality:• Are these attributes valid for describing the spatial

quality of (a subset of) reproduced sounds?• Are scales defined by words interpreted similarly

within a group of subjects?• If such scales are found to be valid, are certain

attributes interrelated and if so, in which way?In order to answer these questions, a new experimentwas designed and conducted in early 2001. The newexperiment made use of the attributes resulting fromthe 1998 experiment. Scales were constructed fromthe attributes and were provided to a new group ofsubjects. The subjects assessed a number of soundstimuli on the provided scales. The hypothesis to betested in the experiment and its alternative were:• If the scales are not relevant for describing parts of

spatial quality of a subset of reproduced sounds,they will have insufficient common meaning to thesubject group, which will not be able to makedistinctions between any stimuli at a significantlevel, i e the data will contain mostly randomlydistributed points.

• If, however, the scales are relevant in this respect,the scales will have sufficient common meaning tothe group, which will be able to make distinctionsbetween some or all of the stimuli in the experi-ment at a significant level.

If the alternative hypothesis is true, the interrelationsof scales and attributes can be analysed subsequently.

The main purpose of the experiment is to investi-gate the feasibility of using verbal attributes ascomponents in assessment of spatial quality of repro-duced sound. It is not the authors’ intention to focus

on the source material used as stimuli, their physicaldifferences or different recording/downmixing tech-niques. Differences in these are mainly used in thisexperiment as means of exciting different dimensionsof the subjects’ perception of sound, in order to spanthe spatial quality space. However, some observa-tions of the interaction between attributes and certainstimuli have been made. The results from thisexperiment are reported in this paper.

ATTRIBUTESThe analyses of the 1998 experiment yielded a num-ber of attributes used by the subjects. Some of theattributes occurred in more than one analysis. Oneproblem encountered during the work was the factthat an attribute could refer to different parts of theauditory event, such as the sound source itself (a sin-gle instrument), a group of sources (a section ofinstruments), groups of sources (an orchestra) or thewhole scene including the reflected sounds (from thehall). This encouraged the authors to assign certainattributes to defined parts of the auditory scene inorder to avoid confusion about what the attributeswere referring to. Since the task of the new experi-ment was to possibly verify the findings of the previ-ous one, all these findings had to be compiled in acomprehensible form with ample size for the newgroup of listeners to consider. This implies a reduc-tion of the number of elicited attributes from the 1998experiment, with the intention to keep the main partof them for the new experiment. This compilationwas made by the authors. The omitted attributes were“externalisation”, “phase” and “technical device”,since they were a result of the use of phase reversedsignals in the 1998 experiment and that no phasereversed signals were considered for the new experi-ment. Externalisation (to perceive sound as comingf ro m ou tside on e’ s h ead in co ntr as t to ‘internalisation’where the sound is perceived as coming from withinthe head) was also considered as being a dichoto-mous attribute hard to grade on the linear scales thatwere going to be used in the experiment. Theattributes included in the new experiment weredivided into four attribute classes:• General attributes – referring to the whole sound as

an entity.• Source attributes – referring to the sound of the

sound source.• Room attributes – referring to sound relating to the

acoustical environment as a result from a soundsource’s initial action, e g reverberation.

• Other attributes – sounds generated neither by thesource nor its interaction with the room.



The attribute classes contained attributes accompa-nied by a written description of each attribute. Theattributes and their descriptions are found inAppendix A. The following 12 attributes (with theirabbreviations and attribute classes) were used in theexperiment• Naturalness nat General• Presence psc General• Preference prf General• Envelopment env General• Source width swd Source• Localisation loc Source• Source distance dis Source• Room width rwd Room• Room size rsz Room• Room spectral bandwidth rsp Room• Room sound level tlv Room• Background noise level bgr OtherWhich attributes could be purely spatial or non-spatial or a mixture of both was not considered, sincethe origin of the attributes is the previous experiment,where subjects came up with these attributes asdescriptors of their experiences. The attributesyielded from the analyses of the previous experimentand their relation to the derived attributes (abbrevi-ated) used in the new experiment are shown in fig. 1.

EXPERIMENTAL DESIGNThe outline of the experiment was to provide a non-naïve group of subjects with a list of attributes withassociated descriptions and, for every attribute, listento a number of different sound stimuli and grade thestimuli on scales defined by the attributes. Thesubjects performed the experiment one at a time in alistening room equipped with loudspeakers and a userinterface in the form of a computer screen, a key-board and a mouse. All communication with thesubjects was made in Swedish.

StimuliThe main part of the experiment comprised gradingof attributes relating to the source or the room. There-fore it was important to have stimuli in the form ofrecordings with a relatively low complexity, in thiscase meaning one single stationary centre-positionedsource within a room or a hall. This was to avoid thedifficulties with assessing too complex recordings,e g containing multiple sources, moving sources,changes of reverberation, etc. Consequently, thestimuli were all recordings of single sources inacoustical environments, i e no anaechoic conditionswere used. To achieve ecological validity (‘real-

ATTRIBUTES FROM PREVIOUS ANALYSES REF ATTRIBUTES FOR NEW EXPERIMENTnat psc prf env swd loc dis rwd rsz rsl rsp bgrG G G G S S S R R R R O

authenticity/naturalness [9] x

lateral positioning/source size [9] x x

envelopment [9] x

depth [9] x

room/reverberation properties: spectral, level and clarity [9] x x x

source width [9] x

(externalisation) [9]

frontal image [9] x

localisation, left – right and front – back [10] x

depth/distance [10] x

envelopment [10] x

width [10] x x

room perception [10] x x x

(externalisation) [10]

(phase) [10]

source width [10] x

source depth [10] x

detection of background noise [10] x

frequency spectrum [10] x

naturalness [11] x

presence [11] x

(technical device) [11]

positive/negative [11] x

Fig 1. Table showing attributes yielded from previous analyses of the 1998 experiment and the derived attributes for the newexperiment. Omitted attributes within brackets. The attribute class is indicated by letters under the attribute abbreviation.



world validity’), sound sources with high probabilityto encounter in a natural listening situation wereused: trumpet, saxophone, speech and flute. The fluteand the trumpet recordings can be characterised asmore reverberant (‘hall’) than the speech and thesaxophone (‘room’). These differences are aimed tospan the scaling of the room parameters.

In the 1998 experiment it was noted that differentmodes of reproduction excited a number of spatialsensations. Since one aim of the new experiment wasto verify the findings in the previous one, this methodwas employed once again. The four originalrecordings were available in a 5-channel format,intended for reproduction on five different speakers.The 5-channel recordings were all downmixed forreproduction in 2-channel stereo and 2-channelphantom mono. In addition to that, the speechrecording only was mixed down to mono. Thedownmix coefficients for 2-channel stereo were

L L C L

R R C R

C L R

L R C L R

L R C L R

F C R

F C R

S S

S S

F F C R R

= + += + += = =

0 71

0 71

0

.

.

, , ,

where

and are the output signals to the

5 - speaker system

and where

, , , and are the original signals from

the 5 - channel recordings.

For the phantom mono downmixes the coefficientswere

L R L R C L R

C L RF F C R R

S S

= = + + + += = =

1 4

0

.

and finally, the coefficients for the mono stimulus

Every sou nd stimulus was as signed a num ber fo ridentification throug hout the tes t, “item 1…1 3”. Th ecom plete list o f soun d stim uli with their identifyingnum ber an d the mixdow n mode used is sho wn in fig. 2.

The flute and the trumpet recordings were col-lected from the part “Musik in Surround”, track id 6and 12 on the “Multichannel Universe” DVD [12],and the saxophone and the speech recording weremade at the School of Music in Piteå, Sweden. Boththe saxophone and the speech recordings were usedin the 1998 experiment and details of the recordingtechnique were given in [1]. All recordings weredownmixed on a ProTools system and stored as*.wav files with a resolution of 16 bits and 48 kHzsampling frequency. The level difference between thedifferent 5-channel recordings were adjusted in thelistening room “by ear” by two persons untilconsensus was reached about the plausible soundlevel for the different sources. The downmixed ver-sions of the original 5-channel recording were levelaligned – also in the listening room – against theoriginal by measuring the equivalent continuoussound level, Leq(A), for the first 10 seconds of thesound files and adjusting the difference to be within±1 dB.

Item Programme Downmix mode Leq(A)

1 Flute 5 65.3

2 Flute 2 65.3

3 Flute p 65.6

4 Saxophone 5 76.9

5 Saxophone 2 77.5

6 Saxophone p 77.1

7 Speech 5 71.6

8 Speech 2 72.0

9 Speech p 71.9

10 Speech m 71.5

11 Trumpet 5 78.0

12 Trumpet 2 78.6

13 Trumpet p 78.3

5 = 5-channel stereo

2 = 2-channel stereo

p = phantom mono

m = mono

Fig. 2. Stimuli used in the experiment, their identificationand measured sound level in the listening roomC L R C L R

L R L RF F C R R

S S

= + + + += = = =

1 4

0

.



SubjectsThe number of subjects completing a whole testsession was 19. All subjects were students from thesound recording programme at the School of Music.They had previously participated in listening testsaimed to assess the total audio quality of codingalgorithms in bit-reduction systems. They had neitherreceived any special training in assessing spatialquality or any instructions in using common languagefor describing the spatial features of recordings. Inconclusion, the subjects should be regarded as expe-rienced listeners of reproduced sound, without anyparticular bias for a certain way of describing spatialquality.

Listening conditionsThe experiment was executed in a reproduction roomat the School of Music. The dimensions of the roomwas 6 × 6.6 × 3.2 m (w × d × h). All reproductionwas made through Genelec 1030A loudspeakers, con-figured according to BS-1116 [13] at a 2 m distancefrom the listening position, fig. 3. The settings ofeach loudspeaker were: Sensitivity = +6 dB, Trebletilt = +2 dB, Bass tilt = -2 dB. Only one subject at atime was present in the listening room during theexperiment. Equipment with fans was acousticallyinsulated to avoid noise in the listening room. Theroom had no windows and the only light in the roomwere a small spotlight at the listening position forreading. This was to increase the subject’s concen-tration on the user interface and minimise visualdistraction from the room.

Experiment equipmentThe experiment was performed on a computer (PC)by which each test session was controlled. All soundfiles were stored on the computer’s disk and playedback via an 8-channel sound card installed in thecomputer. (Only five channels were used.) The soundcard output delivered audio data in the T-DIF format.This was converted by a DA-88 tape recorder intofive discrete analogue signals directly feeding thespeakers. The equipment had been tested for phasedifferences between the channels and as well as forgeneral audio quality by Sveriges Radio (the SwedishBroadcasting Corporation).

For controlling the test, special software wasdesigned. Both playback controls as well ascollecting subject responses were handled by thesoftware.

Experiment executionPrior to the test, every subject received a writteninstruction, where the experiment was described. Alist of the attributes (Appendix A), to be used in thetest accompanied the written instruction. The subjectswere allowed to ask questions about the instruction,but not about the attributes and their descriptions.The instruction and the attribute list were availablefor the subjects during the whole session.

A session started with a training phase with acontent corresponding only to 25% of the actual testto avoid subject fatigue at the end of the test. Thepurpose of the training phase was to familiarise thesubjects with the equipment and the stimuli used inthe test.

Each subject was first presented one attribute withits description. All 13 stimuli were available forlistening by clicking on buttons on the computerscreen. The task was to grade all stimuli one by oneon the attribute presented. This was accomplished byproviding continuos sliders on the screen, one sliderper stimulus. The subjects were instructed to regardthe scale on the sliders as linear. The slider had onlytwo markings, one at each endpoint, the lowermarked “0” (zero) and the upper marked “MAX”.The subject was also instructed to use the MAXgrade for at least one stimulus, but did not necessarilyhave to give any stimulus the grade 0. When thesubject was satisfied with his/her grading on the firstattribute, the scores were stored by clicking a button,whereupon the next attribute was presented. Allstimuli were graded again, but now on the newattribute. This was repeated until all attributes weregraded. When this was completed, the test finished.

L R

C

Ls Rs

30°

110°

Listeningposition

r = 2,0 m

Fig. 3. Loudspeaker set-up



To avoid systematic errors, the presentation orderand assignation of playback buttons were rando-mised: When a session starts the attribute class ischosen randomly. The order of which the attributeswithin the chosen class are presented is also pickedrandomly. When all attributes within the class areassessed by the subject, a new attribute class out ofthe remaining ones is randomly chosen. This isrepeated until all attribute classes with their attributesare assessed. For every new attribute, the assignationof the stimuli to the 13 playback buttons is re-randomised.

Data acquisitionThe slider position representing a subject’s assess-ment of a given stimulus on a given attribute wasconverted into integer numbers from 0 to 100, where0 corresponds to the marking “0” and 100 to “MAX”.The converted grades with proper identification ofsubject, associated stimulus, attribute and date/timewere stored on the computer in one text file persubject. The text files were later converted into MSExcel files for the upcoming analysis.

RESULTS – ANALYSIS OF ATTRIBUTESIGNIFICANCEThe analysis seeks to answer the introductoryquestion in the paper: Is the subject group able tomake significant distinctions between the stimuli inthe test using the provided attributes?

The analysis started with normalisation of the dataand check for normal distribution. Analysis ofvariance (Anova) was used for the significance test.

Data structureThe data acquired consisted of 19 subjects assessing13 stimuli on 12 attributes. This yields 2964 datapoints. Every subject delivered 156 grades.

NormalisingIn order to facilitate the comparison of gradesbetween items, the subjects’ different use of thescales provided must be equalised. This was accom-plished by, for each subject, normalising the gradesgiven to an attribute. This way, each attribute had thesame mean value and standard deviation as the otherattributes. There are 13 stimuli per attribute and themean value

x xik ijkj

==∑1

131

13

and the standard deviation

s x x

x i j k

ik ijkj

ik

ijk

= −

=

=∑1

121

132( )

where

grade given on attribute for item by subject

are used for calculating the z-score

zx x

sijkijk ik

ik=

−

which now is the normalised value of the originalgrade. The mean value of z-scores per subject and perattribute is 0 and the standard deviation is 1.Consequently, the data now consists of normalisedvalues in the form of z-scores suitable for the comingsteps in the analysis.

Normal distribution testTo examine if the subjects’ scores given for eachitem on each attribute were normally distributed, theShapiro-Wilks’ test was performed. Since thesubjects graded 13 stimuli on 12 attributes, thenumber of cases to be tested was 156. The outcomeof this test, expressed as probabilities for normaldistribution for the different cases in the experiment,is found in Appendix B. When the level ofconfidence is set to 95%, the test shows that a normaldistribution can not be excluded in 129 of the 156cases. Since more than 80% of the cases seem to havea normal distribution, this is a first indication of someagreement between the subjects in their grading ofthe stimuli. The existence of normal distribution alsodoes not exclude commonly used statistical methods.

Attribute applicability testAnalysis of variance (Anova) was performed on eachattribute to determine whether it was sufficient or notfor discriminating between stimuli. The dependentvariable was the normalised grade (z-score) and thefactor was stimulus (i tem). Since the data wasnormalised, the F-ratio of the factor subject (subno)became zero, which confirmed that the subject effecthad been removed from the analysis, as intended. Thenull hypothesis



H0 : The attribute provided is not sufficient forenable the subjects to find a significantdifference between any stimuli

and its alternative hypothesis

HA : The attribute provided is sufficient for enablethe subjects to find a significant differencebetween at least one stimulus and the otherstimuli

The analys is sh ow ed th at all th e 1 2 attribu tes h adF-r atio s w ith significance levels p < 0.05. The nullhypothesis was therefore rejected in favour of itsalternative for every attribute. This means that allattributes in the experiment showed to be sufficientfor making distinctions between at least one stimulusand the other stimuli. The attributes must thereforehave some common meaning to the subjects;otherwise, the individual subject differences wouldhave been randomly distributed across the stimuli,resulting in insignificant differences between thestimuli. The Anova tables are found in Appendix B.

Attribute consistency testSince all attributes were found to be sufficient fordiscriminating between stimuli, it is of interest toexamine the attributes for how consistently they weregraded. A relatively high consistency is likely toindicate a more similar perception of the attributethan a relatively low one. To test this, the residual (orerror) variances for the attributes were taken from theprevious Anovas and compared between them. Sincethe between-subject variability was removed earlierfrom the Anova model by the normalisationprocedure, the residual variance only consists of thedifferences in magnitude and direction of the trendsin subject performance. Consequently, a low residualvariance indicates a high consistency in trends [14].

When the attributes’ residual variances wereordered in ascending order and these variances wereinspected, they formed two groups, one withrelatively high variances and another with relativelylow ones. One attribute stood out as being in betweenthe two groups. The group with low residualvariances comprises the attributes (in ascending orderof variances): envelopment, room size, backgroundnoise level, source distance, preference and roomwidth . The group with high residual variancesincludes the attributes in (descending order ofvariances): room spectral bandwidth, localisation,source width, naturalness, and room sound level.Between these groups, the attribute presence is

found. The residual variances and F-ratios are shownin fig 4.

Attribute F-ratio Residual

env 39.54 0.33019

rsz 35.10 0.35783

bgr 33.69 0.36761

dis 32.84 0.37371

prf 28.96 0.40464

rwd 28.77 0.40628

psc 19.90 0.50138

rlv 13.83 0.59701

nat 13.79 0.59776

swd 13.58 0.60161

loc 12.53 0.62226

rsp 10.17 0.67441

Fig. 4. From the Anova: F-ratios and residual variancesof the attributes sorted in ascending order of theirresidual variances.

Mean scores per item and attributeAnalysis of the stimuli themselves was not the mainpurpose of the experiment, but since the attributeswere found to enable subjects to discriminatebetween some stimuli, these were examined to findwhich of the stimuli was separable from the others bythe different attributes. This is shown in fig. 5a-b,where, for every attribute, every stimulus’ meanscore with its associated confidence interval (95%) isplotted. Some observations are commented on here.

The 5-channel and 2-channel stimuli showedhigher scores (not necessarily equal to ‘better’) onmost of the attributes. For some attributes there wereno significant difference between the reproductionmodes of a source, e g room level and backgroundnoise level. If the mono speech stimulus wasdisregarded this applied for room size too.

The 5-channel and 2-channel recordings wereequally scored per source on most of the attributes.The exceptions were on presence and envelopment,where the 5-channel version of both the saxophonesource and the speech source was higher scored thanthe 2-channel version. For preference there was alsoa difference between 5 and 2 channels for speech.



Source distance

Background noise level

Source width

Localisation

Presence

Naturalness

Envelopment

Preference

Fig. 5a. Mean scores on attributes for all items. (8 diagrams)



THE DIMENSIONALITY OF ATTRIBUTESIn order to investigate if some of the attributes usedin the experiment were perceived and thereby likelyto be scored similarly by the subjects, multivariateanalysis of the data were conducted. The methodsused were:• Principal component analysis (PCA) for reducing

the data into fewer components still accounting fora main part of the variance of the scores. In PCAall the variance (both common and unique) of thescores is analysed.

• Principal Factor analysis (FA) with Varimaxrotation of the factors for explaining the reduceddimensionality in terms of the attributes. In FA thecommon variance for the variables is used in theanalysis.

• Cluster analysis for visualising relationshipsbetween the attributes.

• Correlation analysis for calculating the interrelationbetween the attributes.

Principal component analysisA principal component analysis [15] was performedon the set of attributes, which corresponds to thecolumns in the matrix

A =

=

z z

z z

z z

z i j k

jk jk

ijk

1 1 1 12 1 1

1 12

1 13 19 12 13 19

, , , ,

,

, , , ,

L

M M

L

M M

L

where

z - score on attribute for item by subject

and the attributes were normalised before the PCA.The number of components to keep in the analysiscan be determined by the following methods [16]:• Kasier’s criterion – all components with an eigen-

value λ > 1 should be kept in the analysis. Theeigenvalues are shown in fig. 6.

• Cattell’s scree test – a method where a plot ofeigenvalues versus the number of component isinspected. The scree plot is shown in fig. 7.

• Variance dependent – components are brought in tothe analysis until they reach a certain level ofcumulative variance [15], usually 70 or 80 percent.The variances are shown in fig. 6.

Room width

Room size

Room spectral bandwidth

Room sound level

Fig. 5b. Mean scores on attributes for all items (4diagrams)



Component

Number

Eigenvalue Percent of

Variance

Cumulative

Percentage

1 4,83017 40,251 40,251

2 2,59742 21,645 61,897

3 0,84147 7,012 68,909

4 0,78448 6,537 75,446

5 0,68518 5,710 81,156

6 0,50635 4,220 85,376

7 0,36699 3,058 88,434

8 0,33265 2,772 91,206

9 0,28791 2,399 93,605

10 0,27609 2,301 95,906

11 0,26218 2,185 98,091

12 0,22910 1,909 100,000

Fig 6. Extraction statistics from the PCA

The scree plot (fig. 7) shows that three componentsshould be considered in the subsequent analysis.Their cumulative variance was 68,9%.

An inspection of the component weights (fig. 8) orloadings showed that the component accounting forthe highest variance, component 1, was positivelyloaded by all attributes except for localisation. Roomwidth, envelopment and source width together withpreference loaded component 1 mostly. Component 2showed the most positive loading for distance and themost negative for presence. At the positive end ofcomponent 3, source width were found, and at itsnegative, background noise and localisation. Notablewas that source width and localisation seemed to beopposites in any combination of the dimensions.Plots of components weights are found in AppendixC.

The extraction of components in a PCA considersall variance, so the components are likely to consistof more complex functions of the variables (than aFA), which could make the components harder tointerpret [17]. Nevertheless, a pattern can be

discovered; Component 1 could be interpreted as awidth in a general meaning, since the attributesloading the component describes a perceived widthand a feeling of being surrounded by sound. Awider(!) interpretation is an experience of a “biggerevent” or perhaps a larger listening area. Component1 also represents a positive attitude towards thesound. Component 2 seems to account for distance tothe sound since the opposites are source distance –presence and the intermediate attributes follows a‘logical’ order from distance through room attributesvia width to envelopment, stopping at presence. Theloadings on component 3 indicated the oppositebetween source width and localisation. The subjectsseemed to interpret the attributes forming thisdimension as if the source gets wider it becomesharder to localise, which most likely is an imagefocus perception.

Attribute Component 1 Component 2 Component 3

nat 0.271 -0.345 -0.236

psc 0.261 -0.402 -0.151

prf 0.338 -0.233 -0.108

env 0.370 -0.160 -0.185

bgr 0.176 0.313 -0.504

rwd 0.390 0.105 0.013

rsz 0.316 0.304 -0.267

rsp 0.269 -0.179 0.234

rlv 0.205 0.384 0.042

swd 0.326 0.039 0.556

loc -0.305 -0.172 -0.425

dis 0.103 0.478 -0.059

Fig 8. PCA: Component weights on attributes

Factor AnalysisA principal factor analysis was performed on thescore matrix A, where the scores were normalisedprior to the FA in the same way as in the PCA. Factoranalysis is used when an accurate description of thedomain covered by the variables is desired [17]. Thenumber of factors was determined by the same crite-rion as in the PCA and was accordingly set to three.To increase the interpretability, the factors wererotated, using Varimax, to maximise the loadings ofsome of the attributes. These attributes can then beused to identify the meaning of the factors [16]. Thefactor loadings are presented in fig. 9.

After the Varimax rotation factor 1 was loadedpositively by all of the attributes in the Generalattributes class (referring to the sound as an entity):naturalness, presence, preference and envelopment.Factor 2 showed a high loading by room size, back-Fig. 7. Scree plot for the PCA

λ

component number



ground noise level, source distance, room sound leveland intermediate loading by room width. For factor 3,source width loaded strongly positive and locali-sation almost as much but with a negative sign. Plotsof factor loadings are found in Appendix C.

Attribute Factor 1 Factor 2 Factor 3

nat 0.841 -0.036 0.057

psc 0.859 -0.152 0.094

prf 0.787 0.113 0.266

env 0.785 0.266 0.267

bgr 0.099 0.776 -0.079

rwd 0.480 0.520 0.511

rsz 0.247 0.806 0.266

rsp 0.522 -0.041 0.452

rlv -0.098 0.655 0.385

swd 0.284 0.141 0.822

loc -0.156 -0.336 -0.735

dis -0.319 0.707 0.212

Fig 9. Factor analysis: Factor loadings on attributesafter Varimax rotation

From the factor analysis it was noted that factor 1corresponded well to all of the attributes in theGeneral attribute class, whereas factor 2, with one

exception (source distance), seemed to describe theroom attributes. Factor 3 accounted for the sourceattributes; especially those describing something thatcan be interpreted as image focus. An alternativeinterpretation is that factor 1 is a more ‘attitudinal’factor where a positive loading indicates both anappreciation of the sound and that enveloping soundsare an important part of a natural and preferableexperience. As an alternative approach for factor 2, itcould be interpreted as a distance factor, since anincreased distance from the source in a room wouldchange the balance between the direct and thereflected sound and thus increase the audibility of theroom.

Cluster analysisCluster analysis [18] compares and group similarvariables according to the metrics and agglomerationtechnique used. A useful output from a clusteranalysis is the dendrogram, which displays thesimilarity in the form of a tree where the similaritybetween variables are indicated by ‘branches’ joiningthe variables at the point of similarity. The metricsused was squared Euclidean and the agglomerationtechnique was Complete linkage. The resultingdendrogram is shown in fig 10, The agglomerationdistance plot (not shown here), also used in [10] for

Fig. 10. Cluster analysis presented in dendrogram form. The closer to the baseline the attributes are joined,the more similar they are.



determining the appropriate number of clusters(groups) indicated three groups of attributes. Thesegroups with their attributes were:

Group 1NaturalnessPresencePreferenceEnvelopmentRoom spectral bandwidthSource width

Group 2Background noise levelRoom widthRoom sizeRoom levelSource distance

Group 3Localisation

An inspection of the dendrogram reveals subgroupswithin the three groups resulting from theagglomeration distance plot; naturalness, presence,preference and envelopment show a strong relation-ship as well as room width and room size.

Correlation analysisFinally, to get a complete picture of the correlationbetween the attributes, the Pearson product momentcorrelation coefficient, r [19] was calculated. Theresults are given as a coefficient for every combina-tion of the attributes. The correlation coefficients arefound in Appendix B. The strongest correlations( r > 0.6) are found between

naturalness – presence r = 0.727preference – envelopment r = 0.674room width – room size r = 0.646room width – envelopment r = 0.613presence – envelopment r = 0.611source width – localisation (negative) r = -0.602

The lowest correlations are between

naturalness – room sound level r = -0.022envelopment – source distance r = 0.03background noise – room spec. bandw. r = 0.059preference – room sound level r = 0.061presence – background noise r = -0.068

This analysis also verifies the relatively strong inter-relation between envelopment and the attributesexpressing naturalness and a feeling of presence.

DISCUSSION AND CONCLUSIONSThe experiment’s aim was to, if possible, validate thefindings in a previous experiment as well as to under-stand more of how the attributes encountered wereworking as descriptors for assessing the spatialquality of reproduced audio. In order to drawconclusions from any experiment, its limitations mustas far as possible be known to those involved. Theexperiment at hand made use of recordings of singlesound sources in an acoustical environment. There isno imagination needed to realise that the selection isnarrow, compared to all recordings available. This ofcourse reduces the generalisability of the results toother types of sound. The decision to use such recor-dings was a result of some difficulties encountered inthe previous experiment concerning complex soundsources. There is a high factor of ‘reality’ in usingcomplex sounds, but from the experimenter’s point ofview there is a problem in having too manyuncontrolled conditions and a risk to confuse theparticipating subjects. On the other hand, it ispossible to use very simple stimuli but with littleconnection to what most people normally listens tovia loudspeakers, which is sometimes encountered inclassical psychophysics. This experiment tries tosatisfy both a relatively low complexity as well assound stimuli that could be found outside the labora-tory.

The null hypothesis in the Anova was that if theattributes provided did not make sense or did nothave any common meaning to the subjects, the resulton the different attributes would have consisted ofrandomly distributed scores yielding noise only in thedata. Such a result would have invalidated thealternative hypothesis, partly (for some attributes) orin total (all attributes). As stated in the analysissection, every attribute was able to producesignificant differences on the stimuli selected. Thismeans that every attribute has some commonmeaning to the group of subjects under theexperiment conditions.

Although all attributes showed significant F-ratios,it became clear that some attributes are moreconsistently scored than others. If this is due to dif-ferences in perception or variability in the interpreta-tion of the written descriptions of the attributes isimpossible to tell. Most consistently graded wereenvelopment, room size, background noise, sourcedistance, preference, and room width. Least consis-



tently graded were: room spectral bandwidth, whichboth had a short description that may have beeninsufficient for the subjects, in combination with thepossible difficult task to quantify the bandwidth ofthe room sound. The scores on room spectralbandwidth separate the stimuli in basically twohalves; mono and 2/5-channel reproduction, whicheither is a perceived difference in bandwidth or aninterpretation of some other width attribute. Otherattributes with low consistency were localisation,source width, naturalness and room level. Martin etal [20] points at the problem with defining sourcewidth, where there is a risk of confusion betweennarrower image width, increased distance to thesource and less spread of low frequency content. Theresults from this experiment seem to confirm theirfindings.

Some attributes seem to be more independent fromthe number of channels used. The different reproduc-tion modes of a source were not able to cause anysignificant differences in mean score of the sourcewithin the attributes room level and backgroundnoise. When the mono stimulus is disregarded, thisalso applies to room size. This suggests that theproperties described with these attributes are noteasily corrupted by alterations by means of down-mixing.

Looking at the large trends in the form of dimen-sionality, the three dimensions extracted from thePCA and the FA mainly leave us with factorsrepresenting three attribute classes. The first factorseems to contain attributes concerning general widthaspects, naturalness and preference. The secondcontains an element of decreasing distance fromsource distance and room level to presence. The thirdfactor comprises source image focus. This wouldind icate that the sub jects actually per ceive the attributesclasses mainly as orthogonal on the dimensionsGeneral, Source and Room. The remaining class,Oth er, lo ads th e Room facto r. The divis ion of attributesinto groups were known to the subjects prior to theexperiment so this may have biased them. A looserinterpretation is that factor 1 represents an attitudinaldimension containing a wide and enveloping sound.This may not be the case, neither when listeners getmore experienced and able to be more precise in theirdiscrimination, nor when the differences in terms ofspatial quality between stimuli decreases.

The cluster analysis and the correlation analysisboth confirm the trends above in greater detail. Thegeneral attributes naturalness and presence show thehighest correlation, followed by preference andenvelopment.�

In conclusion, the experiment shows that verbalattributes still are a valid approach for describingspatial quality. The findings validate the experiencesfrom the 1998 experiment, showing that theseattributes have a common meaning under theconditions of the experiment. Some attributes aremore consistently used by the subjects, otherattributes are less sensitive to changes in reproductionmodes. The use of attribute classes in order to focus asubject onto different parts of the auditory eventseems to be a successful approach.

Future workThe attributes used emerged from a techniquewithout provided constructs to encourage the subjectsin the previous experiment to find and use wordsfamiliar to them. Via PCA and cluster analysis acommon meaning of these constructs was sought, andattributes appropriate to describe them were formu-lated. One idea for future work is to re-iterate thatprocess, since we now know more about stimuli andperceivable dimensions, with the aim of finding moreand new attributes. Another approach is that, basedon the experiences we have, try to refine the attributedescriptions and repeat an experiment to see if thescoring consistency improves

To carefully examine the use of attributes, thedifference between stimuli can be decreased andmore precisely controlled. This will make it possibleto observe whether the scales depending on certainattributes are still valid under new conditions. Thesedifferences could be created in the recording domain,e g by means of different microphone techniques,without changing the modes of reproduction.

In an experiment, the subjects could also beprovided with reference stimuli to furthermorenarrow down the meaning of the attributes

ACKNOWLEDGEMENTSThe authors wish to thank the students at the Schoolof Music, Piteå, Sweden for their participation in thisexperiment. The test equipment was constructed as ajoint project between the School of Music andSveriges Radio (Swedish Broadcasting Corporation)in which Ola Kejving and Lars Mossberg are thankedfor their support. A special thanks to Jonas Ekeroot,Sveriges Radio, for the construction of the testequipment as well as for the software programmingwhich made this experiment possible.



REFERENCES1 Berg, J. and Rumsey, F. (1999) Spatial attribute

identification and scaling by Repertory GridTechnique and other methods. In Proceedings ofthe AES 16th International Conference on SpatialSound Reproduction, 10–12 Apr. pp 51-66. AudioEngineering Society.

2 Rumsey, F. (1998) Subjective assessment of thespatial attributes of reproduced sound. InProceedings of the AES 15th InternationalConference on Audio, Acoustics and SmallSpaces, 31 Oct–2 Nov, pp. 122–135. Audio Engi-neering Society

3 Kelly, G. (1955) The Psychology of PersonalConstructs. Norton, New York.

4 Danielsson, M. (1991) Repertory Grid Technique.Research report. Luleå University of Technology.1991:23

5 Fransella, F. and Bannister, D. (1977) A manualfor Repertory Grid Technique. Academic Press,London.

6 Stone, H. et al (1974) Sensory evaluation byquantitative descriptive analysis, FoodTechnology, November, pp 24-34.

7 Mason, R., Ford, N., Rumsey, F. and de Bruyn, B.(2000) Verbal and non-verbal elicitationtechniques in the subjective assessment of spatialSound Reproduction. Presented at AES 109th

Convention, Los Angeles. Preprint 5225.

8 Wenzel, E. M. (1999) Effect of increasing systemlatency on localization of virtual sounds. In Pro-ceedings of the AES 16th InternationalConference on Spatial Sound Reproduction,10–12 Apr. Audio Engineering Society. pp 42-50.

9 Berg, J. and Rumsey, F. (1999) Identification ofperceived spatial attributes of recordings byrepertory grid technique and other methods.Presented at AES 106th Convention, Munich.Preprint 4924.

10 Berg, J. and Rumsey, F. (2000) In search of thespatial dimensions of reproduced sound: VerbalProtocol Analysis and Cluster Analysis of scaledverbal descriptors. Presented at AES 108thConvention, Paris. Preprint 5139.

11 Berg, J. and Rumsey, F. (2000) Correlationbetween emotive, descriptive and naturalnessattributes in subjective data relating to spatialsound reproduction. Presented at AES 109thConvention, Los Angeles. Preprint 5206.

12 Surround Sound Forum, Balance and Media City(1998) Multichannel Universe. DVD BAL-95000-3. Balance, Munich.

13 ITU-R (1996) Recommendation BS.-1116,Methods for the subjective assessment of smallimpairments in audio systems includingmultichannel sound systems. International Tele-communication Union.

14 Roberts, M. J. and Russo, R. (1999) A student’sguide to analysis of variance. Routledge, London.

15 Johnson, R. A. and Wichern, D. W., (1998)Applied multivariate statistical analysis. Prentice-Hall, New Jersey.

16 Bryman, A. and Cramer, D. (1994) Quantitativedata analysis for social scientists. Routledge,London.

17 Cureton, E. E. and D’Agostino, R. B. (1983)Factor Analysis – an applied approach. LawrenceErlbaum, New Jersey.

18 Everitt, B. S. and Dunn, G. (1991) Applied Multi-variate Data Analysis. Edward Arnold, London

19 Devore, J. L. and Peck, R. (1986) Statistics, theexploration and analysis of data. West PublishingCompany, S:t Paul.

20 Martin, G., Woszczyk, W., Corey, J. and Quesnel,R. (1999) Controlling phantom image focus in amultichannel reproduction system. Presented atAES 107th Convention, New York. Preprint 4996.

APPENDIX A

A T T R I B U T E S T O A S S E S S I N L I S T E N I N G T E S T M A R C H 2 0 0 1

G E N E R A L NaturalnessGA1

How similar to a natural (i.e. not reproduced through e g loudspeakers) listeningexperience the sound as a whole sounds. Unnatural = low value. Natural = highvalue.

PresenceGA2

The experience of being in the same acoustical environment as the sound source,e g to be in the same room. Strong experience of presence = high value.

PreferenceGA3

If the sound as a whole pleases you. If you think the sound as a whole soundsgood. Try to disregard the content of the programme, i e do not assess genre ofmusic or content of speech. Prefer the sound = high value.

EnvelopmentGA4

The extent of how the sound as a whole envelops/surrounds/exists around you.The feeling of being in the centre of the sound. Feel enveloped = high value.

S O U N D S O U R C E Source widthSA1

The perceived width of the source. The angle occupied by the source. Does notnecessarily indicate the known size of the source, e g one knows the size of apiano in reality, but the task to assess is how wide the sound from the piano isperceived. Disregard sounds coming from the sound source’s environment, e greverberation – only assess the width of the sound source.Narrow sound source = low value. Wide sound source = high value.

LocalisationSA2

How easy it is to perceive a distinct location of the source – how easy it is topinpoint the direction of the sound source. Its opposite (a low value) is when thesource’s position is hard to determine – a blurred position. Easy to determine thedirection = high value.

Source distanceSA3

The perceived distance from the listener to the sound source.Short distance/close = low value. Long distance = high value.

R O O M Room widthRA1

The width/angle occupied by the sounds coming from the sound source’sreflections in the room – not the sound source itself. Narrow room = low value.Wide room = high value.

Room sizeRA2

In cases where you perceive a room/hall, this denotes the relative size of thatroom. Large room = high value. If no room/hall is perceived, this should beassessed as zero.

Room spectralbandwidth RA3

The perceived bandwidth of the room. Room with large bandwidth = high value.

Room sound levelRA4

The level of sounds generated in the room as a result of the sound source, e greverberation – i e not extraneous disturbing sounds. Weak room sounds = lowvalue. Loud room sounds = high value.

O T H E R Backgroundsound level OA1

The level of sounds not generated by the sound source itself. Weak backgroundnoises = low value. Loud background noises = high value.



APPENDIX B

Tables• Test for normal distribution• Attribute correlation• Analysis of variance

item ntl psc prf env bgr rwd rsz rsp rlv swd loc dis

1 0,3184 0,2366 0,2066 0,1137 0,1257 0,2251 0,0019 0,4722 0,2308 0,6900 0,5824 0,0120

2 0,2511 0,7942 0,1344 0,2476 0,1065 0,1159 0,0012 0,7649 0,2965 0,9456 0,3314 0,0068

3 0,7363 0,3397 0,4260 0,0317 0,0025 0,1266 0,0806 0,3680 0,3661 0,4145 0,4519 0,8567

4 0,2448 0,4480 0,6034 0,6680 0,5505 0,2091 0,9945 0,3486 0,3945 0,7444 0,0821 0,5020

5 0,9873 0,7216 0,9604 0,5530 0,0856 0,7024 0,1265 0,4944 0,1836 0,7645 0,9079 0,0278

6 0,0874 0,0021 0,2199 0,4455 0,9296 0,0747 0,8143 0,1827 0,2166 0,0106 0,0041 0,5780

7 0,0183 0,8332 0,3395 0,0876 0,0891 0,7273 0,7654 0,8216 0,8291 0,1899 0,0643 0,4131

8 0,1849 0,0074 0,8695 0,8210 0,6496 0,0519 0,9885 0,7790 0,9635 0,1539 0,6262 0,6831

9 0,0316 0,0308 0,5134 0,3739 0,1258 0,8123 0,0834 0,2877 0,3490 0,8342 0,0006 0,0181

10 0,1596 0,0066 0,0081 0,2906 0,3148 0,0013 0,0193 0,2907 0,5073 0,2283 0,6420 0,2560

11 0,0107 0,0341 0,3358 0,0115 0,2598 0,2001 0,6974 0,4186 0,1521 0,6506 0,0035 0,0801

12 0,3118 0,4028 0,9668 0,0672 0,4059 0,6790 0,1506 0,7505 0,4587 0,5708 0,0259 0,0019

13 0,2540 0,0943 0,3226 0,2275 0,2583 0,0970 0,4732 0,3269 0,4855 0,0392 0,0830 0,5613

Shapiro-Wilks’ test for normal distribution; p-values are shown.

nat psc prf env bgr rwd rsz rsp rlv swd loc dis

nat 0,727 0,593 0,595 -0,091 0,386 0,204 0,387 -0,022 0,252 -0,203 -0,141

psc 0,727 0,595 0,611 -0,068 0,368 0,095 0,386 -0,075 0,301 -0,200 -0,314

prf 0,593 0,595 0,674 0,137 0,538 0,336 0,483 0,061 0,454 -0,353 -0,077

env 0,595 0,611 0,674 0,256 0,613 0,428 0,455 0,208 0,483 -0,434 -0,030

bgr -0,091 -0,068 0,137 0,256 0,370 0,523 0,059 0,316 0,218 -0,272 0,314

rwd 0,386 0,368 0,538 0,613 0,370 0,646 0,458 0,486 0,587 -0,558 0,285

rsz 0,204 0,095 0,336 0,428 0,523 0,646 0,276 0,553 0,397 -0,465 0,527

rsp 0,387 0,386 0,483 0,455 0,059 0,458 0,276 0,090 0,434 -0,222 -0,108

rlv -0,022 -0,075 0,061 0,208 0,316 0,486 0,553 0,090 0,303 -0,383 0,509

swd 0,252 0,301 0,454 0,483 0,218 0,587 0,397 0,434 0,303 -0,602 0,129

loc -0,203 -0,200 -0,353 -0,434 -0,272 -0,558 -0,465 -0,222 -0,383 -0,602 -0,336

dis -0,141 -0,314 -0,077 -0,030 0,314 0,285 0,527 -0,108 0,509 0,129 -0,336

Pearson’s product moment correlation coefficients for the correlation between attributes.



APPENDIX B - CONTINUED

Attribute Factor Sums of

squares

Degrees

offreedom

Mean square F-ratio p

Naturalness nat A:item 98,8848 12 8,2404 13,79 0,0000

B:subno 0 18 0 0,0000 1,0000

RESIDUAL 129,115 216 0,597756

Presence psc A:item 119,702 12 9,97513 19,9 0,0000

B:subno 0 18 0 0,0000 1,0000

RESIDUAL 108,298 216 0,501382

Preference prf A:item 140,598 12 11,7165 28,96 0,0000

B:subno 0 18 0 0,0000 1,0000

RESIDUAL 87,4021 216 0,404639

Envelopment env A:item 156,678 12 13,0565 39,54 0,0000

B:subno 0 18 0 0,0000 1,0000

RESIDUAL 71,322 216 0,330194

Background bgr A:item 148,597 12 12,3831 33,69 0,0000

noise B:subno 0 18 0 0,0000 1,0000

RESIDUAL 79,4033 216 0,367608

Room width rwd A:item 140,244 12 11,687 28,77 0,0000

B:subno 0 18 0 0,0000 1,0000

RESIDUAL 87,7561 216 0,406278

Room size rsz A:item 150,708 12 12,559 35,1 0,0000

B:subno 0 18 0 0,0000 1,0000

RESIDUAL 77,2917 216 0,357832

Room spectral rsp A:item 82,3273 12 6,86061 10,17 0,0000

bandwidth B:subno 0 18 0 0,0000 1,0000

RESIDUAL 145,673 216 0,67441

Room sound rlv A:item 99,0458 12 8,25381 13,83 0,0000

level B:subno 0 18 0 0,0000 1,0000

RESIDUAL 128,954 216 0,59701

Source width swd A:item 98,0533 12 8,17111 13,58 0,0000

B:subno 0 18 0 0,0000 1,0000

RESIDUAL 129,947 216 0,601605

Localisation loc A:item 93,5909 12 7,79924 12,53 0,0000

B:subno 0 18 0 0,0000 1,0000

RESIDUAL 134,409 216 0,622264

Source distance dis A:item 147,279 12 12,2733 32,84 0,0000

B:subno 0 18 0 0,0000 1,0000

RESIDUAL 80,7207 216 0,373707

Analysis of variance for attributes.



APPENDIX C

Principal components plots

Plots of the three extracted components from the PCA.



APPENDIX C continued

Factor analysis plots

Plots of the three rotated factors in the factor analysis

Verification and correlation of attributes used for ... · 2Institute of Sound Recording,...

Documents

Transcript of Verification and correlation of attributes used for ... · 2Institute of Sound Recording,...