A Method for Spatial Upsampling of Directivity Patterns of Human Speakers

by Directional Equalization

Christoph Porschmann1, Johannes M. Arend1,2

1 TH Koln, Institute of Communications Engineering, Cologne, Germany2 TU Berlin, Audio Communication Group, Berlin, Germany

Email: christoph.poerschmann@th-koeln.de

Introduction

In many everyday situations we experience the influenceof the directivity of human speakers. We perceive loud-ness and timbre significantly different when one faces usor when one turns away from us. An important spe-cific aspect of human speech production is its dynamicdirectivity, i.e. time-variant alterations of the directiv-ity which occur while speaking (e.g. [1, 2]) or whilesinging (e.g. [3]). To determine the directivity pattern ofa speaker or singer, the sound radiation into an appro-priately large number of directions needs to be captured.For a reproduction in virtual acoustic environments, mea-surements on a spherical sampling grid are advantageous.Generally, these measurements can be performed sequen-tially for an arbitrary number of directions. Alterna-tively, with a surrounding microphone array [1, 4, 5] themeasurements can be performed simultaneously. By this,loudness or articulation-depending influences on the di-rectivity of natural speakers can be investigated. How-ever, as the setup of such surrounding arrays is restrictedto a limited number of microphones, the spatial resolu-tion is low. Applying the principle of reciprocity allows toregard the radiation from a distinct point on the spherein the same way as a sound wave reaching the sphere.By this, upsampling sparse directivity sets which are ob-tained from measurements with a low spatial resolution,can be handled comparably to measured sets of head-related transfer functions (HRTFs). For HRTFs manyinterpolation methods have been elaborated. HRTF setscan be described in the spherical harmonics (SH) domain[6]. In this case, the HRTFs are decomposed into spher-ical base functions of different orders N , where higherorders correspond to a higher spatial frequency. Spatialupsampling can be applied by evaluating the SH func-tions at the corresponding directions.

Recently, we presented the SUpDEq (Spatial Upsamplingby Directional Equalization) method [7] which removesdirectional components from HRTF sets before SH trans-form. The directivity set is spatially equalized by a divi-sion with a corresponding rigid sphere transfer function(STF), which can be regarded as a simplified directiv-ity set only comprising basic temporal and spectral fea-tures. After spatial upsampling (SH interpolation), a de-equalization by means of a spectral multiplication withthe same STFs is performed and thus a spatially upsam-pled directivity set is recovered. The proposed SUpDEqmethod and its application to human speaker directivi-ties is described in greater detail in this paper.

Method

A directivity pattern can be described by the frequency-depending pressure function pDIR(ω,Ωg) measured at Gdiscrete angles Ωg = (φ1, θ1), . . . , (φG, θG) at azimuthφ, and elevation θ at a predefined distance in the far-field.The corresponding SH coefficients fnm(ω) are obtainedvia the SH transform, often referred to as spatial (orspherical) Fourier transform [6, p. 2]

fnm(ω) =

G∑g=1

pDIR(ω,Ωg)Ymn (Ωg)

∗βg , (1)

sampling weights βg depend on the type of the grid. Thecomplex conjugation is denoted by (·)∗ and the sphericalharmonis of order n and mode/degree m by

Y mn (θ, φ) =

√2n+ 1

4π

(n−m)!

(n+m)!Pmn (cosθ)eimφ , (2)

with the associated Legendre functions Pmn , and i =√−1

the imaginary unit. The inverse spatial Fourier transform

pDIR(ω,Ω) =

N∑n=0

n∑m=−n

fnm(ω)Y mn (Ω) , (3)

can be used to recover pDIR at arbitrary angles. N de-notes the maximal order. If pDIR is strictly order-limited,a sufficient choice of N results in pDIR = pDIR. In casethe order of pDIR exceeds N , order-limitation errors andspatial aliasing occurs. Depending on the spatial sam-pling grid Ωg, the coefficients fnm can be calculated up toa maximum order N without suffering of spatial aliasing.In this case, the number of measured directionsG directlycorresponds to the maximum order N by G ∝ (N + 1)2.Consequently, sparse directivity patterns result in a lim-ited SH order. To avoid spatial aliasing for the full audiobandwidth, a maximum order N ≥ kr with k = ω/c, andr being the head radius is required. Thus, an appropri-ate pre-processing that reduces the spatial complexity ofpDIR will ease the requirement on G. The following sec-tion describes the SUpDEq method applied to directivitypatterns of human speakers. According to the principleof reciprocity the SUpDEq method [7] can be applied tothe sound radiation from the human mouth as shown inFig. 1. First, the sparse directivity set pDIR measuredat S sampling points Ωs = (φ1, θ1), . . . , (φS , θS) isequalized with an appropriate equalization dataset HEQ

pDIR,EQ(ω,Ωs) =pDIR(ω,Ωs)

HEQ(ω,Ωs). (4)

DAGA 2019 Rostock

1458

Equalization De-Equalization

Sparse Sampling

Grid

Dense Sampling

Grid

Nlow

SFT

Nlow

ISFT

Nhigh

ISFT

Equalization De-Equalization

Nhigh

ISFT

EqualizationDataset

SH Coecients

Nhigh

De-EqualizationDataset

SH Coecients

Nhigh

SparseDirectivity Set

DenseDirectivity Set

Figure 1: Block diagram of spatial upsampling of a speaker directivity set with the SUpDEq method. Left panel: A directivityset is equalized on the corresponding sparse sampling grid. The set is then transformed to SH domain with N = Nlow. Rightpanel: The equalized set is de-equalized on a dense sampling grid, resulting in a dense directivity set.

The equalization dataset reduces the directional depen-dency in pDIR to a certain degree and can be regarded asa simplified directivity set which features basic temporaland spectral components but does not carry informationon the fine structure of the head. In this study a rigidsphere transfer function is used [6, p. 227]:

HSTF(ω,Ωg) = P4π

Nhigh∑n=0

n∑m=−n

injn(kr)Y mn (Ωe)Ymn (Ωg)

∗,

(5)

with P denoting an arbitrary sound pressure, jn thespherical Bessel function of the first kind, and the mouthposition Ωe. We determined in informal tests an optimalΩe of φ = 0 and θ = −25 which is in-line with otherstudies [8]. The radius r should match the physical di-mensions of a human head. As the equalization datasetis based on an analytic description, it can be determinedat a freely chosen maximal order, typically, a high or-der Nhigh ≥ 35. In a second step SH coefficients fEQ,nmfor the equalized sparse directivity set are obtained byapplying the SH transform according to Eq. (1) on theequalized directivity dataset given by Eq. (4) up to anappropriate low maximal order Nlow. Then, in a thirdstep an upsampled directivity set pDIR,EQ is calculatedon a dense sampling grid Ωd = (φ1, θ1), . . . , (φD, θD),with D S by using the inverse SH transform describedby Eq. (3). Finally, the directivity is resored by a subse-quent de-equalization by means of spectral multiplicationwith a de-equalization dataset HDEQ:

pDIR,DEQ(ω,Ωd) = pDIR,EQ(ω,Ωd) ·HDEQ(ω,Ωd) . (6)

This last step recovers energies at higher spatial ordersthat were transformed to lower ones in the first step.For de-equalization the STF as given in Eq. (5) is used.Again, pDIR = pDIR,DEQ holds if Nlow and Nhigh are cho-sen appropriately. Otherwise, deviations will be caused

by signal energy which, after the equalization, still is ap-parent at high modal orders N > Nlow. Due to spa-tial aliasing, this signal energy is irreversibly mirrored tolower orders, and we obtain pDIR ≈ pDIR,DEQ. In thefollowing section these influences are analyzed and theSUpDEq method is compared to SH interpolation with-out any pre- or postprocessing.

Materials

For evaluation we measured the speaker directivity of aHEAD acoustics HMS II.3 head and mouth simulator ona dense grid in the anechoic chamber at TH Koln. Wecalculated the head radius according to [9] resulting inr = 8.78 cm. We applied a sequential measurement andused the VariSphear measurement system [10] for precisepositioning of the dummy head at the spatial samplingpositions and for capturing the impulse responses. Asmicrophone we used a Microtech Gefell M296S and mea-sured at a distance of 2 m. An RME Babyface was usedas AD / DA converter and microphone preamp. The ex-citation signal for all measurements was an emphasizedsine sweep with 218 samples at 48 kHz sampling rate.Generally, the measurement procedure and the subse-quent postprocessing is comparable to the HRTF mea-surements described in [11]. We measured a reference seton a Lebedev full spherical grid with 2702 points, whichwas transformed to the SH domain with N = 35. To gen-erate various sparse sets, we simply spatially subsampledthe reference set in SH domain by means of the inverseSH transform on a Lebedev grid Ωs (Nlow = 4, 7, 10, 13;corresponding to 38, 86, 170, 266 positions).

Evaluation

In the following we compare the high density reference set(Nhigh = 35) to sparse sets processed with SUpDEq andto order-limited (OL) directivities, obtained by meansof SH interpolation without pre- or post-processing. We

DAGA 2019 Rostock

1459

0°

30°

60°

90°

120°

150°

180°

210°

240°

270°

300°

330°

-18

-12

-6

0 dB

(a)

0°

30°

60°

90°

120°

150°

180°

210°

240°

270°

300°

330°

-18

-12

-6

0 dB

(b)

0°

30°

60°

90°

120°

150°

180°

210°

240°

270°

300°

330°

-18

-12

-6

0 dB

(c)

0°

30°

60°

90°

120°

150°

180°

210°

240°

270°

300°

330°

-18

-12

-6

0 dB

(d)

90°

60°

30°

Front

-30°

-60°

-90°

-60°

-30°

Back

30°

60°

-18

-12

-6

0 dB

(e)

90°

60°

30°

Front

-30°

-60°

-90°

-60°

-30°

Back

30°

60°

-18

-12

-6

0 dB

(f)

90°

60°

30°

Front

-30°

-60°

-90°

-60°

-30°

Back

30°

60°

-18

-12

-6

0 dB

(g)

90°

60°

30°

Front

-30°

-60°

-90°

-60°

-30°

Back

30°

60°

-18

-12

-6

0 dB

(h)

OL-N4 OL-N7 OL-N10 OL-N13 DEQ-N4 DEQ-N7 DEQ-N10 DEQ-N13 Reference

Figure 2: Radiation pattern in the horizontal plane (a – d) and the vertical plane (e – h) for the order-limited sets (red) andSUpDEq-processed sets (blue). Plotted are Nlow = 4, 7, 10, 13 and the reference directivity set (black, dashed) in dB. (a) and(e): 1 kHz band, (b) and (f): 2 kHz band, (c) and (g): 4 kHz band, and in (d) and (h): 8 kHz band.

used a Matlab-based implementation of SUpDEq, whichhas been presented in [7].

In a first step, the directivity pattern was analyzed in oc-tave bands in the horizontal and vertical plane. As shownin Fig. 2 in the octave bands around 1 kHz and 2 kHz onlyslight differences between the different sparse grids andthe reference occur and are only relevant for Nlow = 4, 7for rear directions. At 4 kHz for the order-limited set de-viations arise for Nlow = 7 and spread at Nlow = 4 overvarious radiation directions both in the horizontal andmedian plane. For the de-equalized (DEQ) set, which isthe spatially upsampled dataset after the SUpDEq pro-cessing, differences to the reference are mainly observablefor rearward directions. While for the order-limited ra-diation patterns at Nlow = 4, 7 in the 8 kHz octave banddeviations of 10 dB and more occur for various directions,the deviations are for the de-equalized set limited to di-rections to the rear and the top. It can be summarizedthat differences to the reference are much larger for theorder-limited patterns than for the de-equalized ones.

To determine the deviations to a reference set over allT measured directions Ωt = (φ1, θ1), . . . , (φT , θT ) of atest sampling grid, we calculated the spectral differencesaveraged across all 2702 measured directions:

∆Gf (ω) =1

NΩt

∑Ωt

|20lg|pDIR,REF(ω,Ωt)||pDIR,TEST(ω,Ωt)|

|, (7)

with pDIR,REF the directivity extracted from fREF,nm

and pDIR,TEST the respectively processed sparse directiv-

ity. Figure 3 illustrates the frequency-dependent spectraldifferences ∆Gf (ω) at Nlow = 4, 7, 10, 13 for the SUpDEqmethod and for an strictly order-limited interpolation. Itcan be observed that the spectral differences are signif-icantly smaller for the SUpDEq method than for order-limited interpolation. Furthermore, for order-limited in-terpolation the spectral differences increase distinctivelyabove 2 dB at aliasing frequencies between 2 and 6 kHz,depending on N . For the SUpDEq method, the spectraldifferences are generally lower and show a much moregentle rise. Here, for orders Nlow = 7, 10, 13, differencesare below or about 2 dB for frequencies up to 10 kHz.

Finally, the spatial distribution of the differences was in-

1k 10k

Frequency [Hz]

0

2

4

6

8

Magnitude [dB

]

OL-N4

OL-N7

OL-N10

OL-N13

DEQ-N4

DEQ-N7

DEQ-N10

DEQ-N13

Figure 3: Spectral differences ∆Gf (ω) in dB between ref-erence directivity set and the order-limited sets (red) andSUpDEq-processed sets (blue) for Nlow = 4, 7, 10, 13.

DAGA 2019 Rostock

1460

180 270 0 90 180Azimuth [°]

90

45

0

-45

-90

Elev

atio

n [°

]

0

5

10

Magnitude [dB]

(a)

180 270 0 90 180Azimuth [°]

90

45

0

-45

-90

Elev

atio

n [°

]

0

5

10

Magnitude [dB]

(b)

Figure 4: Spectral differences ∆Gsp(Ωt) per sampling pointto the reference directivity set for order-limited interpolation(a) and for the SUpDEq method (b) at N = 4 (f ≤ 8 kHz).

vestigated and the directional deviation across all fre-quencies was calculated as

∆Gsp(Ωt) =1

Nω

∑ω

| 20lg| pDIR,REF(ω,Ωt) || pDIR,TEST(ω,Ωt) |

| . (8)

Fig. 4 shows the spectral differences ∆Gsp(Ωt) per sam-pling point for order-limited interpolation and for theSUpDEq method at N = 4 and f ≤ 8 kHz. In this case,the test sampling grid Ωt is full spherical and calculatedfor φ and θ in steps of 1. The plots show that, indepen-dent of the method for spatial upsampling, the spectraldifferences are maximal for directions to the rear. Gen-erally, the order-limited interpolation results in distinctspectral differences spread over the entire angular range,while the SUpDEq method leads to differences mainly forsound radiation to the rear.

Conclusion

We presented an SH based approach for spatial upsam-pling (interpolation) of sparse human speaker directiv-ities, and applied the SUpDEq method which removesdirectional components of the dataset prior to the up-sampling by directional equalization. The analysis of theresults showed that the highest inaccuracies and devi-ations to the reference occur for rearward sound inci-dence. Due to constructive interferences of the sound ra-diated, a bright spot can be observed here. However, theinterference pattern changes rapidly for adjacent direc-

tions, especially towards higher frequencies correspond-ing to a high spatial order Nmax in the SH domain. Theevaluation revealed that for a human speaker already atNlow = 4 with 38 measured directions on a Lebedev grid,a decent full-spherical dense directivity set can be gen-erated, which might be sufficient for various applicationsin the field of virtual acoustics.

In subsequent research we will investigate to what extentthe results can be transfered to surrounding arrays, e.g.to a pentakis dodecahedron with 32 microphones. Sucharrays have been realized e.g. in [4, 5] and allow to mea-sure directivities at Nlow = 4 in real time. Thus, they arenot restricted to impulse-response based measurements,but allow for measuring time-variant influences of the di-rectivity pattern. Furthermore, it has to be investigatedto what extent the model of a sound source located ata considerably small mouth opening holds for a naturalhuman speaker. Here influences of sound radiation byglottis, nose and even towards lower frequencies of radi-ation by the human skull need to be considered.

The research presented in this paper has been carried outin the Research Project NarDasS funded by the FederalMinistry of Education and Research in Germany; Sup-port Code: BMBF 03FH014IX5-NarDasS. The authorsthank Raphael Gillioz for supporting the measurements.

References

[1] Brandner, M., Frank, M., and Rudrich, D., “DirPat -Database and Viewer of 2D/3D Directivity Patterns of SoundSources and Receivers,” in 144th Audio Eng. Soc. Conv., 2018.

[2] Kocon, P. and Monson, B. B., “Horizontal directivity pat-terns differ between vowels extracted from running speech,”J. Acoust. Soc. Am., 144(1), pp. EL7–EL12, 2018.

[3] Katz, B. and D’Alessandro, C., “Directivity measurements ofthe singing voice,” in Proceedings of the 19th InternationalCongress on Acoustics, 2007.

[4] Pollow, M., Directivity Patterns for Room Acoustical Mea-surements and Simulations, Logos Verlag Berlin GmbH, 2015.

[5] Arend, J. M., Stade, P., and Porschmann, C., “Binaural re-production of self-generated sound in virtual acoustic environ-ments,” in Proc. of the 173rd Meeting of the Acoust Soc. ofAm., 2017.

[6] Williams, E. G., Fourier Acoustics - Sound Radiation andNearfield Acoustical Holography, Academic Press, London,UK, 1999.

[7] Porschmann, C., Arend, J. M., and Brinkmann, F., “Direc-tional Equalization of Sparse Head-Related Transfer FunctionSets for Spatial Upsampling,” IEEE Transactions on Audio,Speech and Language Processing, in press, 2019.

[8] Marshall, A. H. and Meyer, J., “The directivity and auditoryimpressions of singers,” Acustica, 58, pp. 130–140, 1985.

[9] Algazi, V., Avendano, C., and Duda, R. O., “Estimation of aSpherical-Head Model from Anthropometry,” J. Audio Eng.Soc., 49(6), pp. 472 – 479, 2001.

[10] Bernschutz, B., Porschmann, C., Spors, S., and Weinzierl, S.,“Entwurf und Aufbau eines variablen spharischen Mikrofonar-rays fur Forschungsanwendungen in Raumakustik und VirtualAudio,” in Proc. of the 36th DAGA, pp. 717–718, 2010.

[11] Bernschutz, B., “A Spherical Far Field HRIR / HRTF Compi-lation of the Neumann KU 100,” in Proc. of the 39th DAGA,pp. 592–595, 2013.

DAGA 2019 Rostock

1461