NO STRINGS ATTACHED: FORCE AND VIBROTACTILE FEEDBACK INA GUITAR SIMULATION

Andrea PassalentiUniversity of Udine

passalenti.andrea@spes.uniud.it

Razvan PaisaAalborg University

rpa@create.aau.dk

Niels C. NilssonAalborg University

ncn@create.aau.dk

Nikolaj S. AnderssonAalborg University

nsa@create.aau.dk

Federico FontanaUniversity of Udine

federico.fontana@uniud.it

Rolf NordahlAalborg University

rn@create.aau.dk

Stefania SerafinAalborg University

sts@create.aau.dk

ABSTRACT

In this paper we propose a multisensory simulation of pluck-ing guitar strings in virtual reality. The auditory feedback is generated by a physics-based simulation of guitar strings, and haptic feedback is provided by a combination of high fidelity vibrotactile actuators and a Phantom Omni haptic device. Moreover, we present a user study (n=29) explor-ing the perceived realism of the simulation and the rela-tive importance of force and vibrotactile feedback for cre-ating a realistic experience of plucking virtual strings. The study compares four conditions: no haptic feedback, vi-brotactile feedback, force feedback, and a combination of force and vibrotactile feedback. The results indicate that the combination of vibrotactile and force feedback elic-its the most realistic experience, and during this condition, the participants were less likely to inadvertently hit strings after the intended string had been plucked. Notably, no statistically significant differences were found between the conditions involving either vibrotactile or force feedback, which points towards an indication that haptic feedback is important but does not need to be high fidelity in order to enhance the quality of the experience.

1. INTRODUCTION

In recent years, the availability of relatively low cost vir-tual reality (VR) hardware devices has seen applications also in the music industry. Several VR musical instruments have been developed both in the academic and commercial world. An overview of design guidelines and applications of VR musical instruments can be found in [1].

In the computer music community, the sounds of stringed instrument have been simulated for decades, starting with the work of Hiller and Ruiz [2], followed a decade later

Copyright: c© 2019 Andrea Passalenti et al. This is

an open-access article distributed under the terms of the

Creative Commons Attribution 3.0 Unported License, which permits unre-

stricted use, distribution, and reproduction in any medium, provided the original

author and source are credited.

by the simulations proposed by Karplus and Strong [3].Moreover, physics-based simulation of such instrumentshas been an active areas of research within the community(see e.g. [4, 5]). However, this research has predominantlyfocused on simulating the sounds generated during inter-action with strings, and visual and haptic feedback remainrelatively unexplored (for a recent exception see [6]).

In this paper we propose a novel multisensory simula-tion of a guitar, which uses efficient yet accurate physics-based synthesis techniques to reproduce the auditory andhaptic feedback accompanying the act of plucking guitarstrings. In the current paper, we use the term haptic in abroad sense to encompass all somatosensory capabilities;that is, sensations that qualify as cutaneous (related to in-teractions at the level of the skin), kinesthetic (related tomovements of one’s body and limbs), and proprioceptive(related to the position of limbs and the static attitude ofthe musculature) [7, 8].

We first describe the system, which simulates the sensa-tion of plucking guitar strings through a combination ofvisual, auditory, force and vibrotactile feedback. Subse-quently, we present a user study evaluating users’ experi-ence of plucking virtual strings. The aim of the study wastwofold: (1) It was meant to determine the degree of per-ceived realism of the system; that is the degree to whichthe system was able to replicate the sensation of pluckinga real guitar string. (2) The aim was to explore the rela-tive importance of force and vibrotactile feedback as ele-ments for the creation of a realistic experience of pluckingvirtual strings. Specifically, it was considered relevant todetermine if a realistic experience can be elicited when thesimulation is devoid of force feedback and only involvesvibrotactile feedback.

2. RELATED WORK

In the last two decades, haptic feedback has received in-creasing interest from the sound and music computing com-munity, due to the strong correlation between auditory andhaptic musical signals. In fact, both signals share highlysimilar and simultaneous temporal analogies, with a higher

sampling rate for the audio channel. For a recent overviewrefer to the work of Papetti and Saitis [9]. Avanzini andcolleagues [6,10] describe a multimodal architecture, whichintegrates physically-based audio and haptic models withvisual rendering. Experiments with stiffness perceptionshowed how auditory feedback can modulate tactile per-ception of stiffness. In her PhD dissertation investigat-ing the role of haptic feedback in digital musical instru-ments, O’Modhrain [11] pioneered research on multisen-sory audio-haptic simulations in a musical context. Asan example, she discovered that the playability of touch-free instruments, such as the Theremin, is significantly in-creased when haptic feedback is provided. Custom madedevices to provide haptic feedback in a musical contexthave been developed together with physics-based audio-visual simulations in the work of Florens and colleagues[12]. More recently, Leonard and Cadoz [13] introduceda system, based on mass-interaction physical modelling,that supports real-time interaction with multisensory vir-tual music instruments. The combination of physicallysimulated strings and haptic feedback has also been ratherextensively explored by Berdahl [14]. In this context, theapplications have been mostly pedagogical and artistic, ratherthan targeted towards perceptual evaluations of auditory-haptic interactions. Berdahl proposes novel fader-basedcontrollers where the plucking action can be felt while in-teracting with a virtual string [15]. The tight multisen-sory coupling between hearing and touch has also been ex-plored in the simulations proposed by Liu and Ando [16].

In the context of VR, although research has primarily fo-cused on providing realistic auditory and visual feedback,haptic feedback has also been investigated. However, thefocus has to a large extent been on cumbersome and ex-pensive force feedback devices [17].

Kinesthetic and proprioceptive information is central toperception of solid objects, but so is cutaneous informationderived from vibrations. The vibrations generated whenan object moves across a surfaces encodes roughness [18]and the vibrations generated during tapping encodes hard-

Figure 1. A user interacting with the system. On the topright a zoomed figure of the Phantom Omni device withattached the vibrotactile actuator. The actuator is insertedinto a 3D printed plectrum.

ness [19]. For these reasons, much research has focusedon increasing the realism of virtual interaction using vi-brations [20]. For example, it has been shown that theaddition of vibrotactile feedback can enhance low-fidelitykinesthetic devices [19], vibrotactile feedback affect per-ceived hardness during tapping of one physical object onanother [21], and vibrotactile feedback can be used to elicitan illusion of compliance when pressing a stylus against arigid surface [22].

3. MULTISENSORY SIMULATION OF PLUCKINGGUITAR STRINGS

The system proposed in the current paper is created withthe intention of eliciting a realistic sensation of pluckingthe strings of a virtual guitar. The system consists of threeseparate stimuli elements: haptic, auditory and visual. Fig-ure 2 shows a diagram visualizing how the elements havebeen connected, and Figure 1 shows a user interacting withthe system.

Specifically, the haptic feedback is provided by a Sens-able Phantom Omni haptic device mounting a 3D printedplectrum on its arm tip. The plectrum is embedded witha Haptuator Mark II vibrotactile actuator manufactured byTactile Labs. Visual feedback is delivered using an OculusRift CV1 head mounted display. Finally, auditory feedbackis provided through a Vox HC30 guitar amplifier.

The signal driving the vibrotactile actuator is producedby an impact model run by the Sound Design Toolkit forMax/MSP [23]. Specifically, the contact force depends onthe velocity and displacement of the objects in contact ac-cording to the following relationship:

f(x(t), v(t)) = kx(t)α + λx(t)αv(t)

for x ≥ 0, and 0 otherwise, where the compression x atthe contact point is defined as the differences between the

Figure 2. Diagram visualizing the software and hardwareintegration.

displacements of the two bodies, and v(t) is the compres-sion velocity. The parameter k is the force stiffness and isa function of the mechanical properties of the two bodies,while λ is the force damping weight, and α is a parameterwhose value depends on the geometry of the contact [6].

The parameters of the impact model were chosen by hav-ing two guitar players empirically experiment with differ-ent settings and choosing the parameters that felt closest toa guitar pluck. For the force feedback, the Phantom Omnihaptic device is driven by a collision algorithm developedin the haptic plugin for Unity [24]. This plugin uses theOpen Haptics toolkit [25] to render the contact force be-tween the pen of the Phantom Omni and the virtual string.

The guitar sound is synthesized by an efficient extendedKarplus-Stong algorithm [26], originally developed by KevinKarplus and Alex Strong in 1983 [3]. The algorithm sim-ulates a string in the form of a feedback digital delay whoselength represents the length of the string. Propagation lossesare simulated using a low-pass filter. Jaffe and Smith [26]proposed improvements towards a realistic guitar sound.Although this simulation is a simplification compared toaccurate physical simulations, it is efficient enough to runin real time with minimum CPU load. This fact is es-pecially important in VR applications. In order to fullysimulate the sound of an electric guitar the output of theKarplus-Stong algorithm was passed through a WamplerSLOstortion high gain drive pedal before being played backthrough a VOX HC-30 guitar combo amplifier. The pedalwas set on overdrive mode with parameters matching thesound of the real guitar present in the training session, thatwas connected to the same amplifier, but on a differentchannel.

The visual stimuli presented an electric guitar which wascreated using the Unity 3D and displayed using an OculusRift CV1 head mounted display.

4. METHOD AND MATERIALS

As suggested, the aim of the study was to (1) evaluate theperceived realism of the proposed system and (2) to ex-plore the relative importance of force and vibrotactile feed-back as elements for the creation of a realistic experienceof plucking strings while interacting with a virtual guitar.To meet this aim, we performed a within-subjects studycomparing four conditions that varied in terms of the hap-tic feedback provided when users plucked virtual strings:no haptic feedback (N), vibrotactile feedback (V), forcefeedback (F), and a combination of force and vibrotactilefeedback (FV).

4.1 Participants

A total of 29 participants (26 male, 3 female) aged between19-44 years (M=28.2 years, SD=7.0) took part to the study.All participants were faculty or students at Aalborg Uni-versity Copenhagen. On average, the participants had 8.2years (SD=8.3) of regular, weekly practice playing a mu-sic instrument, they played 2.4 hours (SD=2.6) each week,and 21 participants reported being able to play one or more

string instruments. All participants gave written informedconsent prior to participation.

4.2 Procedure and Task

Initially the participants completed a questionnaire cover-ing demographic information (i.e., age, gender, occupa-tion, and musical experience). They were then introducedto the setup and task. They were informed that the studywas exploring the perceived realism of virtual strings andwere instructed to pay particular attention to the haptic sen-sations experienced during each condition. No informationwas provided about the variations in feedback across con-ditions).

Because the aim of the study was to explore changes inrealism across the four conditions, the participants wereasked to pluck the strings of a real guitar before exposureto the first condition. They were instructed to pluck all sixstrings and were allowed to do so for no more than threeminutes. It was made explicit to the participants that thistask was meant as a baseline for comparison during thefour conditions, and that they should pay attention to thesensation of touching the real strings, including sense ofstiffness (i.e., the strings resistance to deformation).

During each condition, the participants were required topluck each of the six strings twice in randomized order.The string the participants should pluck was visually high-lighted. Subsequently, the participants were asked to freelyinteract with the virtual strings and they were encouragedto both pluck individual strings and perform strumminginteractions. After exposure to each condition the partic-ipants were required to fill out a questionnaire related totheir experience (see Section 4.3).

The participants were exposed to the four conditions inrandomized order, and the study lasted for approximately20 minutes in total.

4.3 Measures

Because the primary aim of the study was to determinehow realistic the participants found the four conditions,we primarily relied on self-reported measures. Specifi-cally, after exposure to each condition the participants wereasked to fill out a questionnaire including eight items re-lated to their experience of interacting with the virtual strings.The eight items can broadly be divided into four categories:

• Perceptual similarity: Two items required the par-ticipants to explicitly compare the real and virtualstrings in terms of (a) overall similarity and (b) stiff-ness.

• Perceived realism: Three items asked the partici-pants to evaluate (c) the overall experience of real-ism, (d) the sensation of touching physical strings,and (e) the sensation of hearing physical strings.

• Perceived thickness: Two items asked the partici-pants evaluate (f) the connection between the thick-ness of the virtual strings and the sensation of touch-ing them, and (g) the connection between the thick-ness of the virtual strings and sounds they generated.

Table 1. The eight questionnaire items and corresponding anchors of the 7-point (1-7) rating scales.Questionnaire items: Scale anchors:Perceptual similarity:(a) The sensation of touching the virtual and real strings was: Completely different / Identical(b) Compared to the real strings, the stiffness of the virtual strings was: Much lower / Much higherPerceived realism:(c) I found the experience of interacting with the virtual guitar realistic. Strongly disagree / Strongly agree(d) It felt as if I was touching physical strings. Strongly disagree / Strongly agree(e) I felt like I was hearing physical strings. Strongly disagree / Strongly agreePerceived thickness:(f) It felt as if there was a connection between the thickness of the strings and how they felt. Strongly disagree / Strongly agree(g) It felt as if there was a connection between the thickness of the strings and the sounds. Strongly disagree / Strongly agreePerceived ease of use:(h) I found it easy to pluck the strings of the virtual guitar. Strongly disagree / Strongly agree

• Perceived ease of use: Finally, one item (h) askedthe participants to evaluate how easy they found it tointeract with the virtual guitar.

All eight questions were answer using 7-point rating scales,ranging from 1 to 7. Table 1 presents the eight questionsand the corresponding scale anchors.

In addition to the questionnaire administered after eachfour conditions, we also asked the participants to indicatewhich of the four the preferred once they had tried themall. Moreover, the participants were encouraged to explaintheir preference.

Finally, to determine whether the addition of more hap-tic feedback would positively affect the participants abilityto pluck the virtual strings, we logged the number of erro-neously plucked strings during the part of each trial wherethe participants had to pluck predefined strings. Impactsbetween the virtual plecturm and strings were considerederrors, if they occurred after the correct string had beenplucked and before a new string was highlighted. We delib-erately ignored errors made prior to the participants pluck-ing the correct string, as these errors were more likely toresult from visual misperception. That is, errors occurringprior to initial contact with a highlighted string were likelythe result of incorrect visuomotor coordination, rather thanthe sensation of the haptic stimuli itself. Conversely, er-rors occurring after a highlighted string had been pluckedcould be the result of an inability to perceive the hapticstimuli produced while plucking.

5. RESULTS

This section presents the results obtained from the self-reported measures pertaining to the participants’ experi-ence and the behavioral measure related to the number oferrors performed during exposure to each condition.

5.1 Self-reported measures

The data obtained from the eight questionnaire items weretreated as ordinal and analyzed using Friedman tests. Whenstatistically significant differences were found pairwise com-parisons using Dunn-Bonferroni tests were performed.

Perceptual similarity: A statistically significant differ-ence was found in relation to overall perceptual similar-

ity (X2(3) = 15.152, p = .002) and the pairwise com-parisons identified a statistically significant difference be-tween N and F (p = .031), and between N and FV (p =.004). In both cases N yielded significantly lower scores(Figure 3a).

Similarly, a statistically significant difference was identi-fied in regard to stiffness relative to physical strings (X2(3) =17.831, p < .001), and the pairwise comparisons foundbetween N and F (p = .003), and between N and FV(p = .007). N yielded significantly lower scores (Figure3b). Note that both F and FV had a median score of 4,suggesting that the two conditions may have provided thegreatest resemblance with the physical string in terms ofstiffness.

Perceived realism: A statistically significant differencewas found between the scores related to overall realism(X2(3) = 11.757, p = .008), and the pairwise compar-isons indicated that the participants scored FV significantlyhigher than N (p = .026), as apparent from Figure 3c.

The statistical comparison also indicated that the scoresdiffered significantly with respect to the participants’ sen-sation of touching physical strings (X2(3) = 18.253, p =.005), and the pairwise comparisons indicated significantdifferences between N and F (p = .008), and between Nand FV (p = .003). Again, N yielded significantly lowerscores than F and FV (Figure 3d). As indicated by Fig-ure 3e, no significant difference was found in relation tothe item pertaining to the participants’ sensation of hear-ing physical strings (X2(3) = 1.159, p = .763).

Perceived thickness: A statistically significant differ-ence was found between the scores related to the perceivedconnection between string thickness and touch (X2(3) =12.641, p = .005), and the pairwise comparisons suggestthat participants rated FV significantly higher than N (p =.019), as apparent from Figure 3f. No significant differ-ence was found with respect to the item related to per-ceived connection between the thickness and the producedsound (X2(3) = 5.260, p = .154).

Perceived ease of use: No signficant difference wasfound between the scores related to percevieved ease ofuse ((X2(3) = 2.026, p = .567)), which are summarizedin Figure 3h.

Preference rating: When asked to select their preferredcondition technique 41.4% (12 participants) chose FV, 37.9%

Figure 3. Boxplots visualizing the results related to the eight questionnaire items in terms of medians, interquartile ranges,minimum and maximum ratings, and outliers.

(11 participants) chose F, 10.3% (3 participants) chose V,3.4% (1 participants) chose N, and 6.9% (2 participants)had no preference. A Cochrans Q test was run to determineif the percentages of participants who chose each condi-tion differed. Sample size was adequate to use the χ2-distribution approximation. The Cochrans Q test suggestedthat the difference was statistically significant (χ2(4) =19.103, p = .001). Pairwise comparisons using Dunn-Bonferroni tests revealed significant differences betweenFV and N (p = .012), and between F and N (p = .033).

5.2 Number of Errors

One participant was excluded from this analysis becausethe data obtained during one of the four conditions wascorrupted. The results of the behavioural measure of thenumber of erroneously plucked strings was treated as in-terval data. However, the data did not meet the assumptionof normality, as assessed by Shapiro-Wilk test (p > .05),and significant outliers were identified, as apparent fromthe boxplot in Figure 4. Thus, the data was analyzsed us-ing non-parametric methods.A Friedman test indicated thatnumber of errors differed signficantly between conditions(X2(3) = 11.170, p = .011) and pairwise comparisonsusing Dunn-Bonferroni tests indicated that FV yielded sig-nificantly fewer errors than N (p = .047).

Figure 4. Boxplots visualizing the results related numberof errors items in terms of medians, interquartile ranges,minimum and maximum ratings, and outliers.

6. DISCUSSION

The results related to overall perceptual similarity suggestthat the sensation of plucking the virtual strings resembledits real world counterpart the most, when the simulationinvolved force feedback (i.e., both F and FV were sig-nificantly different from N). Based on the distribution ofratings (Figure 3a) it is apparent that some of the partici-pants rated FV higher than F. However, no statistically sig-nificant difference between the two conditions was found.Moreover, the distributions of scores were relatively simi-lar for F and V. It should be stressed that the median scorefor FV only was 4, suggesting that the participants did notexperience a high degree of perceptual similarity. Futurestudies are necessary to determine if more elaborate stringsynthesis models yield more convincing results or if thescores can be attributed to limitations of the haptic render-ing.

The conditions involving force feedback also providedthe best match to the real guitar strings in terms of stiff-ness relative to physical strings (Figure 3b). That is, bothF and FV had a median scores of 4, which indicates that thestiffness of the stings was not perceived as much higher ormuch lower than the stiffness of the real guitar strings.

The scores pertaining to overall realism (Figure 3c) in-dicate that the participants found the experience to be themost realistic when exposed to FV (FV was the only con-dition that differed significantly from N). Moreover, whenthe participants were asked about the degree to which theyhad a sensation of touching physical strings (Figure 3d),both F and FV yielded the highest median scores (both dif-fered significantly from N).

The four conditions yielded largely identical and rela-tively high scores in relation to the self-reported sensationof hearing physical strings (Figure 3e). We take this tomean that most participants felt that the auditory feedbacksounded as if it was generated by a physical string ratherthan an algorithm. It is hardly surprising that no differencewas found between the four conditions, because the sameauditory feedback was used across all conditions.

Based on the questionnaire item related to the perceivedconnection between the thickness and touch (Figure 3f),it would appear that the participants may have perceivedthe virtual strings as having different thicknesses when ex-posed to FV. Even though a significant difference only wasfound between FV and N, it is worth noting that FV is theonly condition that yielded a median score higher than 4.

The results related to the connection between thicknessand sound (Figure 3g), indicate that the participants tosome extent experienced a connection between the thick-ness of the strings and the sound that was produced whenthey were plucked. However, no significant differences be-tween conditions were found. Moreover, the spread of thescores was relatively large with respect to N and V. It ispossible to offer at least two possible explanations for thelarge spread. That is, it is possible that the phrasing ofthe question prompted some participants to compare thesound to the visual appearance of the strings, while othermay have compared the sound the haptic sensation of thestrings. For that reason, we are reluctant to draw any con-clusions from these results.

The finding that the simulations involving force feedbackprovided the most compelling experience is corroboratedby the preference ratings and the associated qualitative feed-back. That is, the majority (23/29) of the participants pre-ferred the two conditions involving force feedback, but analmost equal number of participants preferred FV (12/29)and F (11/29). Notably, 4 out of the 11 participants whopreferred F, explicitly stated that they chose F over FV be-cause the vibration had been too strong. Of the 11 partic-ipants who preferred FV, 7 participants remarked that thevibration either made the haptic sensation of plucking thestrings more realistic or added to the sense that the frictionvaried. Thus, the participants were somewhat conflictedabout the contribution of the vibrotactile feedback, sug-gesting the need for future studies exploring variations invibration intensity.

No differences were found in relation to perceived ease ofuse (Figure 3h). However, we did observe a significant dif-ference with respect to the number of erroneously pluggedstrings after the correct string had been plucked (Figure4); namely the participants plucked significantly fewer er-rors during FV compared to N. Moreover, even though nosignificant differences were found between V and the otherconditions, it is worth noting that V appears to have yieldedfewer errors than both N and F. In other words, the twoconditions devoid of vibrotactile feedback resulted in thehighest number of errors. It is possible that the added vi-brations made the impact between the virtual plectrum andstring more salient, and thus causing the participants to re-tract their hands more swiftly.

7. CONCLUSION

In this paper we proposed a system that allows users topluck virtual guitar strings while receiving multisensoryfeedback in response to this interaction. The system wasevaluated in a user study exploring the perceived realism ofthe simulation and the relative importance of force and vi-brotactile feedback. The results indicate that the two con-

ditions involving force feedback provide the highest degreeof perceptual similiarlity to real guitar strings. While nosignificant differences were found between the two condi-tions, the condition including both force and vibrotactilefeedback yielded scores indicating that it was the best atmimicking interaction with real guitar strings. The self-reported measures related to overall realism yielded simi-lar indications. However, the participants scored the twoapproaches similarly when they were asked to what extentthey felt that they touched real strings. The absence ofsignificant differences between the three haptic conditions,makes it uncertain whether force or vibrotactile is the mostimportant for a realistic experience. Nevertheless, judg-ing by the distribution of scores and the preference ratings,force feedback appears to be central to the participants’ ex-perience of realism, and we suspect that the combinationof force and vibrotactile feedback may serve as the bestproxy for physical strings. It is encouraging that the vibro-tactile condition generally scored higher than the conditiondevoid of any haptic feedback. However, future studies in-volving a wider range of vibrotactile feedback are neces-sary in order to determine if vibrotactile feedback will suf-fice in and of itself. Particularly, it is necessary to comparevariations in the algorithm rather than just comparing thepresence and absence of vibrotactile feedback. Finally, nodifferences were observed with respect to perceived ease ofuse, but the behavioral measure provides some indicationthat vibrotactile feedback may decrease the risk of acci-dentally hitting strings after the intended string has beenplucked. Thus, even if vibrotactile feedback may be lessimportant than force feedback with respect to perceived re-alism, it is possible that it can help reduce the number ofincorrectly plucked strings.

8. REFERENCES

[1] S. Serafin, C. Erkut, J. Kojs, N. C. Nilsson, and R. Nor-dahl, “Virtual reality musical instruments: State of theart, design principles, and future directions,” ComputerMusic Journal, vol. 40, no. 3, pp. 22–40, 2016.

[2] L. Hiller and P. Ruiz, “Synthesizing musical sounds bysolving the wave equation for vibrating objects: Part2,” Journal of the Audio Engineering Society, vol. 19,no. 7, pp. 542–551, 1971.

[3] K. Karplus and A. Strong, “Digital synthesis ofplucked-string and drum timbres,” Computer MusicJournal, vol. 7, no. 2, pp. 43–55, 1983.

[4] C. Erkut, V. Valimaki, M. Karjalainen, and M. Laur-son, “Extraction of physical and expressive parame-ters for model-based sound synthesis of the classicalguitar,” in Audio Engineering Society Convention 108.Audio Engineering Society, 2000.

[5] G. Derveaux, A. Chaigne, P. Joly, and E. Becache,“Time-domain simulation of a guitar: Model andmethod,” The Journal of the Acoustical Society ofAmerica, vol. 114, no. 6, pp. 3368–3383, 2003.

[6] F. Avanzini and P. Crosato, “Haptic-auditory render-ing and perception of contact stiffness,” in Interna-tional Workshop on Haptic and Audio Interaction De-sign. Springer, 2006, pp. 24–35.

[7] G. Robles-De-La-Torre, “The importance of the senseof touch in virtual and real environments,” Ieee Multi-media, vol. 13, no. 3, pp. 24–30, 2006.

[8] D. Waller and E. Hodgson, “Sensory contributions tospatial knowledge of real and virtual environments,” inHuman Walking in Virtual Environments, F. Steinicke,Y. Visell, J. Campos, and A. Lecuyer, Eds. Springer,2013, pp. 3–26.

[9] S. Papetti and C. Saitis, “Musical haptics: Introduc-tion,” in Musical Haptics. Springer, 2018, pp. 1–7.

[10] F. Avanzini and P. Crosato, “Integrating physicallybased sound models in a multimodal rendering ar-chitecture,” Computer Animation and Virtual Worlds,vol. 17, no. 3-4, pp. 411–419, 2006.

[11] S. OModhrain and C. Chafe, “Incorporating hap-tic feedback into interfaces for music applications,”in Proceedings of the International Symposium onRobotics with Applications, World Automation Confer-ence, 2000.

[12] J.-L. Florens, “Expressive bowing on a virtualstring instrument,” in International Gesture Workshop.Springer, 2003, pp. 487–496.

[13] J. Leonard and C. Cadoz, “Physical modelling con-cepts for a collection of multisensory virtual musicalinstruments,” in New Interfaces for Musical Expression2015, 2015, pp. 150–155.

[14] E. Berdahl and J. O. Smith III, “A tangible virtual vi-brating string,” in Proceedings of the 2008 Conferenceon New Interfaces for Musical Expression (NIME08),2008.

[15] E. J. Berdahl, Applications of feedback control to mu-sical instrument design. Stanford University, 2010.

[16] J. Liu and H. Ando, “Hearing how you touch: Real-time synthesis of contact sounds for multisensory in-teraction,” in Human System Interactions, 2008 Con-ference on. IEEE, 2008, pp. 275–280.

[17] G. C. Burdea, “Force and touch feedback for virtualreality,” 1996.

[18] S. J. Lederman, R. L. Klatzky, C. L. Hamilton, andG. I. Ramsay, “Perceiving surface roughness via arigid probe: Effects of exploration speed and mode oftouch,” 1999.

[19] K. J. Kuchenbecker, J. Fiene, and G. Niemeyer, “Im-proving contact realism through event-based hapticfeedback,” IEEE transactions on visualization andcomputer graphics, vol. 12, no. 2, pp. 219–230, 2006.

[20] H. Culbertson, S. B. Schorr, and A. M. Okamura,“Haptics: The present and future of artificial touch sen-sation,” Annual Review of Control, Robotics, and Au-tonomous Systems, vol. 1, pp. 385–409, 2018.

[21] T. Hachisu, M. Sato, S. Fukushima, and H. Kajimoto,“Augmentation of material property by modulating vi-bration resulting from tapping,” in International Con-ference on Human Haptic Sensing and Touch EnabledComputer Applications. Springer, 2012, pp. 173–180.

[22] J. Kildal, “3d-press: haptic illusion of compliancewhen pressing on a rigid surface,” in InternationalConference on Multimodal Interfaces and the Work-shop on Machine Learning for Multimodal Interaction.ACM, 2010, p. 21.

[23] S. D. Monache, P. Polotti, and D. Rocchesso, “A toolkitfor explorations in sonic interaction design,” in Pro-ceedings of the 5th audio mostly conference: a confer-ence on interaction with sound. ACM, 2010, p. 1.

[24] M. Poyade, M. Kargas, and V. Portela, “Haptic plug-in for unity,” Digital Design Studio (DDS), GlasgowSchool of Art, Glasgow, United Kingdom. https://core.ac. uk/download/pdf/28875009. pdf, 2014.

[25] B. Itkowitz, J. Handley, and W. Zhu, “The openhap-tics/spl trade/toolkit: a library for adding 3d touch/spltrade/navigation and haptics to graphics applications,”in Eurohaptics Conference, 2005 and Symposium onHaptic Interfaces for Virtual Environment and Teleop-erator Systems, 2005. World Haptics 2005. First Joint.IEEE, 2005, pp. 590–591.

[26] D. A. Jaffe and J. O. Smith, “Extensions of thekarplus-strong plucked-string algorithm,” ComputerMusic Journal, vol. 7, no. 2, pp. 56–69, 1983.