Beyond Vaporware: Considerations for Meaningful Design with Smell

Abstract Smell is not only among our most marginalized senses, it is also an underrepresented mode of knowledge, communication and literacy. We ask how it is possible to develop meaningful interactions with smell without first examining the sources and influences of our own assumed knowledge and beliefs. Building on our team’s ongoing research of situated learning practices, communicating, and designing interactions with smell, we propose a set of considerations for intersensory technology design that focus on smelling in the ‘present tense.’ Our workshop contribution seeks to support interaction designers in critically and meaningfully working with smell through practices of self-reflexive inquiry.

Author Keywords Sensory interaction; intersensory; senses; smell; tangible and embodied interfaces; cultural artifacts.

ACM Classification Keywords H.5.1. [Information Interfaces and Presentation]: Multimedia Information Systems – Animations; H.5.2 [Information Interfaces and Presentation (e.g., HCI)]: User Interfaces – Haptic i/o, Input Devices and Strategies; Interaction styles, theory and methods; K.3.0 [Computers and Education]: General.

Melanie McBride York University Toronto, ON, Canada mcbride.melanie@gmail.com Daniel Harley Ryerson University Toronto, ON, Canada dharley@ryerson.ca

Ali Mazalek Ryerson University Toronto, ON, Canada mazalek@ryerson.ca Jason Nolan Ryerson University Toronto, ON, Canada jnolan@ryerson.ca

Introduction If, as Hans Moravec [11] declared, the senses “have no future,” it is because we have all but abandoned our sensory present. From this perspective, the neglect of smell from our techno-sensory order is a ‘systematic’ omission [3] and not an inevitability of ‘progress.’ Understanding the social and cultural life of smell is consequential for interaction designers, particularly those who may be newcomers to sensory studies or practices involving smell or taste.

The prospect of scent technology, not unlike the early public reception of sound recordings [6], elicits a similar blend of skepticism and bemusement. From Smell-O-Vision [13] to the Scentee [16], and OPhone [18], scent technologies have yet to become household names. With the exception of experimental developments in scent telemetry and wearable devices [1, 14], many scent technologies are little more than “precision air fresheners” [4] intended to trigger one-smell-fits-all hedonic responses, or immersive ambiances, independent of sociocultural, physiological, and psychological diversities. Few of these technologies have made it past the prototype stage, which is the very definition of ‘vaporware.’

We argue that the failure to ‘harness’ scent begins with faulty conceptual foundations, underwritten by universalist claims of human behavior, cognition, and perception, independent of sociocultural and ecological variables. As organic chemist-artists Agapakis and Tolaas [2] note, “a deeper understanding of odors and olfaction allows us to tolerate smells, to value their importance and their power, and to smell their nuanced and subjective beauty.”

This understanding, however, must begin with context, by design. To illustrate such an approach, we review our design practices for an olfactory interaction that grew from a graduate student project into a larger collaboration [9] between researchers in human-computer interaction, education, and media at the Responsive Ecologies Lab and the Synesthetic Media Lab at Ryerson University and Georgia Tech. Drawing on our experiences from this project and our respective research of smell and interaction, we propose a set of considerations inspired by social research methodologies.

Sensing Context: Smelling the Past through the Present Like so many artifacts typically presented behind glass, 16th century prayer-nuts (Figure 1) originate from a rich sensory context that is rendered inaccessible by purely visual or text-bound display formats. In an attempt to communicate neglected sensory and embodied information of the artifact, we created three interactive prototypes. Each prototype is a 3D-printed tangible object similar in size to the original artifact, with Arduino microcontrollers receiving the interaction input and delivering output in the form of graphics and sound with projections and speakers. Our olfactory interaction, ‘Scents of Power,’ is inspired by the aromatic compositions of 16th century prayer-nuts. As Scholten [17] writes, “A fragrant substance—perhaps a mixture of dough, herbs and dried flowers—would have been put in the empty space between the removable inner reliefs and the outer shell and, used as a rosary pendant, the prayer-nut would have released a sweet scent through the quatrefoil openings.”

Figure 1. The exterior (top) and the interior (bottom) of a 16th century prayer-nut (Image courtesy of the Metropolitan Museum of Art).

Given the neglect of smell in scholarly texts, we noted [7] that the historical documentation of prayer nuts contains little detail of the formulaic composition of particular scents other than a list of ‘possible’ ingredients. From our review of the literature on prayer-nuts, we translated lists of “sweet-smelling ingredients” such as lavender, rose petals, saffron, and cinnamon [5] into two custom scents using raw materials, such as spices, resins, and essential oils. The more complex of the two scents, composed of nutmeg, cinnamon, clove, frankincense, and rose was a composite of several descriptions. The second, a simple combination of lavender and cinnamon, was intended to highlight ingredients that might be easily identifiable to contemporary museum visitors. In each case, we compensated for the missing aroma of the original prayer-nut’s fragrant boxwood housing by adding sandalwood and cedarwood essential oils. Authors McBride and Nolan’s unsatisfactory experiences with ‘thematic’ aromas from commercial scent vendors inspired the choice to source local, low-cost aromas that could be sampled before purchasing.

For early user tests in our lab, we applied a trace amount of the composition to the inside of wine glasses, which were placed top down on a table to contain the scent and to encourage the relatively familiar practice of ‘nosing’ aromas. As novel technologies often require novel interactions, we hypothesized that these embodiments might be reminiscent of familiar practices that do not require the same level of scaffolding. In order to show that the scents would have been inserted into the prayer-nuts, subsequent user tests made use of removable wooden discs housed within the replica prototypes, which were affixed to a table. As a static and self-contained mode

of scent delivery, we did not encounter problems common to passive chemical displays, such as the sequencing, transitioning, control and removal of odor molecules [1].

When the prototype prayer-nut is opened, an animation appears with botanical imagery, overlapping the visuals to denote the blending of aromas, accompanied with descriptions of historical context. Following text prompts, the user is invited to smell the replicas, comparing and identifying the scents (Figure 2). Otherwise ephemeral dimensions of the prayer-nut’s aromatic contents are communicated to the user through the directly embodied experience of olfactory detection and discrimination [7]. Given that breathing is involuntary and we do not choose to inhale, our interaction privileges an active consensual delivery model rather than a passive, or, automated model of olfactory ‘immersion.’

Considerations for Meaningful Interactions with Smell Extending authors McBride and Nolan’s conceptual foundations for intersensory design with smell [10] and our experiences developing the prayer-nut prototypes, we propose the following considerations to help interaction designers account for the influence and consequences of our underlying beliefs and assumptions about smell and the senses.

1. What is your ‘sensory standpoint’? If we are to design meaningful and contextually responsive interactions with smell, we need to start by first accounting for our beliefs, assumptions and disciplinary perspectives of the senses. We refer to this consideration as a ‘sensory standpoint’ [10]. In

Figure 2. “Scents of Power” interaction (top) and animations (bottom)

practice, we ask: What systems of feeling, sensing, or interacting do our sensory paradigms or perspectives reinforce or negate (i.e., neuroscientific, anthropological, phenomenological, etc.)?

2. Sensory autoethnography As sensory ethnographers [15] suggest, researchers should first inquire into our own everyday and lived experiences with smell and the senses. As with the sensory standpoint, designers need to account for their own sensory ‘orders’ before creating objects or interactions for others. This consideration can be engaged either as a team project or as self-directed inquiry. For example: To what degree are you presently engaged in deliberate practices involving smell or the senses? What are some of the cultural or material sources that inform these practices (e.g., movies, television shows, books, products, etc.)?

3. Sensing diversity and inclusion Inclusive sensory interactions are those in which users recognize their own identities and lived experiences in the objects and encounters we design. This consideration asks: To what degree can participants/users contribute their own sensory needs or preferences to the design process? In the case of smell, have we considered differences in smell ability and impairments, such as anosmia, parosmia, etc.?

4. Situating smell as a whole body interaction To take a situated approach to designing with smell reflects not only on the total ecology in which smells are encountered but also their accompanying embodiments. Popular theories of embodiment often, ironically, neglect the actual, practical physicality of the body “within the environment” [8]. Considering the role

of ‘bodily states’ [10] in shaping sensory context we ask: To what degree have we considered the environmental contexts engaged when designing whole body interactions?

5. Scaffolding intersensory literacy and learning Designing for ‘intersensory learning inquiry’ [10] involves planning for ‘unknowable,’ rather than prescribed learning outcomes [12]. For example, encouraging olfactory learning and literacy through the empirical comparison of odors can be a means of establishing further historical or cultural context apart from affective ‘likes’ or ‘dislikes’ [2]. This principle asks how we can scaffold multiple entry-points for sensory inquiry that permit a variety of possible experiences and several levels of engagement, situated within the immediate ecology.

Conclusion and Implications Reflecting on these sociocultural, ecological and applied considerations, we conclude that a sensorial ‘call to context’ [10] is in order if we are to realize a meaningful and situated foundation for intersensory HCI that exceeds universalist or monolithic views of cognition, experience, and perception. Our considerations also highlight the value of transdisciplinary interventions from social research in particular, as a means of critically reflecting on the influence of our underlying beliefs and perspectives about the senses. In order to move beyond the empty promises of vaporware, we might start by learning how to smell.

Acknowledgements This research was undertaken, in part, thanks to funding from the Canada Research Chairs program,

Canadian Foundation for Innovation, and the Ontario Ministry of Research and Innovation. We also thank Jean Ho Chu and Jamie Kwan for their collaboration.

References 1. Yongsoon Choi, Rahul Parsani, Xavier Roman,

Anshul Vikram Pandey, and Adrian Cheok. 2012. Light perfume: designing a wearable lighting and olfactory accessory for empathic interactions. In Advances in Computer Entertainment (pp. 182-197). Springer, Berlin.

2. Christina M. Agapakis and Sissel Tolaas. 2012. Smelling in multiple dimensions. Current opinion in chemical biology, 16, 5: 569-575.

3. Constance Classen, David Howes, and Anthony Synnott. 1994. Aroma: The Cultural History of Smell. Routledge, London.

4. Jim Drobnick. (Ed.) 2006. The Smell Culture Reader. New York, Berg, 348.

5. Reindert Falkenburg and Frits Scholten. 1999. A Sense of Heaven: 16th Century Boxwood Carvings for Private Devotion. Leeds (The Henry Moore Institute).

6. Lisa Gitelman. 2006. Always already new. Cambridge, MA, MIT Press.

7. Daniel Harley, Melanie McBride, Jean Ho Chu, Jamie Kwan, Jason Nolan and Ali Mazalek. 2016. "Sensing context: Reflexive design principles for intersensory museum interactions." MW2016: Museums and the Web 2016.

8. Victoria Henshaw. 2013. Urban Smellscapes: Understanding and Designing City Smell Environments. London, Routledge, 28.

9. Jamie Kwan, Jean Ho Chu, Daniel Harley, Melanie McBride, & Ali Mazalek. Grasping Cultural Context with Multisensory Interactions. Extended Abstract

forthcoming in the Tenth International Conference on Tangible, Embedded, and Embodied Interaction, ACM.

10. Melanie McBride and Jason Nolan. In-press, expected 2016. Including Smell: An intersensory curriculum, by design. In Design With Smell. Victoria Henshaw (Ed.). New York, Routledge.

11. Hans Moravec. 1997. ‘The Senses Have No Future,’ Retrieved January 12, 2016 from http://www.frc.ri.cmu.edu/~hpm/project.archive/general.articles/1997/970128.nosense.html

12. Jason Nolan and Melanie McBride. 2014. Beyond gamification: reconceptualizing game-based learning in early childhood environments. Information, Communication & Society, 17, 5: 594-608.

13. Mark W. Paterson. 2006. Digital scratch and virtual sniff. In The Smell Culture Reader, Jim Drobnick (Ed.). Oxford, Berg, 358-367.

14. Nimesha Ranasinghe, Adrian Cheok, and Ryohei Nakatsu. 2012. Taste/ip: the sensation of taste for digital communication. In Proceedings of the 14th ACM international conference on Multimodal interaction (pp. 409-416). ACM.

15. Sarah Pink. 2009. Doing sensory ethnography. London, Sage.

16. Scentee US. Company Website. Retrieved February 1, 2016 from http://scenteeusa.com

17. Frits Scholten. 2011. A Prayer-nut for Francois Du Puy. Burlington Magazine, 153, 1300: 447–451.

18. Liz Stinson. 2015. This New App Wants to be the iTunes of Smells. Retrieved January 12, 2016 from http://www.wired.com/2015/04/ophone-onotes-itune-of-smell/